I do, yes, and that Google grants that permission relatively (to some others companies) freely is a small part of why I work there. I could quit and work on whatever I want, but eventually I'd need to worry about how to pay my mortgage. I could also go work for a smaller company without such a clause in their contract, but part of what I like about my job is the resources available to tackle problems, and I'd likely lose some of that.

Employment often imposes lots of restrictions on the actions we can take. I'm fortunate enough to have some choice in the set of restrictions I have to live with, so for me this particular restriction is just part of the deal.