The assumption that having something to hide means your doing something wrong isn't accurate anymore. Sure, a lot of shadowy stuff goes on behind VPNs or TOR. On the other hand, people may want to hide searches regarding sexual orientation or fetishes, though all within the bounds of the law. Or, someone may want to hide research for a large purchase from Amazon and Google. There are also lifesaving resources like Erowid that most people probably don't want immediately connected to them. Having something to hide doesn't automatically mean ypunare doing something wrong.
So what I can understand from your comment is that it basically is a matter of right to privacy. It isn't so much about what you are looking at or doing, but that you can do it with anonymity. Am I getting close to the general idea?
Yeah, pretty much spot on. One argument I somewhat like (it's got its flaws of course) is that even if you agree wholeheartedly with the government, you still benefit from the freedom of speech. There is also a bit more conspiracy minded argument that you can never know what the government (or any group with access) might turn your data into or use it for at a later point. Dates can be changed, passwords remembered, and photos smudged to turn a harmless activity into a felony. But like I said, that's a bit far.