That's a common Classical Liberal ("Libertarian") belief. Liberty means I should be able to do whatever I want if it doesn't directly harm you. The problem is, what happens when every baker in the state refuses to bake a cake for gay couples? Then, if you're gay, you simply can't get a cake. Or a haircut. Or clothes. Or a job. That's exactly what happened in the US South, to black people. Other white Southerners didn't call them assholes – they praised them. That's why the US has anti-discrimination laws, and why they're necessary. I have a friend who's mostly Classical Liberal, but uses this analogy: you own a house on some land. I purchase all the land surrounding yours and build hundred-foot walls around it. I've committed no violence against you, per strict Classical Liberal theory. Moral: there is such a thing as economic violence.In my opinion, yes I should be able to refuse to make a cake for a gay couple, but then I shouldn't be surprised when people start calling me an asshole