You are two weeks younger than I am, precisely. What are your options? If you went, where would you go to? Conferences are really important in grad school. It's very unfortunate your boss doesn't see their value. There are some problems you are likely to face anywhere - unsocial colleagues, for example. There are others, like the conferences, that you really shouldn't face in your type of job typically. In short, I wouldn't encourage you to leave a job just because your colleagues aren't very talkative or similar to you, (since you'd just trade the devil you knew for the devil you didn't in such a move) but I would encourage you to leave if you aren't being given the opportunities & respect you both deserve and should generally receive elsewhere.
If I left, I could go anywhere, really. Apply for PhD-positions around Germany or Europe. This conference thing is really bugging me. Specially because we don't have any contact to other researchers. And I don't see anyone of the students take any of the classes provided by the graduate school...
I've heard of poisonous grad school programs. I certainly would say to keep your eyes open and be aware of the situation as it progresses. I want to ask if this guy who's against you guys going to conferences appears to be leading any sort of cult of personality at the school. I don't think it's conducive to a good education to close off from others in your field - in fact, the opposite, really. He seems like he might be a bit egotistical (keeping his techniques to himself so others can't do as well) and wants your unswerving attention (afraid others will 'steal' away 'his' students/assistants/what-have-you). By keeping you from conferences, etc, he's also potentially preventing you from developing a name and reputation for yourself which you could do via networking, presentations, etc, at such venues. He is establishing himself as your only/main educational resource, and denying you the ability to pursue others.