I know exactly how you feel, man. I've never been to a party like that for the same reasons. Also I've never been invited to one.
I'm not sure if I'm introverted or anti-social, and I'm not sure what the difference is to be honest. I feel really uncomfortable even in a group with four or five friends, so big crowds of strangers are definitely a thing I try to avoid. It's cool that you made yourself go, even if you had a terrible time and left early. I would not have had the confidence to show up in the first place.
If it helps at all, I just got back from an event with a similar crowd... I don't know if you've ever heard of the band Slightly Stoopid, but they're some white dudes with hilariously long shorts who play reggae. I went to their concert (my dad is into them and didn't have anyone to go with) and it was pretty awkward. Frat boys, rich suburban girls, and one black guy - then me, whatever generalization I fall under. Even though the purpose of everyone being there was to see a band rather than socialize, I felt excluded from whatever was happening, like they were all experiencing this show together and I was the outsider trying to blend in and they all knew. I'm worried that I sound like an asshole and if that's the case I'm sorry. I actually in a lot of ways admire all those kids. I still think marijuana culture is pretty lame, but it's so cool that they can go to any place and have fun. They can talk and dance and bond with strangers and then go home as if it was just another weekend. I don't know if that mentality is better or worse than mine, but it's certainly no less fulfilling. You might disagree, all I'm trying to say is that I feel your frustration.