I did wonder if I came off as a jerk! To me those things are not the same. It's not that I have a select club of people that I value based on their job/position/etc and forget about the rest. I just think that I should have a (meaningful) conversation with someone before I am going to remember their name and details. Because if I care about someone even a little, I try to remember everything about them. That is something I can't do genuinely for everyone I encounter, so I draw a line at 'is there a chance that this person will matter some day in the future'.
I hear ya, and I understand your thinking. But you are selling your brain short. Remembering people's names is easy when you get the hang of it, it doesn't diminish your capability to remember other stuff (our brains are NOT hard drives that "fill up"), and it keeps ya sharp. And, you never know when it could come in useful ... making a friend of the FedEx guy can have unexpected benefits. ("Hey veen, I noticed that package has a dent in the side. Why don't you open it and check it out. Yeah? Damaged? Ok. No problem. Keep it, and I'll file a report. We'll get you a new one tomorrow. Sorry about that!")