The prevailing opinion on the subreddit I got this from is that the problem is with the word 'privilege' which implies an advantage when the reality is more of a lack of disadvantage.

Privilege is a little more more than just "lack of disadvantage" - that's definitely part of it, but it's also having your arbitrary and unchosen characteristics being actively viewed as a positive and make you more likely to be hired, respected, etc.

Researchers have a good name for when people accept that others are disadvantaged but are reluctant to accept their own advantages, and it's called "half blindness of privilege". There's a really good paper on the topic here: Group Dominance and the Half-Blindness of Privilege.

Some key points from the paper:

People—and even researchers—refer to groups whose identities are problematized and whose social positions are problematic as “underprivileged,”“disadvantaged,”“stigmatized,” or, rarely, “oppressed.” Contrasting groups are rarely described—let alone in the terms that designate their superior social position—as privileged, advantaged, legitimate, or oppressor. This practice of marking the “problematic” group reveals that the unmarked situation of dominant groups is assumed to be normal. Such a stance is only half-blind concerning group privilege, because although it focuses attention on “problematic” groups and may acknowledge group inequality, it does not acknowledge the social position of the referent group as privileged. The phenomenon we address in this paper is half-blindness to privilege—the acknowledgement of social inequality with the implicit assumption that dominance is normal. By taking dominance as normal, superior social positions and greater power do not seem to be privileges.

and

In every sample, participants rated subordinated groups’ status substantially lower than dominant groups’ status. From this finding we can conclude that participants do perceive group inequality. However, recognizing inequality is not the same as recognizing superiority as privilege, as our other results show. Dominants reported relatively low awareness of their dominant group identities. This result helps to explain why from dominants’ viewpoint, their group's situation is normal, not privileged. Consistent with the idea that being a dominant group member is an implicit norm against which other features contrast, subordinates rated the salience of their group membership higher than dominants did. From subordinates’ viewpoint, their situation is not normal and neither is group inequality. For this reason, subordinates are more likely to view group advantage as privilege. Findings like these are not limited to college student participants. In a national random sample of the U.S., ethnic subordinates rate their racial identity more important than Whites do (Hartmann et al., 2009).

The fact that group differences in identity salience were found whether the subordinated groups were numerical minorities (e.g., lesbians, gays, and bisexuals) or were not (e.g., women) indicates that it is not numerical status that makes groups socially normative or noticeable (see also Pratto et al., 2007). In fact, several processes might contribute to greater subordinated group salience. Because of segregation and under-exposure to subordinates in the media, all subordinated groups, including women, might be rare in people's minds. In interpersonal interactions, individuals’ subordinated identity is called out and assigned blame through stereotypes and prejudice. Also, all of the subordinated groups considered here have also built identity-positive social and political movements that could also make such identities salient and have positive significance.

Group identity was less salient to dominants, so their own experience may make dominants wonder why subordinates are so interested in group identity. As DuBois (1897) highlighted, the privilege of having dominant identity is not having to be aware of the identity, nor of the privileges that identity brings, nor having to repair either. In fact, because dominant identity is so normative, it may be easier for members of dominant groups to understand their group identity in contrast to subordinated groups. That is, dominants may use subordinates as a reference point to make clear who they are by identifying who they are not. Doing so is likely to emphasize difference and divisions between groups, and to invoke negative stereotypes of subordinates.