There's a little nuance to the goal I haven't finagled yet, specifically whether or not "every Rolling Stones album" means "every (studio) album" or "every-every album," but I figure I don't need to decide on that one until I have every studio album first.

From someone who once had 1100 albums, I say that you want to collect all the studio albums. The point of having a collection like that is delving deep into the artists' psyche, study their artistic development, and hear the details of what they were able to create when they had time, equipment, and the ability to do it over and over again until they got the recording they wanted.

Live albums and other one-offs are an interesting documentary about one night (or a hand-selected series of tracks from multiple nights), but they only get one shot at the song, the crowd has an effect on their performance, and equipment/recording techniques get in the way of nuance.

So yeah, there my be one or two live recordings that add materially to your understanding of the artist, but "all the studio albums" is where my money's at.