Finally got around to reading this, though it's been on my list since it was posted here. One nit I have to pick with it, is the assertion that GPT is on the same order of magnitude of efficiency as the human brain. It is not. It is not actually close. The brain has an estimated 100,000,000,000 neurons, sure (that's a SWAG actually, but that's another topic--we'll stipulate it here, since it's an oft-quoted number), and GPT has 175,000,000,000 "neurons", but all of GPTs ersatz neurons are focused on language, while only a tiny fraction of a human's are. In fact, like half of your neurons are in the cerebellum, which you can literally live without, though you'll have some problems (albeit not with language, per se). A giant part of the cortex and an even bigger proportion of the mid-brain, brain stem, and spinal cord deal almost exclusively with motor/sensorimotor function. So while I have no idea what the strict proportion of neurons that can be said to be doing language processing is, I'm sure it's more than an order of magnitude lower than GPT. That isn't to say that GPT isn't impressive...it's just to say that the difference in power is so strikingly obvious that it suggests that whatever GPT is doing is qualitatively different that what a brain is doing.

I enjoyed the read immensely, however. Great primer for a total noob like myself.