What I've never understood is why? Why do we want AI that thinks and acts like human intelligence (setting aside the questions of what makes human intelligence human)? Humans are amazing at many things, but are deficient is many areas. Writing novels and making amazing creations in all sorts of media are one of the areas we are decidedly NOT deficient in. All this human modeling from the Blue Brain onward has always struck me as such a navel-gazing waste of resources.
Because the thinking AI/red heifer will demonstrate/fulfill predictions/prophecy about the coming Singularity/Rapture which will sweep all righteous/righteous into an exciting new future/paradise in which the skeptics/nonbelievers will face extinction/tribulation prior to establishing our neoutopia/Kingdom of Heaven.
The altruistic long-future perspective says that the fewer humans have to work in menial jobs, the more free they will be to use their big brains for other things; art, science, whatever. There are all kinds of shit jobs that can be replaced with a moderately capable AI, and those people can now go find meaningful work elsewhere. Of course, if you are an American there is no social safety net to allow you time to develop your art/idea/passion/invention, so you need to come from a rich family who will underwrite you until your product hits the market. But hey - as long as daddy is rich and you have all the free time in the world, you can employ AI to do all the "meaningless" tasks you don't want to do. Doesn't that sound like a wonderful utopia? Yeah nah... me neither.
Effective Altruism argues "I don't need to think about the person in front of me, I need to think about all the hypothetical millions that will come after him (that will be of my stock, educated by me, of my tribe, in my clan, who follow my beliefs)." that's the whole schtick. It's yet another selfish-filter, like libertarianism or anarchism. Apropos of nothing this? Is only as hilariously awesome as it is because it is so blissfully lacking in self-awareness. It is a Spinal Tap-grade examination of the fundamental principles of "fuck around and find out." “This is the true story, of seven selfish assholes, deciding to move to Mexico, grift off each other, and have their lives taped. Find out what happens, when people stop being polite, and start getting Libertarian… The Libertarian World!”