1. The Singularity. Jeron Lanier pointed out that technological atheists regard The Singularity in much the way Pentecostals regard The Rapture - a poorly-understood waving-of-hands and suddenly, we all get to live forever. There's a leap of faith involved in The Rapture - God will make it happen. There's a leap of fallacy involved in The Singularity - computers will solve all the problems involved in climbing "into the box." William Gibson brought up the Singularity before it was cool in Neuromancer. Case uses a "copy" of a hacker he used to know. It's only a copy, though, and even the copy feels fake. How perfect is perfect enough, presuming the technology exists? At what point is the "you" in the box as good as the "you" on the outside? And which one is "you" by the way? Lanier puts it this way in Your Brain In Silicon: If you dupe me and throw me in the box, is that me? I don't think it is - any more than my writing is me, any more than my photos are me, any more than my wife's memories of me are me. I think "the singularity" isn't immortality, it's an extremely intricate cenotaph. *So now your consciousness exists as a series of numbers in a computer; that is all a computer program is, after all. Let's go a little further with this. Let's suppose you have a marvelous new sensor that can read the positions of every raindrop in a storm. Gather all those raindrop positions as a list of numbers and pretend those numbers are a computer program. Now start searching through all the possible computers that could exist up to a certain very large size until you find one that treats the raindrop positions as a program that is exactly equivalent to your brain. Yes, it can be done: The list of possible computers of any particular size is large but finite, and so is your brain, according to the earlier steps in the thought experiment, anyway.
OK, so is the rainstorm conscious? Is it conscious as being specifically you, since it implements you? Or are you going to bring up an essence argument again? You say the rainstorm isn't really doing computation- it's just sitting there as a passive program- so it doesn't count? Fine, then we'll measure a larger rainstorm and search for a new computer that treats a larger collection of raindrops as implementing BOTH the computer we found before that runs your brain as raindrops AS WELL AS your brain in raindrops. Now the raindrops are doing the computation. Maybe you're still not happy with this because it seems the raindrops are only equivalent to a computer that is never turned on.