Were the brooms in The Sorcerer's Apprentice making other brooms? I forget.Margaret Atwood has a foolproof plan to stop our increasingly intelligent and powerful machines from rising up and taking over control of the planet: Make sure any robots we build have an easy-access “off” switch.
I just heard this speech by Margaret Atwood at the CHI conference.
Fun little thought shared with me by a control theorist: When you want to design a self-replicating system, you have to consider what inputs you wish to accept. If those inputs are ordered (Think, gears, nuts, bolts), the machinery required for each individual is relatively simple. Conversely, if you want to accept just about any input (Rocks, crude oil), the machinery in each individual must be complex enough to reprocess that input into forms it can use. In practice that means "robots" can't replicate in the wild until they start to look more like "cells". When you toss out the view of pre-processed parts, though, you start also questioning the environment: Is it stable? Or will you have shale this month and raw iron the next? In the latter, you start needing more heterogeneity within your population and a way to restore that heterogeneity across selection events. That means evolution, that means adaptation, that means mutations. That means a way to kill your "off" switch.