a thoughtful web.
Good ideas and conversation. No ads, no tracking.   Login or Take a Tour!
comment
kleinbl00  ·  3620 days ago  ·  link  ·    ·  parent  ·  post: When autonomous software breaks the law, who's to blame?

There will be a test case and it will be f'ing bizarre.

Culpability, as I understand it, comes down to agency and intent. Those three words are only together because I've hung around enough law students to pick up a little by osmosis. Whippin' out that august body of legislative wisdom, Wikipedia:

    A person is culpable if they cause a negative event and

    (1) the act was intentional;

    (2) the act and its consequences could have been controlled (i.e., the agent knew the likely

    consequences, the agent was not coerced, and the agent overcame hurdles to make the event

    happen); and

    (3) the person provided no excuse or justification for the actions

So whatever test case we're gonna find, it's gonna have to be a fully autonomous event that somehow isn't negligent.

How 'bout...

Teen hacker creates a Roomba plugin ('skynet 9000') that causes the celebrated line of robotic vacuums to chase any movement it detects (cats, etc). However, the hacker programs a failsafe that stops the rogue roomba when its collision sensors trigger.

iRobot pushes a software update that uses a different communication pathway between the collision sensors and the drive motors. As a consequence, Roombas running Skynet 9000 no longer stop upon bumping up against their targets, as Maude Smith discovers when her Roomba 'Mr. Finster' mauls her morbidly obese chihuahua 'Tiddlypoop' when his short little legs and corpulent little belly prove no match for Mr. Finster's murderous rage, courtesy some goofing around by her 13-year-old son Timmy.

Mrs. Smith sues iRobot for the software update. iRobot argues that Mr. Finster was running malicious code. Mrs. Smith's attorneys point to iRobot's Create program and argue that Skynet 9000 was not dangerous to canines until they pushed their update.

Mr. Finster ran on a schedule - there was no agency that set him in motion. Timmy didn't intend for the dog to get mauled - he overcame some hurdles to accomplish his task but that was before the software update. iRobot's only culpability is in whether or not their software should have been so locked down that nobody could hack it, and whether or not they should have been cognizant of all unofficial patches.

I could see a court ruling that nobody was at fault there (see "moral evil" vs. "natural evil"). The question is what kind of precedent it would set.