a thoughtful web.
Good ideas and conversation. No ads, no tracking.   Login or Take a Tour!
comment by briandmyers
briandmyers  ·  3760 days ago  ·  link  ·    ·  parent  ·  post: Should your robot driver kill you to save a child's life?

You've clearly thought a lot about this, and I do see what you're saying.

I'd still like to know what an autonomous car will do in these "oh shit" situations, and I don't believe it's mere sophistry to want those answers. I'd like to see some testing done (say, with pop-ups), designed to throw a monkey wrench into the car's navigation.

To put it really simply - if the car detects something unexpected in its path, does it : A) Swerve to avoid, if it deems that to be safe? If so, what exactly does it consider "safe" to be? B) Stay on course and slow itself as best it can? C) Other?

I'd also be interested in what you (or anyone) thinks are appropriate thought experiments.

One I'd be interested in exploring is "How badly can a determined person or group fuck up an autonomous car?", or "How well would such a car handle a malicious agent intent on (say) causing harm to a passenger in such a vehicle?"





kleinbl00  ·  3760 days ago  ·  link  ·  

Ahhhhhhhhh. Now we have an engineering problem. More than that, we have an engineering problem involving human mortality. Which means we have an engineering problem involving liability and statistics. It's going to come down to inputs and outputs, of which I know neither. But I think I know a little more than you, so let me expound upon shape of the problem as I understand it:

So a Google car relies on three things: LIDAR, vehicular telemetry and a phatty, phatty phatty GIS database. Based on this post we know that Google does not unleash a car on a road that it hasn't mapped in 3d space down to the INCH. Let that sink in for a minute:

Google knows the road so well it can detect a chipmunk. It can probably detect a tin of Carmex. It knows about that Camel Lights hardpack box you threw out the window. In fact, if there was a google car in front of you and a google car behind you when you threw it out the window, it knows YOU threw it out.

So there's an ethical issue to discuss.

Google also knows the telemetry of every Google car that has driven that road, ever. It knows the deviation from its normative map as recorded by that Google car's LIDAR. It knows the tire traction, the ambient temperature, the lateral acceleration and speed of every google car to ever go around the corner. And it not only knows the speed limit on the Blue Ridge Parkway, it knows when it changes due to road conditions.

Google also knows everyone who lives around that tunnel, and probably some of the people vacationing near it. Google likely knows that your family rented a Winnebago, they know that you're a quarter mile up the river, and they know that you have an eight year old daughter.

Google can not only see a chipmunk in the road, it can predict that your eight-year-old has a possibility of jumping in front of your car as you round the bend.

There's another ethical issue to discuss.

So let's talk about your "oh shit situation." The car's not going to violate the law. The car is not going to exceed safe road conditions. The car is not going to outdrive its brakes. The car can tell between a log and a person, between a deer and a 6-year-old. So if the car has passed all these checks and still finds an error in its programming (which is what a soon-to-be-dead 8 year old is, when you get right down to it), it's certainly going to file a bug report. Which means the next time a google car goes around the corner, it'll probably come at the tunnel slower.

But that's just liability to Google's customers. Chances are good that since road conditions put Google in a position of liability, Google is going to sue the highway department for unintentional tort and get the speed limit reduced. Google is going to raise the issue of highway safety at that corner and get a fence put up to keep campers from wandering onto the road. And google is going to point out that its vehicle was obeying every aspect of the law and driving as safely as a human would, and accidents happen.

But really, the scenario that even brings all this up is basically someone lunging out in front of an autonomous vehicle with the deliberate intent of getting hit. Which is suicide, which also doesn't fault Google. What's the car gonna do? The car is going to consider an impermanent hazard in its path to weigh less than a permanent hazard not in its path and it's gonna hit it. It's gonna put on the brakes, it's gonna try to get around the hazard, but it's gonna hit it. Same as if the girl were a deer.

Right?

There's another ethical issue to discuss - the only real one to come out of this whole discussion. In this tiny, moot corner case, Google is saddled with the task of identifying a human as a human and responding to it differently than a deer. But in order to do that, we need to know how and if Google can tell the difference between a human and a deer with LIDAR. Hell, we need to know if and how Google can tell the difference between a person and a mannequin. And I'm willing to bet Google isn't interested in having that discussion. Which is okay for our purposes because the author of this very-not-good article didn't even think to ask it.

So here's the ethical issue at the heart of this: how much responsibility does Google have to road hazards that are in violation of the law? More than a human driver? Less? That's what we're supposed to be discussing. Any court in the land will say "the same" and move on.

There were some kids in my town that decided it would be funny to attach a scarecrow to an overhanging branch on Halloween. A car would come around a blind corner and they'd throw it down to dangle in front of the road from a noose around its neck. Ha ha. People swerved. Ha ha. People cussed. Not so ha ha. One of them wrecked and had to go to the hospital. Yer damn skippy the kids were charged with reckless endangerment.

A human might be better able to distinguish a scarecrow on a rope from a real live person than a Google car will be. One thing about the Google car, though - the actions it takes will be scripted by someone calm, rather than someone trying not to run over a suicide victim.

thundara  ·  3760 days ago  ·  link  ·  

    But really, the scenario that even brings all this up is basically someone lunging out in front of an autonomous vehicle with the deliberate intent of getting hit. Which is suicide, which also doesn't fault Google.

To be fair, this happens in Russia quite frequently and people try to make easy money suing if they survive (Though usually it's aimed at only getting injured >_>). I wouldn't put it past people targeting a vehicle that they /knew/ would respond in a predictable way. Doubtful Google would lose in court, but hey, no court case is the best court case.

kleinbl00  ·  3759 days ago  ·  link  ·  

Absolutely - thus the proliferation of dashcams and with them, hilarious .gifs of Russians faking traffic accidents. Because if you have documentation of it, you can prove it was deliberate.

Thing of it is, a Google car kicks the shit out of a dashcam. All the collision avoidance sensors, the LIDAR, any sort of video, they're all streaming. And LIDAR takes up a lot less data than video. If I were Google, I'd set it to cache several minutes worth of data in RAM and then, if any sensor registered an anomalous event of any kind I'd write that shit to memory and upload it to the mothership. I mean, Google is going to be in the business of wanting to know about unknown potholes and shit in the road, not just fraud-minded jumpers.

And they're Google. They could be emailing you a fully-rendered Sketchup flythrough of the accident scene in 3D space, raw LIDAR traces helpfully ghosted over the map, GPS coordinates accurate to the inch, timestamped to within 40 nanoseconds of the NIST atomic clock before you finish dialing 911.

Which is another ethical issue to consider: Google is going to have lots of data about you and they'll analyze the shit out of it whether you personally need it or not. As far as our fraudster, though, the last car you want to go toe to toe with in court is gonna be a self-driving car.

thundara  ·  3759 days ago  ·  link  ·  

100% agree, except:

    If I were Google, I'd set it to cache several minutes worth of data in RAM and then, if any sensor registered an anomalous event of any kind I'd write that shit to memory and upload it to the mothership

If at all manageable, I'd upload 100% if I were Google (Or 0% if I was not). You want big data, "every car on the road" is huge data. Crowd-source street maps (They already do this with cell phones for roads, iirc), plot traffic patterns, study wild-life, get ambulances in the area before an accident even occurs, submit request tickets to cities to alter traffic laws where the traction has become a bit too low. A camera on ever corner is both a sci-fi writer's and data scientist's dream.

Even if they just took the position of selling (or opening) that data, that's A+ value to a business trying to pick out the next site to expand their offices / restaurants / outlets.

kleinbl00  ·  3759 days ago  ·  link  ·  

I'd expect some thinning, but yeah, that's about the gist of it.