Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

 Peregrine wrote:

 sebster wrote:
As long as AI cars show a record of being clearly safer than human drivers they'll be adopted just fine.

I kind of disagree with that. From an engineering point of view it's absolutely true. Zero fatalities is a stupid goal, and the moment automated cars can demonstrate a better safety record than human drivers they should be put into mass production. The system should continue to be refined, of course, but holding back on a major safety improvement because it isn't quite perfect yet is insanity. But we aren't dealing with a pure engineering decision. Automated cars are going to be a political decision, and public perception of the situation is clearly not rational. People will get stupid and emotional and blow every accident out of proportion, and politicians will listen to their idiot base and impose regulations on automated cars.

Bam. Touching on part of this, the issue here is public and political perception. There are fears and ignorance tied to AI in general, and pop culture tech gone wrong stories and movies don't help this. Add in that you have one side of the debate wanting to use this tech to basically replace ALL human transport industry drivers, and one side that wants NONE on there, the people in the middle that see a useful asset are left with making sure that they "sell" the tech as well as possible, which means FAR less screw ups than would be expected by anyone who engineers these things.

 Just Tony wrote:

Bam. Touching on part of this, the issue here is public and political perception. There are fears and ignorance tied to AI in general, and pop culture tech gone wrong stories and movies don't help this. Add in that you have one side of the debate wanting to use this tech to basically replace ALL human transport industry drivers, and one side that wants NONE on there, the people in the middle that see a useful asset are left with making sure that they "sell" the tech as well as possible, which means FAR less screw ups than would be expected by anyone who engineers these things.

I'm in the NONE braket. I've seen too many screw ups and failures to think this is a safe idea.

You saw the blowback from the older lady on a bike? Wait til it's a kid that gets mowed down by a vehicle that commits an error along that line. Support will evaporate fast.

 BaronIveagh wrote:
I'm in the NONE braket. I've seen too many screw ups and failures to think this is a safe idea.

And this is just proving my point. You have seen screw ups and failures, but you take for granted that human drivers will regularly screw up and fail and kill people and you don't call for banning human drivers. Automated vehicles, even automated vehicles that occasionally screw up and kill people, can still be much safer than human drivers and provide a net improvement in safety.

 Peregrine wrote:

 BaronIveagh wrote:
I'm in the NONE braket. I've seen too many screw ups and failures to think this is a safe idea.

And this is just proving my point. You have seen screw ups and failures, but you take for granted that human drivers will regularly screw up and fail and kill people and you don't call for banning human drivers. Automated vehicles, even automated vehicles that occasionally screw up and kill people, can still be much safer than human drivers and provide a net improvement in safety.

I think the difference here is the potential magnitude of a single error.

With human drivers. if a human driver makes an error its just isolated to that particular human driver. His failure doesn't guarantee other drivers will make errors.

With a driverless car, if there is a programming error like the one that occurred with the biker, that error is duplicated across thousands or even hundreds of thousands of vehicles. Not to mention, lets say a terrorist hacks into the software updates for a particular brand of driverless car and plants a directive that would cause them all the crash with a specific environmental trigger.

I think there will be a number of different driverless systems, one from each manufacturer. This will make it more difficult to hack all the cars simultaneously.

Of course it would be disastrous even if only one brand of car got hacked. There is also the possibility that Google's software will be installed in every manufacturer's cars.

 Grey Templar wrote:
With human drivers. if a human driver makes an error its just isolated to that particular human driver. His failure doesn't guarantee other drivers will make errors.

It really isn't isolated. For example, a great many human drivers have the "using phone while driving" bug in their software. Its presence may not guarantee an accident for any particular driver, but it's sure as hell a recurring problem. We just take this baseline level of errors in human drivers for granted, and only consider it a "real error" if they do something exceptionally stupid.

With a driverless car, if there is a programming error like the one that occurred with the biker, that error is duplicated across thousands or even hundreds of thousands of vehicles.

Again, you're making the mistake of comparing to perfection rather than comparing to human drivers. The question is not "is this error going to be present", it's "is an automated vehicle with this error present more or less dangerous than a human driver and that human driver's known errors". And to answer it you have to look at failure rates caused by the error. Is it an error that happens every time you drive the car? Is it an error that only happens when certain rare conditions are met? If the error causes 100 deaths a year across all vehicles you accept the error and the deaths it causes because it's still safer than human drivers in those vehicles.

Not to mention, lets say a terrorist hacks into the software updates for a particular brand of driverless car and plants a directive that would cause them all the crash with a specific environmental trigger.

This is a bad movie plot not a realistic threat. Aside from the technical difficulties of executing such an attack terrorists aren't that good. Look at the overwhelming theme of terrorist attacks. They aren't spending years working their way into a company's engineering staff and getting enough of their friends hired that they can slip an attack through, they're buying an AR-15 at the local gun shop and heading for the largest concentration of people they can find nearby. These are incredibly simple attacks using weapons that are easily available. In terms of terrorism any vulnerability to a complex software attack is going to be more than offset by introducing automated rental trucks that shut down if you try to drive them into a crowd.

I can see it now: 'BIGGEST TERRORIST ATTACK IN HISTORY! THOUSANDS OF HACKED CARS IN NEW YORK RUSH HOUR!'


Automatically Appended Next Post:

 Peregrine wrote:
Aside from the technical difficulties of executing such an attack terrorists aren't that good.

Yeah, they'd never make all that effort to learn to fly planes or take over a civilian jumbo jet for a suicide attack! That's silly!

 BaronIveagh wrote:

 Peregrine wrote:
Aside from the technical difficulties of executing such an attack terrorists aren't that good.

Yeah, they'd never make all that effort to learn to fly planes or take over a civilian jumbo jet for a suicide attack! That's silly!

Well, to be fair, hijacking and then crashing a plane avoids two of the most difficult aspects of flight, take off and landing.

 A Town Called Malus wrote:

Well, to be fair, hijacking and then crashing a plane avoids two of the most difficult aspects of flight, take off and landing.

True, but simply dismissing the enemy as too incompetent to pull off a successful attack leads us down the path to things like 9/11, Isandlwana, and St Clair's Defeat.

 Peregrine wrote:

 Grey Templar wrote:
With human drivers. if a human driver makes an error its just isolated to that particular human driver. His failure doesn't guarantee other drivers will make errors.

It really isn't isolated. For example, a great many human drivers have the "using phone while driving" bug in their software. Its presence may not guarantee an accident for any particular driver, but it's sure as hell a recurring problem. We just take this baseline level of errors in human drivers for granted, and only consider it a "real error" if they do something exceptionally stupid.

With a driverless car, if there is a programming error like the one that occurred with the biker, that error is duplicated across thousands or even hundreds of thousands of vehicles.

Again, you're making the mistake of comparing to perfection rather than comparing to human drivers. The question is not "is this error going to be present", it's "is an automated vehicle with this error present more or less dangerous than a human driver and that human driver's known errors". And to answer it you have to look at failure rates caused by the error. Is it an error that happens every time you drive the car? Is it an error that only happens when certain rare conditions are met? If the error causes 100 deaths a year across all vehicles you accept the error and the deaths it causes because it's still safer than human drivers in those vehicles.

Not to mention, lets say a terrorist hacks into the software updates for a particular brand of driverless car and plants a directive that would cause them all the crash with a specific environmental trigger.

This is a bad movie plot not a realistic threat. Aside from the technical difficulties of executing such an attack terrorists aren't that good. Look at the overwhelming theme of terrorist attacks. They aren't spending years working their way into a company's engineering staff and getting enough of their friends hired that they can slip an attack through, they're buying an AR-15 at the local gun shop and heading for the largest concentration of people they can find nearby. These are incredibly simple attacks using weapons that are easily available. In terms of terrorism any vulnerability to a complex software attack is going to be more than offset by introducing automated rental trucks that shut down if you try to drive them into a crowd.

Nope. All it would take is a disgruntled programmer, maybe one who works for one of these companies, getting radicalized by some terrorists or doing it of his own accord as a “lone wolf”. Criminals are usually less intelligent individuals, key word ‘usually’. It might be bad movie plot material, but as technology becomes more common it’s going to become reality soon enough. Hackers won’t always stick to stealing credit cards and identities.

 Grey Templar wrote:

Nope. All it would take is a disgruntled programmer, maybe one who works for one of these companies, getting radicalized by some terrorists or doing it of his own accord as a “lone wolf”. Criminals are usually less intelligent individuals, key word ‘usually’. It might be bad movie plot material, but as technology becomes more common it’s going to become reality soon enough. Hackers won’t always stick to stealing credit cards and identities.

Hey! Patlabor 1 was a good movie!

BaronIveagh wrote:Yeah, they'd never make all that effort to learn to fly planes or take over a civilian jumbo jet for a suicide attack! That's silly!

And that problem was solved with sturdier cockpit doors and passengers who now know that terrorists are actually willing to fly planes into buildings so they have incentives to fight back instead of hoping for a regular old-school hijacking.

Has there even been a successful aircraft hijacking since 9/11?

 Vulcan wrote:
Has there even been a successful aircraft hijacking since 9/11?

There was that co-pilot who locked out the pilot when he went to the loo and crashed the plane into a mountain a few years ago. It wasn't a terrorist attack though, I think it was a bitter suicide. Can't think of any others.

Exactly. For decades, successful skyjacking depended heavily on the willingness of the passengers to go along quietly. And for decades, going along quietly was the best way to survive a skyjacking.

9/11 changed the rules, and now the passengers are not willing to go along quietly anymore.

 Vulcan wrote:
Has there even been a successful aircraft hijacking since 9/11?

Ethiopian Airlines Flight 702

Turkish Airlines Flight 1476

EgyptAir Flight 181

Air West Boeing 737 was hijacked over Sudan, but landed safely at N'Djamena, Chad.

a Sun Air (Sudan) Air Boeing 737 flying from Nyala

CanJet Flight 918

Aeroméxico Flight 576

Afriqiyah Airways Flight 209

Just last month there was the Air China CA1350 hijacking (extra credit on this one, one guy took the whole plane with a knife)

Those were the successful ones that they took over, post 9/11. There were about ten unsuccessful ones including the Germanwings Flight 9525 were the copilot crashed the plane.


Automatically Appended Next Post:

 Vulcan wrote:

9/11 changed the rules, and now the passengers are not willing to go along quietly anymore.

That's just hilarious in the light of how many actually successful hijackings occurred in that time span.

 Peregrine wrote:
I kind of disagree with that. From an engineering point of view it's absolutely true. Zero fatalities is a stupid goal, and the moment automated cars can demonstrate a better safety record than human drivers they should be put into mass production. The system should continue to be refined, of course, but holding back on a major safety improvement because it isn't quite perfect yet is insanity. But we aren't dealing with a pure engineering decision. Automated cars are going to be a political decision, and public perception of the situation is clearly not rational. People will get stupid and emotional and blow every accident out of proportion, and politicians will listen to their idiot base and impose regulations on automated cars.

People get stupid and emotional about things they don't experience directly, or only experience occasionally. There will be lots of concerns at first, but as automated cars are increasingly seen and experienced, the irrational concern will be replaced by an understanding learned from actual experience.


Automatically Appended Next Post:

 Grey Templar wrote:
I think the difference here is the potential magnitude of a single error.

With human drivers. if a human driver makes an error its just isolated to that particular human driver. His failure doesn't guarantee other drivers will make errors.

The only way that complaint makes sense is if we assume it is an error so blatant that all cars in the same situation will fail in the same way, and it's a circumstance that is common enough to happen often. Which is silly, as that's a level of programming automated cars moved past in their earliest stages.

And while focusing on that issue, you missed the great strength of that situation. When a mistake is found in a human driver, maybe we fix that driver's technique, or maybe not. A few billion other drivers carry on, with many suffering from that same problem. But with a car if its found that it reacts badly to a situation more than it should, we can improve the software and systems. We can begin a process of failing forward that we can't do with human drivers.

 Grey Templar wrote:
Nope. All it would take is a disgruntled programmer, maybe one who works for one of these companies, getting radicalized by some terrorists or doing it of his own accord as a “lone wolf”. Criminals are usually less intelligent individuals, key word ‘usually’. It might be bad movie plot material, but as technology becomes more common it’s going to become reality soon enough. Hackers won’t always stick to stealing credit cards and identities.

Uh, no. It would take way more than that because safety-critical things like the software for automated vehicles are not given to a single programmer. Aside from deliberate sabotage concerns that produces a single point of failure where any mistake the one programmer creates and doesn't catch gets into the final product. Any code is going to be thoroughly reviewed and tested by a team of engineers before it makes it into a final release. You aren't going to slip a blatant "crash all cars" attack past that unless you compromise an entire department of a company. And TBH if you're talking about that level of resources and dedication you're better off driving a bunch of rented trucks full of bombs into every crowd you can find. It's almost certainly going to cause more damage than crashing a bunch of vehicles that are designed to maximize crash resistance and passenger survivability.


Automatically Appended Next Post:

 BaronIveagh wrote:
Yeah, they'd never make all that effort to learn to fly planes or take over a civilian jumbo jet for a suicide attack! That's silly!

9/11 is actually strong support for my point. It was a very unsophisticated attack that exploited the fact that everyone assumed that a hijacking is resolved by a hostage situation and not by a deliberate crash. Once you find that assumption all you need is some people with knives and a willingness to die murdering thousands of innocent people. It doesn't take any particularly impressive skill or resources, just willingness to die. And resorting to that kind of attack demonstrates an inability to do anything more effective. Where a real threat would fire a barrage of cruise missiles (preferably armed with nukes) from a safe distance the 9/11 attacks were designed to compensate for the fact that the terrorists had very limited resources beyond willing martyrs. It's a kind of clever way around it, sure, but still a very clear reflection of their limits.

Multiple programmers didn’t stop a bug that makes a car think a person walking across the street was a plastic bag. If multiple people double checking can miss bugs and errors they could easily miss a line designed to cause a crash. There are lots of opportunities to insert this sabotage, not just the initial creation of the software. The countless updates that a smart cars computer would have would give plenty of opportunities to hide it. The line of code could also be quite small and difficult to spot.

It’s not a “crash all cars” command. It would be something simple like “turn wheels 90 degrees right and apply brakes if speed = 65 and there is another vehicle to the right “. Disguised as an evasive maneuver. And then the update goes out, with this hidden in its mass of other updates, and now every car has this line of code with nobody the wiser.

 Grey Templar wrote:
Multiple programmers didn’t stop a bug that makes a car think a person walking across the street was a plastic bag. If multiple people double checking can miss bugs and errors they could easily miss a line designed to cause a crash. There are lots of opportunities to insert this sabotage, not just the initial creation of the software. The countless updates that a smart cars computer would have would give plenty of opportunities to hide it. The line of code could also be quite small and difficult to spot.

It’s not a “crash all cars” command. It would be something simple like “turn wheels 90 degrees right and apply brakes if speed = 65 and there is another vehicle to the right “. Disguised as an evasive maneuver. And then the update goes out, with this hidden in its mass of other updates, and now every car has this line of code with nobody the wiser.

The bug in the OP got through because it was an edge-case scenario where, under certain unlikely conditions, the system could make the wrong decision. It slipped through because testing can't cover every possible scenario. But to have a successful terrorist attack you need something that happens constantly, or at least very frequently. That's going to be much easier to catch in testing, and it's going to be much harder to disguise it by making it happen as the unpredictable consequences of a piece of code rather than a direct instruction to do something wrong.

 Peregrine wrote:

9/11 is actually strong support for my point. It was a very unsophisticated attack that exploited the fact that everyone assumed that a hijacking is resolved by a hostage situation and not by a deliberate crash. Once you find that assumption all you need is some people with knives and a willingness to die murdering thousands of innocent people. It doesn't take any particularly impressive skill or resources, just willingness to die. And resorting to that kind of attack demonstrates an inability to do anything more effective. Where a real threat would fire a barrage of cruise missiles (preferably armed with nukes) from a safe distance the 9/11 attacks were designed to compensate for the fact that the terrorists had very limited resources beyond willing martyrs. It's a kind of clever way around it, sure, but still a very clear reflection of their limits.

You and I have very different ideas about the effectiveness of that attack, or the level of sophistication involved.

That said, by your logic a much more effective attack would have been to highjack a LNG carrier and do a Texas City in the middle of New York Harbor. If done right, it'd be the equivalent of a megaton range nuke.


Automatically Appended Next Post:

 Grey Templar wrote:
Multiple programmers didn’t stop a bug that makes a car think a person walking across the street was a plastic bag. If multiple people double checking can miss bugs and errors they could easily miss a line designed to cause a crash. There are lots of opportunities to insert this sabotage, not just the initial creation of the software. The countless updates that a smart cars computer would have would give plenty of opportunities to hide it. The line of code could also be quite small and difficult to spot.

It’s not a “crash all cars” command. It would be something simple like “turn wheels 90 degrees right and apply brakes if speed = 65 and there is another vehicle to the right “. Disguised as an evasive maneuver. And then the update goes out, with this hidden in its mass of other updates, and now every car has this line of code with nobody the wiser.

Just think how many game companies have things inserted by underpaid overworked coders that cause embarrassment later. Hot Coffee, anyone?

 BaronIveagh wrote:

 Vulcan wrote:
Has there even been a successful aircraft hijacking since 9/11?

Ethiopian Airlines Flight 702

Turkish Airlines Flight 1476

EgyptAir Flight 181

Air West Boeing 737 was hijacked over Sudan, but landed safely at N'Djamena, Chad.

a Sun Air (Sudan) Air Boeing 737 flying from Nyala

CanJet Flight 918

Aeroméxico Flight 576

Afriqiyah Airways Flight 209

Just last month there was the Air China CA1350 hijacking (extra credit on this one, one guy took the whole plane with a knife)

Those were the successful ones that they took over, post 9/11. There were about ten unsuccessful ones including the Germanwings Flight 9525 were the copilot crashed the plane.


Automatically Appended Next Post:

 Vulcan wrote:

9/11 changed the rules, and now the passengers are not willing to go along quietly anymore.

That's just hilarious in the light of how many actually successful hijackings occurred in that time span.

Fair point. I guess I've been a bit oblivious.

 Grey Templar wrote:
Multiple programmers didn’t stop a bug that makes a car think a person walking across the street was a plastic bag. If multiple people double checking can miss bugs and errors they could easily miss a line designed to cause a crash. There are lots of opportunities to insert this sabotage, not just the initial creation of the software. The countless updates that a smart cars computer would have would give plenty of opportunities to hide it. The line of code could also be quite small and difficult to spot.

Cars right now have software that controls breaking, acceleration, steering and a lot of other stuff. To the extent that cars could be hacked to kill people, we're already in that world.

 sebster wrote:

 Grey Templar wrote:
Multiple programmers didn’t stop a bug that makes a car think a person walking across the street was a plastic bag. If multiple people double checking can miss bugs and errors they could easily miss a line designed to cause a crash. There are lots of opportunities to insert this sabotage, not just the initial creation of the software. The countless updates that a smart cars computer would have would give plenty of opportunities to hide it. The line of code could also be quite small and difficult to spot.

Cars right now have software that controls breaking, acceleration, steering and a lot of other stuff. To the extent that cars could be hacked to kill people, we're already in that world.

Yes and no. You could only hack a current car by physically connecting to it, since they don’t ever go online after they hit the road. A ‘smart car’ in the future would connect periodically, if not continually, for software patches, map updates, etc. like my computer at home does.

So cars right now are effectively immune to hacking because they are closed systems. A smart car system would be infinitely more susceptible because if you hack a software update you’ve now hacked all the cars, not just one, and it’s not a closed system anymore.

 BaronIveagh wrote:

Just think how many game companies have things inserted by underpaid overworked coders that cause embarrassment later. Hot Coffee, anyone?

Hot Coffee wasn't inserted by a malicious coder. It was an originally planned feature which was later cut. As with most games, when you cut something you don't delete all the files, just remove enough that it will no longer activate during play, and so people who know what they're doing can get them back by going through the games files to find unused assets and setting them up to be used again.

 Grey Templar wrote:
Yes and no. You could only hack a current car by physically connecting to it, since they don’t ever go online after they hit the road. A ‘smart car’ in the future would connect periodically, if not continually, for software patches, map updates, etc. like my computer at home does.

First of all you were talking about malicious software being included during development. You need to follow your own argument.

Second of all, it's a completely blind assertion that automated cars will be constantly updated, while other modern cars never will be. Maybe automated cars will only have software updates at dealerships, much like you get satnav maps updated now. Or maybe in the future all cars, automated or not, will have live updates. There's no reason that automated cars will be any different, claiming that's a unique risk to automated cars is just you making stuff up to try and ignore the flaws in your idea.

 Grey Templar wrote:
Multiple programmers didn’t stop a bug that makes a car think a person walking across the street was a plastic bag. If multiple people double checking can miss bugs and errors they could easily miss a line designed to cause a crash. There are lots of opportunities to insert this sabotage, not just the initial creation of the software. The countless updates that a smart cars computer would have would give plenty of opportunities to hide it. The line of code could also be quite small and difficult to spot.

It’s not a “crash all cars” command. It would be something simple like “turn wheels 90 degrees right and apply brakes if speed = 65 and there is another vehicle to the right “. Disguised as an evasive maneuver. And then the update goes out, with this hidden in its mass of other updates, and now every car has this line of code with nobody the wiser.

The bolded is something that I have a legitimate concern about. Anyone with electronic devices knows that processing speed is affected the more stuff you fill a hard drive with. Can they rate exactly how many updates this computer can take before the slower processing speed prevents it from performing a safety correction in time? Do we have to basically swap out the drive once a year? Are we going to have official versions of the programming with the updates that are more compressed? I have a desktop at my house that is only about five or six years old. When I first got it, it could run Mark of Chaos: Battle March with no lag time at all. Load times in game were phenomenally fast. It may have been two years in when running the exact same game started to experience noticeable lag when trying to load battles or that town screen where you replace your troops etc. I can't imagine that a self driving car is going to have some magical drive that doesn't suffer from t hose issues with updates.

 BaronIveagh wrote:
You and I have very different ideas about the effectiveness of that attack, or the level of sophistication involved.

Clearly we do. Mine are right, yours are wrong. Virtually everything about the 9/11 attacks is a result of limited resources. It was a fairly clever case of getting the most from limited resources and exploiting a one-time security hole as effectively as possible, but it was still an attack that anyone could have done. I could have done it, if I was a murderous cultist willing to die killing thousands of innocent victims. All you needed was some easily accessible training and tools, a willingness to die, and an utter lack of a conscience.

Just think how many game companies have things inserted by underpaid overworked coders that cause embarrassment later. Hot Coffee, anyone?

That is not at all comparable. Aside from the question of deliberate insert vs. failure to properly delete old material you're also talking about a video game. Video games are frivolous entertainment, the worst that can happen if you screw up a video game is that the customers aren't happy with it. Therefore a video game's code isn't going to be subject to the same review and testing standards as safety-critical software like an automated vehicle's driving code. It's very easy to slip something in when nobody cares enough to put much effort into stopping it. It's very hard to do it when you have a whole team of people dedicated to catching every possible flaw and procedures that are carefully designed to ensure that no single person can just throw something in.


Automatically Appended Next Post:

 Just Tony wrote:
The bolded is something that I have a legitimate concern about. Anyone with electronic devices knows that processing speed is affected the more stuff you fill a hard drive with. Can they rate exactly how many updates this computer can take before the slower processing speed prevents it from performing a safety correction in time? Do we have to basically swap out the drive once a year? Are we going to have official versions of the programming with the updates that are more compressed? I have a desktop at my house that is only about five or six years old. When I first got it, it could run Mark of Chaos: Battle March with no lag time at all. Load times in game were phenomenally fast. It may have been two years in when running the exact same game started to experience noticeable lag when trying to load battles or that town screen where you replace your troops etc. I can't imagine that a self driving car is going to have some magical drive that doesn't suffer from t hose issues with updates.

You don't need a magical drive, you just need proper software design. Unlike a PC you aren't going to be installing a bunch of new stuff on top of the existing data. In fact, you aren't going to have control over the software at all. An update can wipe the drive entirely at every update cycle (other than a small amount of space required for the updater code) and replace it with a fresh copy of the latest version of the software.

Commercial aircraft have been using autopilots for years. Are there any examples of those being hacked? I should think car autopilots will be held to the same standards, which means if there have been a lack of plane incidents, that would suggest car incidents are similarly likely to be incredibly rare.

I am not talking about failures of autopilot programming/sensors that have been corrected due to having pilots onboard, I mean incidents of autopilots being deliberately hacked to cause damage (which would probably still result in the pilots taking control).