Forum adverts like this one are shown to any user who is not logged in. Join us by filling out a tiny 3 field form and you will get your own, free, dakka user account which gives a good range of benefits to you: No adverts like this in the forums anymore. Times and dates in your local timezone. Full tracking of what you have read so you can skip to your first unread post, easily see what has changed since you last logged in, and easily see what is new at a glance. Email notifications for threads you want to watch closely. Being a part of the oldest wargaming community on the net. If you are already a member then feel free to login now.

 sebster wrote:

 Grey Templar wrote:
Yes and no. You could only hack a current car by physically connecting to it, since they don’t ever go online after they hit the road. A ‘smart car’ in the future would connect periodically, if not continually, for software patches, map updates, etc. like my computer at home does.

First of all you were talking about malicious software being included during development. You need to follow your own argument.

Second of all, it's a completely blind assertion that automated cars will be constantly updated, while other modern cars never will be. Maybe automated cars will only have software updates at dealerships, much like you get satnav maps updated now. Or maybe in the future all cars, automated or not, will have live updates. There's no reason that automated cars will be any different, claiming that's a unique risk to automated cars is just you making stuff up to try and ignore the flaws in your idea.

You weren’t paying attention then. I was talking about both initial construction and updates after the fact. Maybe I wasn’t 100% clear, but it was obvious enough. And you are deluding yourself if you think self driving cars will not get updates with similar frequency to your desktop computer. Especially since such ability to update would be necessary to fix any bugs that are found after development. If your car is unsafe because of a bug you can’t drive it to a dealer for an update, it would need to update from your garage/driveway for that to work. Just for liability reasons if nothing else. And that same functionality will make them extra vulnerable to hacking.

And manually driven cars will still be different than driverless cars. You might be able to hack a regular car remotely eventually, but you’d never be able to actually drive it. That limits what could be done with your hack.

 Haighus wrote:
Commercial aircraft have been using autopilots for years. Are there any examples of those being hacked? I should think car autopilots will be held to the same standards, which means if there have been a lack of plane incidents, that would suggest car incidents are similarly likely to be incredibly rare.

I am not talking about failures of autopilot programming/sensors that have been corrected due to having pilots onboard, I mean incidents of autopilots being deliberately hacked to cause damage (which would probably still result in the pilots taking control).

Yes and no. As you define it 'to cause damage' then no, not that I know of, but given that there are also several unexplained and missing aircraft, maybe? 'To take control of the aircraft' yes. The Department of Homeland Security managed to pull off penetrating the avionics system of a Boeing 757 (how is classified) in September of 2016 (but not revealed until November of last year) though rumors and FBI warrants have suggested it may have happened before, possibly as far back as 2008, sometimes using the onboard inflight entertainment systems as a point of entry into the aircraft's network.

Boeing, of course, claims that God Himself could not hack our planes.

I meant with the intention of causing damage, because it would be difficult to actually do so sufficiently quickly with the pilots ready to take over. A plane cannot exactly swerve into the car a metre away, so it takes longer to do something bad to the plane.

Ok, so some probable isolated incidents, and one by a massive state actor working in a research position to test the system security.

 Haighus wrote:
I meant with the intention of causing damage, because it would be difficult to actually do so sufficiently quickly with the pilots ready to take over. A plane cannot exactly swerve into the car a metre away, so it takes longer to do something bad to the plane.

Ok, so some probable isolated incidents, and one by a massive state actor working in a research position to test the system security.

According to one FBI affidavit, a hacker bypassed the autopilot and directly took over the thrust control system. This would preclude pilots being able manually take control of the plane again without the hacker allowing it. However, again, this was done by someone ON the airplane. So...

Haighus wrote:Commercial aircraft have been using autopilots for years. Are there any examples of those being hacked? I should think car autopilots will be held to the same standards, which means if there have been a lack of plane incidents, that would suggest car incidents are similarly likely to be incredibly rare.

I am not talking about failures of autopilot programming/sensors that have been corrected due to having pilots onboard, I mean incidents of autopilots being deliberately hacked to cause damage (which would probably still result in the pilots taking control).

I wouldn't compare those two (aviation and cars). In short: In aviation the defining principle is "humans make mistakes" and all processes are based on that. That's why they have a multitude of checklists and failsafe mechanisms. If a plane crashes it gets investigated and the results are used to improve the process.

With cars the expectation is that the driver has to perform as perfect as possible to stay safe. There are not numerous checklists, no copilots, or similar features. Cars improved from an "it works" situation to become safer while planes start with "it has to be safe because there are hundreds of people on this thing".

Grey Templar wrote: You weren’t paying attention then. I was talking about both initial construction and updates after the fact. Maybe I wasn’t 100% clear, but it was obvious enough. And you are deluding yourself if you think self driving cars will not get updates with similar frequency to your desktop computer. Especially since such ability to update would be necessary to fix any bugs that are found after development. If your car is unsafe because of a bug you can’t drive it to a dealer for an update, it would need to update from your garage/driveway for that to work. Just for liability reasons if nothing else. And that same functionality will make them extra vulnerable to hacking.

And manually driven cars will still be different than driverless cars. You might be able to hack a regular car remotely eventually, but you’d never be able to actually drive it. That limits what could be done with your hack.

While you are partly correct the comparison of a car's computer with a regular desktop is rather bad. They use different hardware and operating systems (usually much simpler) and your car doesn't have to be able to interface with thousands of printers and all kinds of USB knick-knacks. That means they usually have a much, much smaller target area for attacks (but they are still not magically invincible). The same goes for the idea that updates will slow them down like they do with regular computers. Car MCUs usually don't collect cruft like PCs for generic everyday use (because they are used for very specific jobs and are not used to run the entertainment system where convenience is king).

Car companies have some problems when it comes to programming as their industry had to evolve once cars included more and more electronic components but they are also rather conservative when it comes to that. They don't work like games companies or VC funded web startups (quick and reckless) because recalls at that scale and are really expensive for them and they want to avoid those if possible (and they still feth up a lot). If I remember correctly the creator μTorrent worked as a programmer for a company in the automotive industry before switching to the consumer after μTorrent. In the early μTorrent versions you could see traces of his old work pattern. Very compact yet efficient, few bugs (probably mostly due to how the program had to interface with all kinds of consumer OS features).

Cars have already been hacked. Some time ago there were a few rather public articles about how somebody was able to gain control of some parts of a Land Rover (I think) electronics and shut it down and do some minor mischief which doesn't sound bad until you think about what could happen if somebody were to shut down your car on a highway at high speed. And there were multiple instances of electronic keys not being sufficiently secure. An autopilot isn't needed to create a lot of damage with or to a car.

Haighus wrote:Commercial aircraft have been using autopilots for years. Are there any examples of those being hacked? I should think car autopilots will be held to the same standards, which means if there have been a lack of plane incidents, that would suggest car incidents are similarly likely to be incredibly rare.

I am not talking about failures of autopilot programming/sensors that have been corrected due to having pilots onboard, I mean incidents of autopilots being deliberately hacked to cause damage (which would probably still result in the pilots taking control).

You're also looking at whether or not they keep the autopilot connected to a wifi signal to allow the Autopilot constant updates. If my computer, for instance, is disconnected from an internet source, during that period the ONLY way someone could hack it is to physically walk up to it. Harder to do that to a plane. Now, if they're satellite linked and constantly updating, then that's the issue. And also can the autopilot be isolated once connection is made? All questions we don't have ready answers for unless we have some commercial pilots or other professionals in the travel industry on here to sign in and chime in.

Peregrine wrote:

 Just Tony wrote:
The bolded is something that I have a legitimate concern about. Anyone with electronic devices knows that processing speed is affected the more stuff you fill a hard drive with. Can they rate exactly how many updates this computer can take before the slower processing speed prevents it from performing a safety correction in time? Do we have to basically swap out the drive once a year? Are we going to have official versions of the programming with the updates that are more compressed? I have a desktop at my house that is only about five or six years old. When I first got it, it could run Mark of Chaos: Battle March with no lag time at all. Load times in game were phenomenally fast. It may have been two years in when running the exact same game started to experience noticeable lag when trying to load battles or that town screen where you replace your troops etc. I can't imagine that a self driving car is going to have some magical drive that doesn't suffer from t hose issues with updates.

You don't need a magical drive, you just need proper software design. Unlike a PC you aren't going to be installing a bunch of new stuff on top of the existing data. In fact, you aren't going to have control over the software at all. An update can wipe the drive entirely at every update cycle (other than a small amount of space required for the updater code) and replace it with a fresh copy of the latest version of the software.

Then somebody needs to tell AT&T that is possible while I still have memory left on my phone...

Ok, lots of discussion on the aviation comparison. My thoughts are that there is clearly a precedent, so car companies should be liasing with the existing industry, and should ideally be held to similar standards in many regards.

The current car industry operates differently, but I think the new automated industry could operate more simarly to the aviation model if they are regulated as such in these early stages. Although robust regulation in this day and age seems to be a rarity...

 Grey Templar wrote:
And you are deluding yourself if you think self driving cars will not get updates with similar frequency to your desktop computer.

I am explaining to you that you are are making up completely unfounded assumptions. There is nothing saying automated cars will be treated like PC software. It is unknown, but it is at least as likely that they'll be treated like the existing software in cars is right now - with updates performed at dealerships. This is even likely to be regulated, as implementation testing will probably be required after each update. Hell, the industry will probably lobby for this as it ties you to your dealership and makes it more likely that's where you'll get your car serviced.

If your car is unsafe because of a bug you can’t drive it to a dealer for an update, it would need to update from your garage/driveway for that to work.

What in the hell? Right now cars are regularly recalled. They are driven to the dealers and fixed. Just last year my car was caught up in the air bag debacle, and I drove it to the dealership where the air bag was replaced.

Automated cars that are due for software updates will be approached the same way. Because people understand that just because a car is identified as having a problem that will cause some accidents as 10,000 units each drive their next 100,000 kms, that doesn't mean those cars are deadly dangerous death on wheels that can't be driven to the dealership.

Just for liability reasons if nothing else.

You raise liability issues, but don't see the liability issues in remote, blind software updates.

What's really ridiculous about this is you've identified a whole bunch of problems with remote, blind updates to the software. If you were being sensible you would conclude that remote updates are a bad idea, and that's probably why they won't be used. Instead you just insist remote updates are bad and lurch to 'and therefore automated cars are bad', without ever stopping to think that maybe if remote updates are bad they just won't use them.

It's all very silly and you should stop.

Automatically Appended Next Post:

 BaronIveagh wrote:
Boeing, of course, claims that God Himself could not hack our planes.

Sure, but God is really old. The old people I know type with two fingers and refer to it as 'the google'. And God is older than them. God's hacking skills are probably pretty limited, it what I'm saying.

Here is the flaw in your view, illustrated with a story.

We have some driverless cars. Suddenly, there is a recall regarding their software. They all have a glitch which causes them to mistake pedestrians crossing the street for harmless trash or leaves blowing in the wind. Now, all the cars need a software update to fix this. But they can only be updated at a dealership.

Oooops, We have a problem here. In order to fix the issue, the cars all have to drive to the dealership. But their current software guarantees that if they encounter a pedestrian crossing a road they'll fail to avoid the pedestrian. All those driverless cars obediently go to a dealership to get the update, but a few of them will hit some pedestrians on the way there and cause even more havoc. Which the driverless car companies will be liable for because they are, legally speaking, the ones responsible for what the cars do while they are driving on the road.

That is why these updates will happen at home via the internet and not at a dealership, just like your computer sometimes updates its OS via the internet. You'd be a moronic car company not to do your software updates just like a company like Microsoft does the updates for their computer operating systems.

It would also be far cheaper than having every dealership do each car individually as it comes in instead of just queuing up an update and having each car update its software while its parked somewhere with WiFi access, most likely overnight while its not being used. This is probably an even bigger reason you'd have remote updates, just trying to save some money. In this case a lot of money.

Customers as well I think wouldn't appreciate being told "Your car's software needs an update. Your car will now detour to the nearest licensed dealer for an update. We apologize for the inconvenience!" while they're in the middle of trying to go somewhere. And its not like a software update for a complex computer system necessary on a driverless car would be a once a year thing, it would be a monthly if not weekly occurrence. Just going by how often my computer has to download small updates.

There is liability either way for these driverless car manufacturers, but for them the better choice would be having the cars update remotely. Because its looks less bad for your cars to get hacked by a terrorist via your remote updates than for your cars to kill people while they're traveling to get an update that you told them to get/its worth the risk to save $ by not having dealerships update cars manually.

A lot of modern cars are online all the time.

They have wifi for the passengers. They connect to smartphone apps to let the driver prep them or lock them remotely. They tell the system where they are and what they are doing, so the satnav can provide the best route and the maker can get remote data on performance.

 Kilkrazy wrote:
A lot of modern cars are online all the time.

They have wifi for the passengers. They connect to smartphone apps to let the driver prep them or lock them remotely. They tell the system where they are and what they are doing, so the satnav can provide the best route and the maker can get remote data on performance.

So at worst, currently, an outside source could foul someone's GPS and misdirect them. Cars are still piloted by humans currently, so the damage would be minimal. Now, a great test run for someone trying to hack would be to see if they could cause the parallel park assist to guff up. If that was plausible, then sitting in wait until the transportation industry automates and utilize what they've learned to pull a truck off the side of the road to rob it. I'm not even thinking as big as turning a vehicle into a WMD, I'm looking at something as small as create a vulnerability window to take advantage of. You wouldn't even have to feel threatened by the anti-automation crowd sabotaging the process, tech savvy gang members could ruin the whole thing.



Fun fact: as I'm sitting here at work, being as vague as possible to not violate any NDA I might violate, an automated system is down for the 3rd time this week. Near as I can see, it's a simple communication issue between aspects of the operation, but the robot is still refusing to process the entire rest of the job that is unaffected by that one section of the cell. Now for a nifty exercise: picture that happening with a vehicle that has a family in it at 75 MPH on the interstate because of trust in the AI.

 Just Tony wrote:
And also can the autopilot be isolated once connection is made?

Isolated while still functioning? Depends on the autopilot, but probably not a very relevant question. The autopilot is not your PC/phone, you don't type in an address and tell it to go download something. You can't send the pilot an email with a fake link to the update page and get them to download your hack. It's connecting to a very specific address to send/receive very specific data, and none of that in any way interacts with the software that actually moves the controls. You're downloading GPS updates, current weather information, etc, that goes into an entirely separate location. It's possible that someone with detailed knowledge of a particular autopilot and airplane combination could find a way to exploit it, but it is certainly not an easy task at all. You'd probably have to start by compromising the other end of the system and getting your malicious code onto the server the plane is trying to connect to for its updates. And you'd almost certainly have to find a major bug in the software to exploit, without something seriously wrong with the system's code it would be an impossible task.

Isolated, who cares if it functions? Effortlessly. Pull one circuit breaker, which can be reached within seconds, and the plane is immediately back on manual control. Any pilot knows this resolution to a malfunctioning autopilot.

(Minor nitpick that the "autopilot" on a plane is a particular piece of hardware that probably never receives updates, and things like the flight planning tools/GPS database/etc are separate components. But the general discussion applies just fine to the broader category of the airplane's computer systems as a whole.)

Then somebody needs to tell AT&T that is possible while I still have memory left on my phone...

Again, you're making an invalid comparison. Your phone can't use the same approach to updates and accumulated junk data because that approach is not compatible with what you as a customer want. AT&T could easily make your phone work indefinitely without slowing down, but it would require a complete wipe and factory-new reset with every update. And I suspect you'd be pretty unhappy if every update deleted everything you have on your phone. So, instead, they have to use an update system where everything already on the phone is preserved and you never get a complete "nuke it all" reset. That allows junk data to accumulate, more and more processes to start running in the background and eating resources, etc.

The same is not true of a car's control computer. You don't have any data to keep, each update can completely wipe the hard drive and install a fresh copy of the software. You can't have accumulated garbage slowing everything down because nothing is allowed to accumulate. And you certainly don't have to allow idiot users to download every random app they find, regardless of the source. The manufacturer has full control over exactly what is on that computer at all times.


Automatically Appended Next Post:

 Just Tony wrote:
Fun fact: as I'm sitting here at work, being as vague as possible to not violate any NDA I might violate, an automated system is down for the 3rd time this week. Near as I can see, it's a simple communication issue between aspects of the operation, but the robot is still refusing to process the entire rest of the job that is unaffected by that one section of the cell. Now for a nifty exercise: picture that happening with a vehicle that has a family in it at 75 MPH on the interstate because of trust in the AI.

Ok, I'm picturing it. The AI system encounters the communication issue and determines that it is a fatal one preventing continued safe operation. The AI turns on the hazard lights, brings the engine to idle, and applies controlled braking to stop the vehicle. Once the vehicle is stopped it either shuts down there (safest, but inconvenient to other drivers) or enters a recover mode where the vehicle's speed is capped at 5mph and location can not exceed a short distance of the initial incident, allowing the vehicle to be pulled off the road. This happens because the engineers involved in the project understand how safety-critical software works and have paid attention to fail-safe systems and good error handling.

Now picture this alternative scenario: the human driver is texting while driving at 95mph on the interstate because something very important is happening. The driver loses control of their vehicle, crosses the median, and hits another car head-on. Everyone in both vehicles is instantly killed.

Now could you explain why the AI situation is considered an unacceptable level of risk, while the situation with the human driver is just taken for granted as the price of having cars and nobody uses it as an argument for banning human drivers?

You answered my question fairly quick about autopilot, and it can be disabled/isolated from external influence on the fly.

To me, yes I CAN deal with a phone wipe/reinstall. I save my contacts to my SIM, and my SD holds my pics, so anything else is completely superfluous and doesn't need to save every last whatever that's been done with it. But that's me, not the populace at large.

So when they automate the entire world, lag and hard drive space won't be a problem. Nifty. A little unbelievable, but nifty.

Last but not least, in direct response to you picturing the engineers involved who paid attention to good error handling: pedestrian=plastic bag. We already have a documented example of how "infallible" this sort of design by committee could be.


And since we're playing whataboutism and cherry picking the worst possible scenario to further our points, I submit the middle aged person who doesn't text and maintains situational awareness while driving safely wherever they need to go without ever experiencing an accident despite having people littering his traffic stream who are foreign college students with no driving experience and drive constantly distracted (This'd be me, for the record) and we'll compare it to a comp crash that shuts down the engine completely which lock up the gearing preventing the wheels of the car from turning and flip it end over end in high traffic on the interstate at 75 MPH because reasons.

Now, if you want to have a genuine dialogue, can you say with a serious face that AI and automated systems operate REMOTELY flawlessly enough to not warrant at least some concern?


For the record, I don't view some 16 year old twit who's Facebooking while driving as an acceptable level of risk. I'd personally love it if cars had jammers inside rendering phones inoperative.

 Just Tony wrote:
So when they automate the entire world, lag and hard drive space won't be a problem. Nifty. A little unbelievable, but nifty.

Again, you're comparing situations that are not at all similar, considering "automate the world" as a single problem, and ignoring the reasons why lag and hard drive space aren't an issue in certain cases. Automating the world will produce issues with lag and hard drive space, in cases where the user has data to preserve, gets to install their own software on a device, etc. In fact, it may even be a problem if cars get phone-like entertainment devices for the passengers. But it won't be an issue for the AI driving software because the manufacturer controls it, not the user. Tools that are not available for consumer-controlled devices like a phone are available for manufacturer-controlled devices, and those tools can eliminate lag and hard drive space.

Last but not least, in direct response to you picturing the engineers involved who paid attention to good error handling: pedestrian=plastic bag. We already have a documented example of how "infallible" this sort of design by committee could be.

You are confusing rare edge-case scenarios with common error handling. The vision system interpreting an image incorrectly is an entirely different failure case compared to the question of how, when the car encounters an internal error, that error is handled.

And since we're playing whataboutism and cherry picking the worst possible scenario to further our points

I'm not cherry picking the worst possible scenario, I'm describing the risk level of a common human driver. Lots of people text while driving, and only by sheer luck do they avoid having accidents. A real worst-case scenario on the level of yours would be someone texting while driving 95mph on the interstate while drunk and also deliberately attempting to kill people in an act of terrorism and suffering a medical crisis that prevents them from turning the steering wheel properly.

we'll compare it to a comp crash that shuts down the engine completely which lock up the gearing preventing the wheels of the car from turning and flip it end over end in high traffic on the interstate at 75 MPH because reasons.

This has nothing to do with AI driving software. The engine control software that could completely shut down the engine like that is already running on your car. If this was actually a relevant risk we would see it happening already. But we don't. And the fact that you present this as if it was a relevant risk just shows how poorly you understand the issues involved.

Now, if you want to have a genuine dialogue, can you say with a serious face that AI and automated systems operate REMOTELY flawlessly enough to not warrant at least some concern?

The fact that you are asking this question at all just demonstrates that you have very little understanding of the subject. The standard is not "flawless", it's "better than human drivers, who are absolutely terrible at driving and produce horrifying risk levels". Of course first-generation automated systems will not be flawless, and we should continue to work on improving them. But even in a badly flawed state they can be better than the alternative, and we should replace human drivers with that flawed AI for a net improvement in safety. The only argument against this is to take for granted that human drivers will kill people and that's ok, while using every single death to an AI vehicle as grounds for banning AI vehicles.

 Grey Templar wrote:
Here is the flaw in your view, illustrated with a story.

Your story is obviously ridiculous, and it's clear you're not even trying to think about this sensibly.

You suggest a critical software flaw that produces a significant chance of a fatality every time the car travels a short distance. Which is a staggeringly silly scenario to begin with, but then you double down on that by assuming that if that were to happen, then there's no alternative but to just send the cars to the dealership with fingers crossed. You're trying to pretend that tow trucks don't exist.

That's the sum total of your argument. 'What if they released automated cars that are so crappy they kill people every few miles, then everyone forgets tow trucks exist... then they'd need to have remote updates and that means they could be hacked by terrorists turning all the cars in to death traps!'

I don't like doing this. I don't want to be mean. But what's the alternative when you keep putting this nonsense up? So please just admit you really didn't think your initial idea through, then you got caught up in protecting your ego and starting making stuff that didn't make any sense, and we can all stop.


Automatically Appended Next Post:

 Kilkrazy wrote:
A lot of modern cars are online all the time.

They have wifi for the passengers. They connect to smartphone apps to let the driver prep them or lock them remotely. They tell the system where they are and what they are doing, so the satnav can provide the best route and the maker can get remote data on performance.

There's a big and obvious difference between being on-line for satnav etc, and being able to update core software.

 sebster wrote:
You're trying to pretend that tow trucks don't exist.

You don't even need tow trucks; the dealership will be connecting the car to a laptop or doing some short-range update, so there's no reason the dealers technician can't come to the broken car(s).

But you (as in one) needs to appreciate that cars are tested before going out into production and again before the real world, so a bug that causes them to mow down every pedestrian they encounter will never make it onto the road. There will be edge cases like this pedestrian-pushing-bike-on-freeway issue, and the odds of that being triggered if automated cars have to return to a dealership or update point are minimal. If the update goes wrong, it should be detectable before the car is allowed to drive off.

It's not without flaws; there will be some way to confuse the cars, but there should be sufficient fail-safes in place to prevent them doing more than getting lost or stopping in the middle of the road.

As said, the bar for automated cars is to be better than people, which at the moment is 1.18 fatalities per million vehicle miles traveled. Ideally we want that to drop to 0 but anything under 1.18 is better than humans are doing.

The problem at this stage is more political/social than technological - people are terrified of letting cars drive themselves. I suspect that by the time my kids or their kids can drive, if they even need to, then automated cars will be no big deal.

Herzlos wrote:
You don't even need tow trucks; the dealership will be connecting the car to a laptop or doing some short-range update, so there's no reason the dealers technician can't come to the broken car(s).

You're not considering that in the future the same space virus that destroyed the tow trucks also caused all employees of car dealerships to develop an acute form of agoraphobia that prevents them ever leaving the dealership.

But you (as in one) needs to appreciate that cars are tested before going out into production and again before the real world, so a bug that causes them to mow down every pedestrian they encounter will never make it onto the road.

I thought I covered that in a very unsubtle way;
"You suggest a critical software flaw that produces a significant chance of a fatality every time the car travels a short distance. Which is a staggeringly silly scenario to begin with..."

 Grey Templar wrote:

 sebster wrote:

 Grey Templar wrote:
Yes and no. You could only hack a current car by physically connecting to it, since they don’t ever go online after they hit the road. A ‘smart car’ in the future would connect periodically, if not continually, for software patches, map updates, etc. like my computer at home does.

First of all you were talking about malicious software being included during development. You need to follow your own argument.

Second of all, it's a completely blind assertion that automated cars will be constantly updated, while other modern cars never will be. Maybe automated cars will only have software updates at dealerships, much like you get satnav maps updated now. Or maybe in the future all cars, automated or not, will have live updates. There's no reason that automated cars will be any different, claiming that's a unique risk to automated cars is just you making stuff up to try and ignore the flaws in your idea.

You weren’t paying attention then. I was talking about both initial construction and updates after the fact. Maybe I wasn’t 100% clear, but it was obvious enough. And you are deluding yourself if you think self driving cars will not get updates with similar frequency to your desktop computer. Especially since such ability to update would be necessary to fix any bugs that are found after development. If your car is unsafe because of a bug you can’t drive it to a dealer for an update, it would need to update from your garage/driveway for that to work. Just for liability reasons if nothing else. And that same functionality will make them extra vulnerable to hacking.

And manually driven cars will still be different than driverless cars. You might be able to hack a regular car remotely eventually, but you’d never be able to actually drive it. That limits what could be done with your hack.

My friend works for Centcom at MacDill Airforce Base, and the first thing he did when he got his new truck was take the blue tooth out. Thats all I need to know about wireless tech in my car.

I will never, ever trust a self-driving car. Nor do I want to get used to riding in one to where I sit and watch Netflix without a care. But I know trucking companies drool over this, so I see if happening at some point.

But I don't see this asked much, nor answered, but for insurance and legal reasons, if your 'auto-pilot' is responsible for an accident, especially one that results in a death, who is responsible? The manufacturer for the auto-pilot? What if you missed a key update? Is it all on you? I mean, the litigation storm on the horizon is going to be messy. And what if an auto-pilot hits you and doesn't stop? I mean, its going to be f'in crazy I can't believe people think this isn't going to be a massive cluster f.

I enjoy driving, so not going to give that up. Plus I am a control freak. I don't even like friends and family driving, so damn will never let my car drive itself while I nap.

 KTG17 wrote:
My friend works for Centcom at MacDill Airforce Base, and the first thing he did when he got his new truck was take the blue tooth out. Thats all I need to know about wireless tech in my car.

Oh really? Does your friend actually work in a security-related area where he would be expected to have significant technical knowledge about computer security, especially for such a specialized application? Or are you just assuming that because he works in the building he must be an expert? I mean, technically the janitor there "works for Centcom at MacDill Airforce Base", but I wouldn't trust them with security decisions or give any credibility to their choices.

I will never, ever trust a self-driving car.

That's a pretty short-sighted view given that self-driving cars will, if not already, soon be safer than human-driven cars. Do you have a good reason for believing that human drivers, with all their well-demonstrated flaws and constant fatal accidents, are going to be safer? Or just reflexive paranoia about technology?

I enjoy driving, so not going to give that up.

You likely won't have a choice about it. Once self-driving cars demonstrate a better safety record than human drivers (a very low bar to clear) they will likely become mandatory and human drivers will no longer be allowed on public roads.

 sebster wrote:

 Grey Templar wrote:
Here is the flaw in your view, illustrated with a story.

Your story is obviously ridiculous, and it's clear you're not even trying to think about this sensibly.

You suggest a critical software flaw that produces a significant chance of a fatality every time the car travels a short distance. Which is a staggeringly silly scenario to begin with, but then you double down on that by assuming that if that were to happen, then there's no alternative but to just send the cars to the dealership with fingers crossed. You're trying to pretend that tow trucks don't exist.

That's the sum total of your argument. 'What if they released automated cars that are so crappy they kill people every few miles, then everyone forgets tow trucks exist... then they'd need to have remote updates and that means they could be hacked by terrorists turning all the cars in to death traps!'

I don't like doing this. I don't want to be mean. But what's the alternative when you keep putting this nonsense up? So please just admit you really didn't think your initial idea through, then you got caught up in protecting your ego and starting making stuff that didn't make any sense, and we can all stop.

Really? They're going to send Tow Trucks to collect potentially thousands upon thousands of cars? And they're going to do this with every software update? Instead of the easy solution of remote updates via the internet, that every other similarly complex computer uses already. And yes, in your scenario the only way for the cars to be fixed is for them to come to the dealership. Because you are suggesting that cars will not be hackable because they'll not be connecting to the internet at all and will receive updates only via direct linkups while being serviced. You proposed this scenario dude.

And I didn't pull this critical software flaw example out of thin air. It was an actual software flaw that happened with a real driverless car, and it actually killed someone. Your proposal to make these cars immune to remote hacking would make such a critical flaw(again, a flaw that actually happened) more difficult to fix, because the cars would have to be brought to the dealership. Either driving themselves(meaning those flawed vehicles are on the road with a critical error), sending technicians out to each car individually(impractical and hideously expensive given how often a complex computer like a self-driving car would need to be updated), or have a towtruck bring the cars in individually(again impractical and hideously expensive).

Many factors are in play here forcing driverless cars to be developed in a certain way, and they are going to result in cars which can be hacked remotely. Its far easier, cheaper, and makes more sense in every way for these cars to get software updates remotely. The only negative here is that they become susceptible to remote hacking. The question is, is that an acceptable downside for the benefits of Driverless Cars?

And I don't get why you insist on bringing ego into. The only person making any deal over ego is you Seb. Stop trying to appear and feel superior with your "If you'd only think this through" and "I don't like doing this" crap.


Automatically Appended Next Post:

Herzlos wrote:

But you (as in one) needs to appreciate that cars are tested before going out into production and again before the real world, so a bug that causes them to mow down every pedestrian they encounter will never make it onto the road.

I'm sure the self-driving car that mistook the jaywalker for a plastic bag got tested with jaywalkers before it got sent out, and yet someone still got killed.

Bugs can also be created unintentionally with software updates. Changing one part of the code can have unexpected consequences elsewhere. And thats ignoring any deliberate acts of sabotage.

Sure, a bug that causes a car to maliciously run over every pedestrian will get caught. One which causes the car to run someone over when there is a certain light level or there is additional debris might not, and this bug might happen in the real world much more often than you'd think.

 Grey Templar wrote:
Really? They're going to send Tow Trucks to collect potentially thousands upon thousands of cars? And they're going to do this with every software update?

You claimed it was for updates that rendered cars so unreliable they're dangerous to be on the roads. Now you shift to 'every software update'. You're all over the place.

Instead of the easy solution of remote updates via the internet

Out of one side of your mouth you're claiming remote updates are so easy, then out of the other side you're claiming remote updates make cars . You have to pick one of these things - either remote updates are just fine, or remote updates are a terrible security risk.

that every other similarly complex computer uses already.

It isn't about complexity, but about the consequences of a bad update. A computer update is a bit buggy then Battletech won't load. A car update is a bit buggy, and there's a ton of metal moving at 80mph with a sensor failure.

Because you are suggesting that cars will not be hackable because they'll not be connecting to the internet at all

Nope, didn't say that. You're making things up.

And I didn't pull this critical software flaw example out of thin air. It was an actual software flaw that happened with a real driverless car, and it actually killed someone.

You're ignoring the difference between prototype models and final consumer production versions. You're ignoring the very concept of 'prototype'.

How much longer are we going to keep doing this? Seriously, I'm getting that puppy kicking feeling and it's not fun. I know you don't like me mentioning that, but it is what it is. So seriously, you're argument isn't going to suddenly came back from the ashes if you just make one more post throwing in new made up stuff. And knowing me, I'm gonna feel compelled to just keep pointing out all the mistakes in your theory every time. So where does this silly nonsense end?

 Grey Templar wrote:
One which causes the car to run someone over when there is a certain light level or there is additional debris might not, and this bug might happen in the real world much more often than you'd think.

Now you're back to using the wrong standard. Yes, a bug like that can slip in, but does it really matter? It's a rare edge-case situation, and we still haven't done anything to solve common bugs with human drivers such as "driving while drunk and killing someone" or "texting while driving and killing someone". It would be nice to fix the bug, obviously, but even if the solution is "fix it next time the owner brings their car in for an oil change" it may result in fewer deaths than allowing human drivers to continue to exist.

 sebster wrote:

 Grey Templar wrote:
Really? They're going to send Tow Trucks to collect potentially thousands upon thousands of cars? And they're going to do this with every software update?

You claimed it was for updates that rendered cars so unreliable they're dangerous to be on the roads. Now you shift to 'every software update'. You're all over the place.

You're the one who suggested towtrucks in the first place, so you tell me what the towtrucks are for.

Instead of the easy solution of remote updates via the internet

Out of one side of your mouth you're claiming remote updates are so easy, then out of the other side you're claiming remote updates make cars . You have to pick one of these things - either remote updates are just fine, or remote updates are a terrible security risk.

Obviously its both. The updates are easy, and safe in the sense that if there is a critical flaw that is discovered the car can be patched remotely. While at the same time it is a security risk because it is exposing the car to outside interference. It can be both. I'm sure you are able to see that.

that every other similarly complex computer uses already.

It isn't about complexity, but about the consequences of a bad update. A computer update is a bit buggy then Battletech won't load. A car update is a bit buggy, and there's a ton of metal moving at 80mph with a sensor failure.

Yes.

Because you are suggesting that cars will not be hackable because they'll not be connecting to the internet at all

Nope, didn't say that. You're making things up.

You seem to be arguing that bad guys couldn't hack driverless cars for terrorism purposes, so yes.

And I didn't pull this critical software flaw example out of thin air. It was an actual software flaw that happened with a real driverless car, and it actually killed someone.

You're ignoring the difference between prototype models and final consumer production versions. You're ignoring the very concept of 'prototype'.

I'm sure that makes a big difference to the person who got run over. And like being a "final production version" will magically make the car less likely to have fatal flaws or software glitches. Computers still get software glitches after they've been released to the consumer, so I'm not sure what you're even trying to say.

How much longer are we going to keep doing this? Seriously, I'm getting that puppy kicking feeling and it's not fun. I know you don't like me mentioning that, but it is what it is. So seriously, you're argument isn't going to suddenly came back from the ashes if you just make one more post throwing in new made up stuff. And knowing me, I'm gonna feel compelled to just keep pointing out all the mistakes in your theory every time. So where does this silly nonsense end?

Again with the saying stuff to make yourself feel morally and intellectually superior. Its a habit you have, and not an endearing one. I suggest you be an adult and leave if you can't exercise some self-control.

 Grey Templar wrote:
You're the one who suggested towtrucks in the first place, so you tell me what the towtrucks are for.

Don't play dumb. I pointed out vehicles are taken to dealerships right now to fix issues. You said this was flawed, and invented the story of a glitch so terrible that "a few of them will hit some pedestrians on the way there". I explained that a software flaw so extreme would be extremely unlikely in a final consumer model, and if such an error were to present then tow trucks could be used to get the deadly dangerous cars to the dealership. At which point you claimed that tow trucks would be needed for every single update that happened. I pointed out that was ridiculous, because the tow trucks would only needed in your hypothetical of cars with a glitch so serious it's deadly just putting them on the road.

At which point we get to your last post, where you've now decided you're now very confused about the concept of tow trucks, and when they would and wouldn't be needed.

Obviously its both. The updates are easy, and safe in the sense that if there is a critical flaw that is discovered the car can be patched remotely. While at the same time it is a security risk because it is exposing the car to outside interference. It can be both. I'm sure you are able to see that.

Which means either the security risk of remote updates is so minimal that it's not worth talking about, or the security risk is very serious, in which case we won't use remote updates. You're trying to have it both ways, presenting remote access as a real threat, but also a threat no-one will avoid with a simple measure.

Yes.

And yet you insist that systems will updated with the same 'it should be okay' approach we take to updating PCs. Okay.

You seem to be arguing that bad guys couldn't hack driverless cars for terrorism purposes, so yes.

Nope, not arguing that at all. You're just making up random stuff.

And like being a "final production version" will magically make the car less likely to have fatal flaws or software glitches.

That's literally what the development process is. That's why we have prototypes, to identify and reduce flaws and glitches before production.

You've now put yourself in a place where you're arguing against the concept of product development.

Again with the saying stuff to make yourself feel morally and intellectually superior. Its a habit you have, and not an endearing one. I suggest you be an adult and leave if you can't exercise some self-control.

Why would I try and endear myself to you? What on earth makes you think I'm sitting here hoping the guy spouting ever expanding nonsense about nothing is someone I really want to have endeared to me?

 Grey Templar wrote:
Computers still get software glitches after they've been released to the consumer, so I'm not sure what you're even trying to say.

You're making the same common error that has already been made far too often in this thread. Your average PC is not a relevant comparison for two reasons:

1) Normal PC software development is not safety-critical software like a car's automated driving software. It doesn't have to go through the same level of review and testing, and companies are encouraged to use a "sell now, patch later" approach because customers want their stuff ASAP. Nothing is at stake if a bug happens, so why spend vast amounts of time and money on preventing them in advance? Safety-critical stuff, on the other hand, does get that investment and bugs are much less likely.

2) Normal PC software has a major comparability burden to deal with. It has to run on a near-infinite combination of hardware, it has to play nicely with everything else you have installed, the operating system has to allow any random software to run, etc. Because the customers demand the ability to do whatever they want with their PC the software developers can't use some powerful tools to keep bugs (and active hacking) under control. And testing all of these possible interactions is effectively impossible, especially with the budget constraints imposed (see factor #1). Single-task software like a car's driving system doesn't have to deal with that. It runs on known hardware with known other software, and the original manufacturer has full control over both parts. There's no "oops, this other game you installed modified a key file and now your game can't run" because nobody but the original manufacturer has any access to that computer.

The more relevant comparison would be with the engine computer your car is currently using. How often have you patched the software on it? I'd be willing to bet that the answer to that is extremely rarely, if ever.

I personally find Sebs approach to cutting through bs very endearing.

 A Town Called Malus wrote:
I personally find Sebs approach to cutting through bs very endearing.

Well sure but I'm trying to be endearing to you

 sebster wrote:

 A Town Called Malus wrote:
I personally find Sebs approach to cutting through bs very endearing.

Well sure but I'm trying to be endearing to you

Oh you

Also, just realised that was my 7000th post

 Peregrine wrote:

 KTG17 wrote:
My friend works for Centcom at MacDill Airforce Base, and the first thing he did when he got his new truck was take the blue tooth out. Thats all I need to know about wireless tech in my car.

Oh really? Does your friend actually work in a security-related area where he would be expected to have significant technical knowledge about computer security, especially for such a specialized application? Or are you just assuming that because he works in the building he must be an expert? I mean, technically the janitor there "works for Centcom at MacDill Airforce Base", but I wouldn't trust them with security decisions or give any credibility to their choices.

Yeah! He is the janitor! Does all the floors and everything! When I told him about your post, he assured me that he knew what was going on, and not to be worried, 'he knows a guy'. Then he showed me his new mop and explained how it cleaned up the floor better than other mops.

Humor aside, do you really think I am going to put your opinion over his? Come on.

I will never, ever trust a self-driving car.

That's a pretty short-sighted view given that self-driving cars will, if not already, soon be safer than human-driven cars. Do you have a good reason for believing that human drivers, with all their well-demonstrated flaws and constant fatal accidents, are going to be safer? Or just reflexive paranoia about technology?

Something to consider about drivers and how seriously we take the dangers of human drivers.

We let 16 year olds operate cars anywhere they want once licensed with few restrictions. However, in a commercial kitchen a 16 year old is not even allowed to use a knife with no option to be licensed for safety reasons.

Let's all think about that for a moment.

Clearly, as a society we have let convenience of travel and movement trump safety concerns about human drivers. Why are we being stricter on robots driving?