There are known knowns. These are things we know that we know. There are known unknowns. That is to say, there are things that we know we don't know. But there are also unknown unknowns. There are things we don't know we don't know.Naturally, the language generated all sorts of derision from the people of the press who aren't familiar with formalized problem-solving. Most of the engineers I know just quietly nodded in understanding - or thought it was pretty unremarkable. Generally speaking, it's the unknown unknowns that cause your biggest problems, as Rumsfeld said in that speech.
The problem of unknown unknowns is one of the biggest challenges facing autonomous vehicles and driver assistance systems (ADAS). I find it a bit funny that there's a growing undercurrent in the industry that goes, "hey! this is harder than we thought!" Suddenly, the predictions are becoming less rosy. There are questions that maybe they're pushing ahead into road tests a bit too soon.
Trade magazine Microwaves & RF takes a dive into some of the issues and comes away talking about unknown unknowns.
It seems to me that one of the biggest problems in ADAS has been that an assumption has been taken as fact. That assumption is that autonomous cars will be safer than human-driven cars. This assumption is far from proven. It's a popular assumption because it makes sense on some level: we all know about accidents caused by inattention: falling asleep at the wheel, texting or other distractions. We also know about accidents caused by following too closely where a robotic system can have faster reflexes (perhaps aided by vehicle to vehicle communication). Just as I expect my CNC machine to not get distracted and turn a screw too many times, we expect the autonomous car to not text another car or otherwise not pay attention.
What escapes our notice is that humans deal with an astonishing number of variables while driving; from changes in the environment (rain, sleet, dust, degraded lane lines, etc.) to changes in the road itself. Everyone recognizes that younger, less experienced drivers make more mistakes than older, more experienced drivers, but they expect the "younger, less experienced autonomous cars" to make even fewer mistakes.
Maybe the safety record of human drivers really isn't that bad. Maybe driving is very hard and people are really quite good at it.
The RAND corporation did a study (pdf warning) to compute the number of miles some autonomous systems would have to drive to demonstrate the same or better safety as a human driver. The results are surprising. They conclude these systems would need to drive 275 million miles to statistically prove they're equivalent to the safety a human can provide. This plot shows the number of vehicle miles, in millions, vs the requirement and the number of miles driven by the fleets of three testing companies.
Don't say you're going to use artificial intelligence, that just puts the question off onto verifying the AI. As the article puts it:
There’s a general agreement that the only way autonomous vehicles can become a reality is through the application of machine learning. The possible scenarios a vehicle could encounter are basically infinite and it’s impossible to hard-code the algorithms to successfully negotiate all of them. Instead, massive data sets are being recorded along with how humans react to the driving scenarios that are then fed into neural networks.Right now, the industry is kind of in the "Wild Wild West" mode: everything is changing as developers try to get an edge on their competitors. Worldwide safety and testing standards haven't quite been finalized. This all combines to create a situation that demands closer looks at everything. Test engineers have a rough task in front of them.
While this allows design engineers to reasonably tackle the problem of algorithm design, it makes the test engineer’s job much harder. Algorithms are now a black box. This requires more extensive testing because you don’t have a fundamental understanding of the code that can be used to generate test scenarios. Rather, you need to test against almost every conceivable scenario to ensure the algorithms function properly.