Friday, November 17, 2017

Autonomous Cars - the Sensor Problem

In May of 2016, a Tesla car under "autopilot" control was involved in an accident that killed the person in the driver's seat.  Inevitably, whenever this accident is mentioned, someone feels the need to show up and say that no one is supposed to mistake autopilot for autonomous control.  If something goes wrong, the driver is responsible, not Tesla.  Nevertheless I find the accident instructive if we want to think about the kinds of problems autonomous cars need to get right all the time. 
In that collision, which occurred at about 4:30 in the afternoon on a clear day, a truck turned left in front of the Tesla which didn't brake or attempt to slow down.  This is the kind of thing that happens every day to most drivers, right?  Should be a priority to program cars to not kill people in this sort of scenario.  The Tesla's optical sensors didn't detect the white truck against the bright sky, and its radar didn't react to it either.
The Tesla went under the truck, decapitating the driver, then drove off the road onto a field near the intersection. 

It's not hard for a human with vision good enough to get a driver's license to see a truck against the sky background.  As I've said many times before, once a child knows the concept of "truck" and "sky" - age 3? - they're not going to mistake a truck for the sky or vice versa. 
Tesla’s blog post followed by Elon Musk’s tweet give us a few clues as to what Tesla believes the radar saw. Tesla understands that vision system was blinded (the CMOS image sensor was seeing “the white side of the tractor trailer against a brightly lit sky”). Although the radar shouldn’t have had any problems detecting the trailer, Musk tweeted, “Radar tunes out what looks like an overhead road sign to avoid false braking events.'"
The way I interpret that statement is that in an effort to minimize the false/confusing returns the radar sees In Real Life (what radar guys call clutter), which is to say in an effort to simplify their signal processing, the radar antenna was positioned so that its "vision" didn't include the full side of the truck.  It shouldn't be impossible to distinguish a huge truck almost on top of the car from large street sign farther away, by the reflected signal and its timing.  Perhaps they could have worked at refining their signal processing a bit more and left the radar more able to process the return from the truck.  The optical sensors have the  rather common problem of being unable to recognize objects.  On the other hand, we've all had the experience of a reflection temporarily blinding us.  Maybe that's the sensor equivalent.  

A recently created electronics industry website, Innovation Destination Auto, a spinoff of Electronic Design magazine, runs a survey article on automotive radars for the Advanced Driver Assistance System (ADAS) market.  There is a lot of work being done on radars for cars.  Radar systems for cars are nothing new; that has been going on for decades.  What's different this time is the emphasis on sensing the total environment around the car. 

It's all about enabling the car to know everything going on around it, which it absolutely has to do.

Electronic devices such as millimeter-wave automotive radar systems are helping to evolve the automobile into a fully autonomous, self-driving vehicle. The Society of Automotive Engineers (SAE) International has actually defined six levels of driving automation, from level 0, with no automation, to level 5, with full automation and self-driving functionality. Different types of sensors within a car, including millimeter-wave radar transceivers, transmit beams of energy off different objects within their field of view, such as pedestrians or other cars, and detect the reflected returns from the illuminated objects. Sensor outputs are sent to one or more microprocessors to provide information about the driving environment for assistance with driving functions such as steering and braking to prevent collisions and accidents.

Multiple sensors are needed for 360-deg. detection around an ADAS automobile. Often, this involves sensors based on different forms of electromagnetic (EM) energy. Automotive radar sensors typically incorporate multiple transmitters and receivers to measure the range, angle, and velocity of objects in their field of view. Different types of radar systems, even different operating frequencies, have been used in ADAS systems, categorized as ultra-short-range-radar (USRR), short-range-radar (SRR), medium-range-radar (MRR), and long-range-radar (LRR) sensors or systems.
The article is "Sponsored By" Texas Instruments, among the largest semiconductor companies in the world, and links to some radar Systems On A Chip they've developed for the automotive market. 

The different types of radar serve different purposes, such as USRR and SRR sensors for blind-spot-detection (BSD) and lane-change-assist (LCA) functions and longer range radars for autonomous emergency braking (AEB) and adaptive-cruise-control (ACC) systems. USRR and SRR sensors once typically operated within the 24-GHz frequency band, with MRR and LRR sensors in the 77-GHz millimeter-wave frequency range. Now, however, the frequency band from 76 to 81 GHz is typically used, due to the high resolution at those higher frequencies—even for shorter distance detection.
It seems to me that these are going to be fairly simple systems with low power transmitters and receivers.  Even the "LRR" (long-range-radar), shouldn't be too demanding on design.  There's a lot of variables I'm sweeping under the rug here, but a car needs to see a few hundred yards at most, and the demands on those radar transmitters and receivers don't strike me as being severe.  

This is just the beginning.  Truly autonomous cars should probably communicate with each other to work out collision avoidance similar to how aircraft do.  It has been proposed.  It should be easier for cars.  Cars can stop.  Aircraft can't.
After the August eclipse, there were reports of horrific traffic jams in several places.  I know I posted about it, as did Karl Denninger and some other people.  What this means is that the road infrastructure is incapable of handling the traffic when it goes above some normal range.  I recall hearing that in a metropolitan area, like around Atlanta where there always seems to be trouble, adding lanes to the interstate costs millions per mile.  No sooner are the lanes built than more lanes are needed.  One of the attractions of autonomous cars is that they should be able to drive higher speeds in denser patterns, getting the effect of more carrying capacity in the highway without adding lanes.  Since they're all communicating with each other, chances of an accident should drop precipitously.  I think that's one reason the governments seem to be pushing for autonomous vehicles. 


  1. I dislike almost all of the "automatic" things on new cars. I dislike the smart keys. All my keys are on one ring except that stupid smart key which is too fat to fit on the ring and I must carry it seperately a guaranteed technique to make sure I lose it some day. Worse if I lose it my copy (which I do keep on my ring of keys) will open the door (bit of course set off the alarm) but it won't start the car. I hate the alarm. I don't want an alarm. But it isn't optional. I don't like the fact that the doors lock automatically, I fully expect that some day it will lock me out. In fact I usually roll the wind down before I get out of the car if I have to leave the key in it for some reason. I hate the little alarm that keeps dinging because my seat belt isn't on. I hate that unlocking the door with the key button turns on my headlight (why? Why do I need the headlight on?) I hate that my dash lights dim when I turn on my headlights. I live in a rainy part of the country and need the headlights sometimes during the day, but then the dash lights are so dim I can't read the dash indicators. I have to turn off the headlights to read whatever indicator I need and then turn the headlights on again while driving along. Don't know what the other drivers think I'm doing...

  2. Right-o.
    This will all totally work out.

    Because when all the vehicles become the internet of things, nothing will ever interfere with that, either by accident or design, and turn it into a glorious flaming single point of failure.

    This is why computer networks never get hacked, and power grids never fail.

    I swear to Buddha, this is technology invented by baby ducks, where everything every day is the first time anything's ever happened to them.

    And BTW, Musk's Monday-morning-quarterback supposing isn't the same thing as knowing what the problem is, but the reality is his car has already violated the first law of robotics, and already has a body count as high as Teddy Kennedy driving drunk. Let's hook this up to busses and such, and see how that goes.

    Tell ya what, how's about these dipshits try automating rail travel, which eliminates 90% of the problems doing it with cars, and after they've got 50 years of accident-free rail travel under their belts, we can start talking about putting it in wheeled vehicles.

    And if they do make the jump to autos, try demonstrating it on every car at the Indy 500 and NASCAR races, and let's see it in action. Like they've done with most of the genuine safety innovations on cars since forever. I'll just wait right here for that to happen.

    Until then, anybody putting one on the road should be shot on sight, on general principles, for attempted murder, and a bounty paid for their hides.

    And +10 to Anonymous, above.
    I'll probably learn vehicle electrical systems in my retirement, simply to thwart and cut every last g**d*** bell, buzzer, dinger, whistle, and "feature" some Detroit 20-something f**khead thought my car needed to tell me to do, or do for me whether I wanted it done or not. I paid for the m*****f*****, and I'll damned well be the one deciding what I want it to do which, when, and whether.

    If I could pay a $1000 cash lump sum to be able to beat the engineer that thought up that happy, rose fertilizer, with a tire iron, I'd brink a sack of cash to the dealer every time I shopped for a new car, and come out ahead of the game, even if I never bought a car.

    Don't even get me started on the auto-tracking GPS/remote disable/nannybox/dataminer 2000 the crapheads have started to put in everything with wheels, the disabling of which kills the engine and voids the warranty.

    That should have gotten someone with a warbird to start napalming corporate HQs in MI, and I'd crowdfund the effort if it happened.

    1. Aesop, don't blame the car companies. It's not the car company employees who point guns at you if you dare to import a new 50 MPG Ford from Mexico. I bet you obey laws even more carefully and completely than most of the liberals I know. What would a jury conclude about your values and motivations from that behavior?

    2. But it IS the car companies who lobby the swill in the District of Corruption to MANDATE these "improvements". And the owned politicians want to be re-elected and fully understand what they must do to receive the appropriate funding for their next campaign.

    3. The human animal has genetic political instincts inherited from their ape ancestors. These instincts evolved to work in political groups of 20-120 individuals. As there is only one set of human genetic political instincts, there is only one human political structure, the direct democracy. Each individual voter's brain controls each individual voter's index finger, which can pull a gun trigger or a voting lever. Military power lies with the bulk of the obedient voters/serfs, who as a group voluntarily pay taxes. When humans form social groups of 100 million, the elites direct the power of millions of allies, not just ten. The results of that system destroy human societies every 200 years.

      I think blaming the car company employees, the legislators, or the voters is misunderstanding the situation. We are stuck with these instincts, but many have learned to intellectually ignore the urges of racism, sexism, etc. Perhaps we can learn to ignore the instinctual urges of politics, too.

    4. @Anonymous 4:24A

      Clearly you have no idea what my driving record looks like, and have thus failed the mid-term for Prognostication 101.

      Oh, and 10 points from Gryffindor for extrapolation based on your prejudices.

      The only one who hears that dog whistle, is, in fact, you.

  3. The first problem with aircraft collision avoidance systems was who operates the system. Yup, the FAA wanted control of the system and the pilots were to listen to the ATC for avoidance instructions. Greyhound operated forward radar systems on its buses for awhile but found that cars ahead of it would slam on the brakes as their radar detectors picked up the radar signals. If 50% of the cars on the road have operating radar how do individual cars differentiate between their signal and the signal of nearby cars and even reflected signals. Having worked on Cat 3 aircraft I can already tell you that autonomous vehicles will likely never be affordable and will never be maintained at "airworthiness" standards. Elon Musk could not afford the feature on his personal car if in fact he had a operable system. If he actually had to get somewhere then he would call Uber. If you think there are a lot of caution and warning lights in the instrument panel now just wait until "autonomous" shows up on the caution and warning panel...and the passengers will be happily texting away with earplugs to cut out the ding, ding, ding...... indyjonesouthere

    1. If 50% of the cars on the road have operating radar how do individual cars differentiate between their signal and the signal of nearby cars and even reflected signals.

      This isn't a hard problem. Aircraft weather radars have to address this. They use a few methods: a unique channel hopping sequence, a unique pulse spacing at some time in the return and a unique pattern of pulses. If they're transmitting on channel X, they only listen on X, and other cars are not likely to be on that channel. They can hop from channel to channel so that on any given T/R sequence they might get interference, but on the next T/R sequence it's not likely. Given something reasonable like 20 channels, there are thousands of hopping patterns available. Plus they can use different pulses sequences.

      Of course, responding to reflected signals is what radars are supposed to do.

      All of these signals, except possibly the longest distance ones, are going to be returning in nanoseconds. The speed of light is about 11.4 inches per nanosecond, and with the majority of things they're interested in being, say, less than 10 feet away, that means very short pulses and responding in under 10-1/2 nsec. The longest distances maybe 100 feet to 150 feet, and that's still under 160 nsec.

      A few years ago, there was terrible mid-air collision in Germany, I believe it was this one, that happened because the air traffic controller told the pilot of the passenger jet to do the opposite of what the TCAS told the crew and they thought the ATC knew better.

  4. So now I'll be bombarded by microwave radiation every time I drive or am near traffic? That can't have any health effects, can it?

    1. No, not with the kind of powers and antenna beamwidths they'll be using.

  5. I;m a software engineer kind of guy. I read recently the world is essentially full of crappy software that doesn't work real well or at all. I don't thing I want to trust my life to someone elses crappy software.

  6. You never win when the goal is to "optimize False Positive vs. False Negative". Sadly, I don't think that we can easily get away from this - there are too many unanticipatable situations.