Sunday, November 19, 2017

Autonomous Cars - the Sensor Problem - Part 2

The first part of this look at the problems with these systems talked about a handful of radar systems that are likely to be on every car.  These are being proposed to work at millimeter-length wavelengths, frequencies very high in the microwave spectrum.  TI proposed 76 to 81 GHz, but think of them as someone offering a solution, rather than a consensus of system designers.

Let's take a look at radar systems, starting with the basics.

Radar is an acronym that has turned into a word: RAdio Detection And Ranging.  Radio waves are emitted by a transmitter, travel some distance, and are reflected back to the receiver, which is generally co-located with the transmitter (there are systems where they can be widely separated - bistatic radars).  Their signals can be any radio frequency, but higher frequencies (microwaves and higher) are favored because as frequency goes up, size resolution - the ability to accurately sense the size of something - gets progressively finer. If you're making air defense radars, it's important to know if you're seeing one aircraft or squadron flying in tight formation.  Higher frequencies help. 

What can we say about systems like TI is proposing?  A wavelength at 78 GHz is 3.84mm, 0.151" long.  The systems will be able to sense features 1/2 to 1/4 of that wavelength in size, and distinguish things as distinct that are only about 8/100" apart.  That simply isn't needed to look for nearby cars, pedestrians, or even small animals in the road.  If you're looking for kids on bikes, you don't need to resolve ants on the sidewalk.  On the other hand, these frequency bands are lightly used or unused, containing lots of available room for new systems. Which they'll need.

The other thing to know about radar is that since it's a radio wave, it travels at the speed of light, like anything in the electromagnetic spectrum including visible light. This means that for ADAS uses, a radar system is going to need to transmit and receive very fast.  The speed of light is roughly 186,000 miles/second; expressed in inches that's 11.8 billion inches/second.  Stated another way, light travels 11.8 inches in one nanosecond.  For our purposes, we can say light travels one foot per nanosecond in air.  Ordinary radars, whether tactical radars or weather radars, are intended to operate over miles; these vehicle systems won't operate over more than 10 or 20 feet, with the exception of something looking forward for the next car, which needs to work over hundreds of yards.  Radar system designers often talk about a "radar mile", the time it takes for a radar signal to go out one mile and bounce back to the receiver.  (A statute radar mile is 10.8 microseconds.)  We don't care about miles, we care about "radar feet". 

A car in the next lane won't be more than 20 feet away, giving some room for uncertainty in the lane position, so it doesn't seem like a system needing to look a lane or two over would care about returns from more than 40 feet away.  In "radar time" that's (40 feet out and 40 feet back) 80 feet at 1 ft/nsec, so the time from transmit to receive is 80 nsec.  A system could put out a pulse, likely corresponding to a few inches, like 0.25 nsec, listen for its return for up the desired distance, then repeat.  It could repeat this transmission continuously, every 80 nsec (plus whatever little bits of time it takes to switch the system from receive back to transmit), but that would require blazingly fast signal processing to handle continuous processing of 80 nsec receive periods and I think it doesn't have to.  Things in traffic happen millions of times slower than that, fractions of a second, so it's likely it could pulse several times a second, say every 1/100 second, listen for the 82 nsec and then process the return. 

For looking a quarter mile down the road, 440 yards each way, that becomes listening for 2.64 microseconds. 

I'm not a "radar algorithms guy", so I don't have the remotest feel for how much processing would be involved, but allowing 1/100 of a second to complete the processing from one 82 nsec interval, and allowing the same or even a little more time to complete processing for a 2.64 microsecond interval doesn't seem bad.  

Asking what sorts of power they'd be transmitting starts to involve more assumptions than I feel comfortable making about what antennas they'd use, the antenna patterns, their gain, and far more detail, but some back of the envelope path loss calculations make me think that powers of "10-ish" milliwatts could work.  That shouldn't be a problem for anyone. 

Chances are you have, or know someone who has, a car with back up Sonar in it: sensors that tell the driver as they get within some "too close" distance to something behind them.  The senors are typically small round spots on or near the rear bumper that measure the distance to things behind the vehicle by timing the reflections of an ultrasonic signal (I've seen reference to 48 kHz) - they're the round black spots on the bumper in this stock photo.

Since the speed of sound is so much lower than the speed of light, the whole description above doesn't apply.  While I don't have experience with ultrasonics, it seems the main thing it gives up is the resolution of the radar, which is already finer than we need.  Ultrasonics might have their place in the way autonomous cars can be implemented. 


  1. Wonder how accurate some of these sensors are when covered with mud, salt and grime, if they work at all. I suspect that sooner rather than later we are going to see one of these autonomous vehicles murder a family of children or a school bus full of kindergarden kids and the legal fallout will end this experimentation for a very long time.

  2. One of the things that I wonder about is signal pollution/discrimination issues. It's not to hard to see how (with a certain amount of handwavium) one of these systems could work in a some density of other cars/objects of concern. What is it going to do to performance and safety when you have a hundred cars operation in the same bands in a non-regular concrete tunnel. Differentiation of my signal from one of my neighbors signal from a superposition of Nth reflections in the area. I can see how we could wind up with something similar to network storms.

  3. I am a radar signal processing guy, and we had the ability to process these signals in real time in 1986. The other thing is (unless I missed it) many of these vehicle radars are FMCW (Frequency Modulated Continuous Wave), where range is determined by differencing the transmitted signal with the return signal and chirping the carrier frequency, thus producing a simple audio-spectrum output proportional to distance. These can be processed via a 59-cent chip or simply an FFT running on a slow (by today's standards) processor. The operating range is determined by the slope of the chirp modulation.

    Dan, above, has the real question to ask in my opinion. Any system with multiple sensors has to make logical decisions based on the realtime assessment of individual sensor health, and I really don't trust the authors of these algorithms to do this right!

    1. Thanks for coming by with real information and input!

      Somewhere between me tossing ideas around in my head to write and then hitting the "publish" button, FMCW radars got lost. They aren't used often compared to pulse radars, but might be useful here. They're used in aircraft radar altimeters, and in some chemical storage tanks to measure the level of the fluid. If there's a car in the adjacent lane in their viewing angle, there would be one received tone and a different tone if there were no car there or it was two lanes over.

      The way I think of the buildup of the ice/sludge/crap on the antenna, it's more likely to cause signal loss - attenuation - than fundamentally not working. That can be adjusted for by increasing transmitter power. I'm not envisioning high gain, narrow transmitter beam systems. That might be wrong.

      A car is a hostile environment, especially under the hood and the closer you get to the exhaust manifold, but if the antennas were mounted behind a fender (plastic/composite) and more isolated from the engine, that's good. And you know that no system can be made that won't fail.

  4. Small radar guided missiles use FMCW. "Speed guns" used by the police are FMCW. They are actually all around because they are cheap.

    Processing algorithms can get quite sophisticated (for instance, is it attenuation due to mud, or is it variation in the cross section of the scene?) and here is where the failures will be in autonomous vehicles, a lack of imagination of the programmer. There must be redundancy, and pretty intelligent pattern recognition. Add in polarization effects and it can get quite wild.

    It's fairly easy to isolate these radars from each other because carrier frequencies can be modulated with a unique digital signature. Nonetheless, I will never trust autonomous vehicles and will not ride in one.

    But, in truth, I really miss my old job of finding and implementing algorithms for separating desired (targets) from background clutter, and finding stable points on the (targets) so that terminal guidance isn't thrown off, causing your very expensive (weapon system) to bury itself harmlessly in the mud many yards from the (target). This job was the highlight of my career, and like many who look back on their lives, I didn't realize it at the time. Oh, well.

  5. From the Agnostic Chair of Applied Physics, Just asking:

    And this will help with vehicles approaching from 90-degree, exactly?

    And how will Car A's receiver differentiate between your signals going out and returning, from vehicles headed the opposite direction coming towards you? Or the car in front of you shooting signals backwards?

    Given the sophistication required, what's the MTBF of a far less complex system on a fighter jet, and the one on an everyday-driver car, which has to work to 6-sigma reliability every day, forever?
    I can't just skip work because the collision-avoidance system on the self-driving car needs downtime for maintenance. Or takes a crap, and deadlines the vehicle and pulls off the road when I'm 200 miles from home. I ask, because I put 170K miles on an Astro-van, and Chevy couldn't even make a gorram alternator (OEM or rebuild replacement) that could survive three consecutive years without totally sh*tting the bed while I was out and about, using technology that was mature in 1920, and yet now I'm going to trust my life to their ability to build and field radars, computers, and software via engineers for same, with components and a constantly-evolving driving problem that's 10∞ times more complicated "engine rotate wheel = charge battery"? Sh'yeah, and here's my flight control plan for jet-powered pigs, while we're on the topic.

    How do you deal with the (multiple) states that already prohibit non-LEO cars from mounting and using a radar generator?
    What is somebody going to do when someone (inevitably) mounts jammers, spoofers, etc., to make your car slow down while they weave in and around you, or boost the signal, or twenty other ways to monkeywrench any system you invent, that's happened to every gadget we've ever seen?
    (This is before we get to deliberate destructive sociopathic folks trying to make it all break and kill a lot of people.)

    If any of this worked for five seconds in the real world, the inventors would be hailed as gods.

    From where I'm sitting it's sheer vaporware, searching for a government subsidy and legislation to force adoption of something with all the real-world practicality and the scientific likelihood of smart guns and ballistic fingerprinting, using Underpants Gnome brochure technology.

    Imagine, if you will, trying to put everyone in Disneyworld into a self-driving wheelchair 24/7/365 for their trips inside the park.
    Call me when someone can do just that successfully for so much as one summer day, and then we can talk about extrapolating such a system to the real world at speeds higher than walking pace.

    My predicted date for viability with the requisite reliability is sometime around 2525 AD, right after the Federation nails down ion drives for Starfleet.

    Being dead at that point for some 480-500 years, you can guess my ultimate interest level.

    But I love the idea of government deciding what technology is best for us poor peons.
    Say, how are low-flush toilets and CFL bulbs working out for y'all?

    And I agree with Dan:
    This will be implemented with all the intelligence of pulling on corn stalks to make them grow, right up until a Hindenburg moment of spectacular failure kills 50-100 kids, and that will be the end of the idea for a century or two.

    If they throttled this idea until they were within a country mile of a viable system, they'd probably do better than they will trying force-feed immature tech and require adoption because the gee whiz salesmen sold them some magic beans, and got out of town before that giant and that huge beanstalk falls onto mom's house.

    People will accept SODs (Some Other Dumb@$$) killing them at much higher rates than they'll tolerate so much as a sprained feeling because of a DAM (Dumb @$$ Machine) using 100M suckers as the beta-testers.

  6. "And how will Car A's receiver differentiate between your signals going out and returning, from vehicles headed the opposite direction coming towards you? Or the car in front of you shooting signals backwards?" That much is easy (and I explained it), but I agree with the rest of the post.

    1. Right - that problem is among the easiest to solve. Channelized systems where cars each use different channels and hop between them with different schemes. Given a relatively small number of allocated channels (20 or 30), the system could be designed for very low probability that they use the same channels.

      If the radar pulses 100 times a second, and is jammed once because two cars end up on the same channel, it doesn't matter. Nobody cares.

      As for the rest of the points, these are not random postings. I really am going somewhere.

    2. I asked that because I'm not a tech guy per se, except when forced to be.
      And IMHO, a limited number of channels shows a decided lack of familiarity with the problem on certain urban freeways.

      I've been on movie sets where the cell phone concentration was so high we killed the relay tower. For a day.

      Annoying with phone calls to one's agent or trying to book tomorrow's gig; a bit more troubling at 65MPH on the interstate.

    3. I've been on movie sets where the cell phone concentration was so high we killed the relay tower. For a day.

      Annoying with phone calls to one's agent or trying to book tomorrow's gig; a bit more troubling at 65MPH on the interstate.

      I believe someone was yanking your chain, because there's no scenario I can calculate that can put that much power into a receiver antenna more than a foot or two away. If a million people transmitted on their cellphones to a receiver 300' feet away, their powers combined might approach a level that could overload a receiver, but I doubt it. It wouldn't physically damage it.

      Something else broke. Now, could they handle a million calls? Nope. The base station "getting confused" isn't killing the hardware.

      The problem you're worrying about happens around you every day, and is managed by the way the systems are designed.

      Ever flown commercial into LAX? Of course you have. Every airplane on approach is using a weather radar that's managing the same frequency problem. It can be done.

    4. No one was yanking my chain; we overloaded the local ability of the network to process calls. I'm not saying anything happened to the hardware physically, just too many pigs trying to suckle on too few teats. It's better now than it was a decade ago, but anyone in a disaster knows what happens when everyone hits the network at once.
      Which is exactly what driving rush hour is, every day, forever: every driver trying to drive at once. And for a bonus, on their gorram cellphones ;) !

      "No service" is fine when you're just standing there.
      Not so fine when you're moving in traffic at 65MPH.

      And there may be the capability at LAX for a finite set of highly-managed/tracked objects moving along fixed routes between waypoints, but the problem isn't 100-200 planes an hour moving towards a fixed target in two straight lines, inbound and outbound, and it requires God Alone Knows how many people besides the pilots to make it work - when it mostly does. No one's even suggesting we automate it "because we can". (At least, no one with a whit of common sense.)
      We won't even talk about the skills gap between the average CATP and the average holder of a non-commercial driving license in a non-automated system, because the annual death toll for the last twenty years tells that tell with precision. Air travel is annoyingly boring, because, at least for commercial American carriers, the risk of not arriving alive is as close to zero as it's likely to ever get. The risk for getting to the Qwikiemart two miles down the road in your jalopie is astronomically higher.

      So I'm just saying that taking a system that "works" because of the intense efforts of thousands of highly skilled CATPs, and hundreds of air traffic controllers, and generations of safety improvements paid for in blood, isn't the best exemplar of how the problem with automating every private vehicle over every foot of travel over every road in America, or anywhere else, is going to go.

      Now let's imagine every passenger passing through LAX was flying every which way simultaneously, in a single-person plane.
      LAX, the system, and most of the fliers, would crash.
      The area within three miles of the airport in all directions would be a scorched and pock-marked impact area.
      And that's what a system dealing with ground vehicle traffic is supposed to be going to handle, with functional flawlessness.
      Color me a wee bit skeptical.

      A mapmaker can handle border conflicts using only five colors.
      But there's only 190 or so countries, and they aren't trying to drive past each other at 70MPH.

      Any urban freeway system, plus the street grid, would require a solution set that'd make rooms full of programmers and engineers eat a pistol just to attempt.

    5. (cont.)
      Anyone who's ever been in a simple straightforward ride at Disneyland/Disneyworld, and had it pack up and crap out while they're on it, with a short, fixed route on a track, knows the problems with doing the whole thing in the real world completely wirelessly automated are damn near insurmountable, for any definition of terms. Or else Disney, with a far higher financial investment in flawless performance at every park would have done it already seven days a week, and they wouldn't have killed a dozen or so people and injured dozens more over the years.

      So I can believe the hype, or my lying eyes.

      Am I interested?
      Do I want research to continue?
      Of course.
      Do I think they could get traffic deaths down from 10K/yr?
      I hope so.
      Do I think they'll ever make a fully automated driving system that isn't an nightmare of biblical proportions in my lifetime?
      Not bloody likely.

      Someone telling me they can manage 150M vehicles driving around is on a par with someone telling me he can predict the shape of clouds to the molecular level, in advance, when they can't even predict the weather.

      But I'm open to evidence of progress, even though it mainly just reveals the lack of respect for the level of complexity in the real world.

      Real life is analog. Or fractals cubed moving in the fourth dimension of time. But never, ever is it digital.

      The only way to make it so is to simplify the problem, by restricting the choices. The model for that is taking the African veldt biopsphere, and turning it into the zoo.

      Zebras and wildebeests sit still for that.

      Drivers will burn your world down for trying it with them.
      And then they'll back up over you a couple of more times, just to make sure they got the job done.

  7. I understand what you're saying - and I play with sensors and radar design professionally a bit myself - but I believe in Murphy ... and the fact that all this present high-tech semi-non-critical stuff in my new-ish vehicle doesn't work as well as advertised in adverse - or sometimes nice - conditions (such as the back-up radar). I recall one time my on-the-fly 4-wheel drive went out right when I needed it. Long walk out of the desert. The technology's different but the negative concept's the same and the consequences will be more serious.

    I think Malatrope has it right - at least for the reasonably near future. There's more to these systems than the technology and the airline/military developers work under different rules than auto mfgs - I'm thinking politicians and commercial management styles and goals.

    My vent selection in a 2015 vehicle has binary positions - 4 - and I can't just select #2 if I'm on #3; I have to punch the button through the entire cycle - while I'm driving. A distraction that I didn't have when I could once upon a time tactically tell the switch position. And the choice I desire is not among those 4. I can see a committee having meetings deciding which 4 options to design into the system - and management deciding 8 choices isn't needed. And heavens forbid the choices aren't binary. This for a simple vent selection ... expanding these issues into the control of an entire vehicle operation?

    Accidents and mishaps happen now, but there's some "satisfaction" perhaps in knowing someone visible messed up. Me, that other driver, the rock off that truck. A different game when the fault comes down to a nameless programmer in Big Corp or some sub-contractor. I recall the attempt to pin the Challenger on some poor engineer at Thiokol. Lawyers and insurance companies will have a field day with these things.

    An interesting field to be working in but I suspect there's a fair bit more development needed before we have fully-autonomous cars. And a bit more to get the "real" bugs worked out (everything works in the lab, eh?). But the coming of such is inevitable.