Hands-off with the first true hands-free car in the US, and it’s not Tesla

Mercedes invited us out to Santa Monica to drive its EQS in some of the worst possible traffic conditions – and we were happy about it, because it let us test its new DRIVE PILOT system, the first actual hands-free “Level 3” autonomous drive system certified to operate in the US.

Autonomous drive systems are ranked from level 0-5, describing how much capability the system has. In general, levels 0-2 are primarily human-driven, and levels 3-5 are primarily autonomously driven.

Most manufacturers selling cars today have some sort of driver assistance system that can be categorized as level 2. Every other system in the US (except driverless taxis like Waymo/Cruise, which are geofenced level 4 systems) caps out at level 2, which means that drivers still have ultimate liability for anything that happens while they are behind the wheel.

(A note: GM calls “Super Cruise” a “hands-free” system, which does let you drive without touching the wheel, but it is still a level 2 system, and drivers must keep their eyes on the road and are still ultimately responsible for the vehicle while behind the wheel.)

But Mercedes Drive Pilot is a level 3 system – the first validated to work in the US. Mercedes described this gulf between level 2 and level 3 as a “moonshot,” a step change in how people can use their vehicles.

What that means is this is the first system in the US that is actually validated to take full control of the vehicle, under certain circumstances, freeing you to do other tasks than driving. The system has been out in Germany for about a year now (and Mercedes claims it has tallied zero accidents while enabled, so far), but is only just coming stateside.

To do this, the vehicle uses a dizzying array of sensors and redundancies (including three types of rain sensors – as opposed to zero) to ensure that it has a good view of everything around it. It takes advantage of vision, radar, lidar, GPS, ultrasonic, and acoustic recognition of its environment (did we miss any?):

However, level 3 is still conditional automation, which means that it only works within Mercedes’ “Operational Design Domain.” This basically means heavy traffic conditions under 40 mph, with a lead car available, good weather, good road conditions, and in well-mapped areas. Also, a fallback driver needs to always be available to take over control whenever the car tells you to.

It has been known for quite some time that your commute is killing you. The more time you spend driving to and from work in traffic, the more exhaust you breathe, the more engine vibration you experience, the more money you spend on car fuel and maintenance, etc., the worse your life is. This is true in many measurable ways.

So anything we can do to take back some of that time, and some of that attention, to give us something else that we want to do instead, is positive. And if you live in that is choked with traffic because it has no public transit (because real-life cartoon villains conspired to destroy it, as retold in the movie Who Framed Roger Rabbit?, which, amazingly, is based on a true story).

So, given the need for high traffic and good weather, where better to test it than rush hour in Santa Monica where the 10 meets the 405 (seriously, I’ve hit traffic there at midnight on a Sunday night, it’s always bad). This was no choreographed demonstration on a test track, like Seth tried out last year, but a real test with a bunch of real Angelenos doing the same weird things they do every day on the freeway.

Imagine being happy about traffic

The first thing I did was sit in the car and… watch a seven-minute video describing Drive Pilot’s operation and limitations. Mercedes requires that all drivers watch this video before activating the system.

Then I noticed the “augmented reality heads-up display,” which was more extensive than any other heads-up display I’ve tried (and which actually worked with polarized sunglasses, well done Mercedes). The HUD shows a lot of the information that Drive Pilot is basing its decisions on, and the in-car visualization is quite detailed (compared to Tesla’s, it’s not quite as detailed/responsive, but feels less nervous/jiggly). The quality of this visualization, especially being able to see it at a glance in the window in front of you, gives a lot of confidence in the system.

Then, upon getting on the freeway and being stuck at 8 mph, the EQS quickly notified me that Drive Pilot is available by lighting up white lights on the steering wheel. The system is activated by pressing the buttons associated with the lights, then pressing “OK.” After this, the lights turn turquoise, which Mercedes has proposed as a standard “this car is using self-driving” indicator (the car is surrounded by turquoise exterior lighting, as well, so other drivers know).

And once the system is active, I was now officially not driving the car. I could use the car’s screen for whatever function I like, I could read a newspaper (though CA regulations say not to hold anything hard between yourself and the airbag), I could relax (without sleeping).

Mercedes has included several apps, though these basically seemed to boil down to a browser, a media player, YouTube, and some simple games, so far (no Steam or Twitch app, unlike Tesla). We were told the German version has an e-mail app, but that this doesn’t currently exist on the US model. We expect more apps may come later.

So I took the chance to watch some videos in traffic without a care in the world:

I was surprised how comfortable I got, and how quickly

Honestly, it was kind of amazing how immediately comfortable I was with this system. I’ve spent plenty of time in Teslas, driving both Autopilot and FSD, and frankly I generally prefer to drive without those aids turned on. And other manufacturers’ systems all have their pluses and minuses as well.

They have their place (traffic, hours-long straight roads), but they also make a lot of weird decisions. I’ve never had one that I didn’t feel like I had to keep my eyes on at all times (and I do – that’s what level 2 is about).

So when I spent two minutes playing shufflepuck, or 20 minutes (and about two miles of road…) discussing the nuances of the system with my Mercedes rep while the car took care of the driving (if you can call ~6 mph “driving”), I found that I had been paying little enough attention to the road that I wasn’t sure which exits had passed and which were up ahead.

This brings up a potential ownership consideration: You might want to keep the navigation lady’s voice turned on to warn you if you’re going to miss your exit, as one thing level 3 won’t do is change lanes. My Mercedes rep wasn’t sure if the car would issue a fallback warning asking you to take over if your exit was coming soon. This could be nice to see in a future update…

But even with drivers acting as inattentively as myself, Drive Pilot tries to be a courteous road user. With the myriad cameras all around it, it will even attempt to detect lane-splitting motorcycles (it’s legal in California) and move slightly over in the lane to give them a little more room. During my test, one of the motorcyclists even put his hand up to wave and thank “me” for moving over for him. Thanks for earning me some bonus social credit, car.

While level 3 won’t change lanes, Mercedes has added Automatic Lane Change to its DISTRONIC level 2 system, which worked confidently in my test. It will only change lanes when there’s a wide opening, but will make sure you’re in the proper lane for interchanges and exits and will choose faster lanes for passing, then move to the right when not passing (very important in Germany… and almost entirely unheard-of in Los Angeles).

One issue I could see happening is if people forget the difference between level 2 and level 3. While there are a lot of indicators, drivers might get comfortable enough with hands-free level 3 mode that, when activating level 2 mode, they might find themselves treating it like level 3. Mercedes has a lot of redundancies and indicators to remind drivers what mode they’re in, but drivers do tend to pay less attention than they’re supposed to on a level 2 system already, and given how quickly I got comfortable with Drive Pilot, I could see drivers having some lapses while only level 2 Distronic is activated.

Liability – who’s really driving?

Mercedes was careful not to explicitly say that they take liability for things that happen when Drive Pilot is activated. Instead, they repeatedly stated “we stand behind our product.” The question of the trolley problem also came up, and Mercedes dodged it stating that the car wouldn’t get into a situation like that, since it only operates in traffic jams. So we’re still kicking philosophy down the road for the time being (or letting Waymo and Cruise deal with it for now, anyway).

In reality, liability will end up being decided on a case-by-case basis depending on the accident report, how responsive drivers were to fallback warnings, information recorded on the car’s data recorder, and so on. Mercedes also says that drivers must maintain the vehicle to be in proper working condition, including recommended tire pressures, tread depth, kept clean so sensors work properly, and so on (surely every automaker has wanted an excuse to require that their cars stay looking pretty – and Mercedes finally found it).

When Drive Pilot is activated and the car decides that it can no longer drive on its own, it will issue a fallback warning. These warnings are quite obvious – the screen turns red, media stops playing, audio warnings chime, the car tugs at your seatbelt, and so on. At that point, you have 10 seconds to take over. If you don’t, the car will come to a stop as safely as possible (but without changing lanes), turn on the hazard lights and the dome lights, unlock the doors, and call emergency services.

Fallback warnings can happen if any of the previous conditions end (rain starts, sunset happens, etc.), if the fallback driver is no longer deemed as present (eyes looking at the back of the car, seat reclined, asleep, etc.), or if something abnormal intrudes onto the road (pedestrians on the freeway, emergency vehicles, construction zone, etc.).

On the emergency vehicle point, the car has stereo internal microphones to detect siren noises and issue a fallback warning if detected (though you can’t trick them with a YouTube video, as I tried testing):

I got fallback warnings several times on the drive, mostly on purpose. The driver camera at the top of the instrument cluster needs to be able to see your eyes, so looking toward your passenger, into the backseat, or having the steering wheel or any other object in the way will trigger a warning (but you can wear sunglasses – even polarized ones).

I did notice one odd behavior when driving alongside a school bus, wherein the car decided that I needed to take over due to an emergency vehicle nearby. I suspect this had to do with the (non-illuminated) stop signs on the side of the bus – or maybe another nearby Drive Pilot car told us an emergency vehicle was coming, which the system is supposedly able to do. Regardless, it was easy enough to re-activate the system a few seconds later without complaints from the car.

Price & availability

The system will be available on Mercedes’ EQS and S-Class (but who would want that one? it runs on gas, yuck) in the US later this year. Significant hardware is required and can’t be retrofitted, so you’ll need to get a level 3-capable car on purchase. The hardware itself won’t change the vehicle’s price, but using the system requires a subscription, which costs $2,500 for the first year. Mercedes hasn’t yet committed to prices for subsequent years.

Drive Pilot will only work in areas with a high-resolution map available. These maps are precise down to an inch or two, and even account for continental drift.

Mercedes said that the system will initially cover Los Angeles, San Francisco and the Bay Area, Sacramento, San Diego, Las Vegas, and the freeways in between these big cities. A map showing its coverage area will be available from Mercedes soon.