The World's First Level 3 Autonomous Driving Car Appears at Last: What are the Hottest AI/IoT Technologies for Automobiles?

In the world of autonomous vehicle development where automakers around the world compete fiercely, a historic event occurred in July 2017. Audi in Germany announced that it will put a car capable of Level 3 autonomous driving on the market for the first time in the world.

New Audi A8 featuring the world's first Level 3 equivalent autonomous driving capability (Source: Audi AG)

Vehicles at Level 3 and Up Have a Responsibility for Driving Tasks

The autonomous driving capability of all vehicles on the road today falls under Level 1 or 2, which is focused on supporting drivers' safe driving.

Under these levels, even when using the autonomous driving function, the driver is always responsible for driving because the function must be used under the driver's control. However, at Level 3 and up, the vehicle is totally responsible for driving a vehicle in the autonomous driving mode.

Differences in levels of driving automation

This means that drivers will be able to have a rest from driving to check email and converse face-to-face with other passengers. This freedom will be limited to driving in particular areas, such as highways and expressways, for the time being. Even so, however, it is of great significance that drivers are freed from the task of driving. Taking into account the speed at which autonomous driving technology is being developed, there will be even more scenarios for autonomous driving steadily appearing.

At present, however, the driver is indeed legally responsible for the tasks of driving on public roads in many countries. Therefore, Level 3 capability will first be available in regions where relevant laws are already in place. Nevertheless, as a consequence of Audi's declaration to equip production cars with Level 3 capability: the people in charge of each country's regulatory systems will review the legal system and engineers at automakers and related companies will be forced to review their systems and equip cars with Level 3 capability at an early stage.

Three New ICT Technologies Needed for Level 3 Autonomous Driving

In the world of Level 3 autonomous driving, in addition to improving autonomous driving functionality, developing ICT technologies to realize the functionality vital for Level 3 autonomous driving is required. The following introduces three typical examples.

First is driver monitoring, which uses AI technology to monitor the driver's condition. Second is OTA updating, which updates autonomous driving software and other software for many electronic control units (ECUs) installed in vehicles. Lastly is Cloud-to-Car, which collects information from sensors installed in vehicles in the cloud, analyzes collected Big Data in real time and distributes the latest traffic information to vehicles.

Driver Monitoring: Continuous Monitoring of Whether the Driver Can Return to Driving Task

Driver monitoring technology is vital for implementing the driving task takeover function required for Level 3 autonomous driving. At Level 3, when the vehicle driving in the autonomous driving mode becomes unable to continue driving, it should warn the driver to take over control of the vehicle.

For example, Audi AI Traffic Jam Pilot, Audi's Level 3 autonomous driving function, kicks in when the following four conditions are met:

- On expressways or highways with two or more lanes where a central divider, crash barrier, etc. are provided
- When the distance from the vehicle in front and behind, including the adjacent lanes, is close
- The vehicle is moving at up to 60 km/h
- There are no traffic signals or pedestrians within the range of the vehicle's mounted sensors

As such, when the driving speed exceeds 60 km/h during automated driving, the vehicle warns the driver to take over the driving task. If at that time the driver is asleep, there is no guaranteeing that the driving task can be resumed immediately or that the driving safety can be ensured.

In such cases, the system will not hand over the authority to drive the vehicle to the driver, but instead will stop automatically. For this reason, vehicles capable of Level 3 autonomous driving require driver monitoring technology to monitor the driver and always monitor whether they are ready to perform the driving task.

Driver monitoring technology monitors the facial expressions and body temperature of the driver to determine their condition. Specifically, cameras installed in the vehicle and sensors mounted to the steering wheel record facial expressions, body temperatures and pulses. Then, the driver's condition is diagnosed with a decision-making tool based on Big Data analysis using AI to determine whether they are in a condition capable of driving the vehicle.

So far driver monitoring technology has been used by transportation business operators and insurance companies for the purpose of encouraging professional drivers to drive safely. In order to deploy this technology on vehicles supporting Level 3 autonomous driving, developing technology for accurately diagnosing or judging whether the driver can resume the driving task will be required.

OTA Updating: Remotely Adding or Improving Autonomous Driving Functionality

OTA is an abbreviation of Over the Air. It means a function for executing software updates using wireless communication technology such as LTE and Wi-Fi. OTA updating is a standard function for all smartphones, and it is beginning to be used gradually in the automotive industry.

Tesla in the U.S. is a high performance electric vehicle (EV) maker famous for its extensive use of OTA updates. Tesla adopts a strategy to strengthen or enhance the functionality of its vehicles through software updates. The company provided the autonomous driving functionality it is currently using by updating the software.

Tesla announced in October 2016 that it would install hardware required for fully automated driving to all Tesla cars to be manufactured. At that time, the company also indicated its policy for making the fully automated driving functionality available through OTA updating of software in the future.

Tesla Model 3 which began shipping in July 2017 (Source: Tesla, Inc.)

Until recently, OTA updates for vehicles were presumably used for the purpose of updating ECU software. This is because such updates are convenient for automakers as they can update software when a bug is found in ECU software without having to respond with a massive recall.

Going forward, providing autonomous driving functionality through OTA updates, as in the case of Tesla, is expected to increase because it is difficult to provide vehicles capable of Level 3 autonomous driving unless a legal system is established as explained above. If it is possible to implement the functionality in vehicles through OTA updates, they can sell vehicles now with the ability to update functionality later. Then, functionality can be added via OTA updates when a legal system is established.

Given the feature of allowing functionality for optimizing hardware functions later, automobiles may evolve into IoT devices more worthy of updating software via OTA than smartphones.

Cloud-to-Car: Vehicles as IoT Sensors Create Big Data

At Level 3 autonomous driving, there are scenarios where vehicles autonomously perform the driving task, and so driving operation with even greater peace of mind and safety is required. To this end, there is an urgent need to develop a high-precision 3D digital map that is vital for autonomous driving and construct a dynamic map for distributing the latest traffic control and road conditions combined with high-precision 3D digital maps to vehicles. Cloud-to-Car is an essential technology for constructing, updating and distributing this dynamic map.

The basic mechanism of Cloud-to-Car consists of transmitting information from in-vehicle sensors from vehicles to the cloud and distributing Big Data analysis results from the cloud to vehicles.

For example, when an accident happens somewhere, the cloud first collects information--information on the accident and traffic congestion conditions on surrounding roads from various information sources and other information from in-vehicle sensors (position information, speed information, braking frequency, etc.). Next, it analyzes the current situation and predicts future traffic conditions. Then, the cloud disseminates the latest traffic information to vehicles travelling in the relevant area and shows a detour to avoid traffic congestion.

Toward Realizing a Safe, Secure Autonomous Driving Society

In this article, we discussed three technology areas where active research and development will be carried out toward developing vehicles capable of Level 3 autonomous driving. Realizing a safe, secure autonomous driving society requires not only sophisticated vehicles, but ICT technology for allowing users to accept such a society from many different perspectives.

Fujitsu provides the Mobility IoT Platform, a platform for providing related services to connected vehicles and autonomous driving vehicles toward realizing an autonomous driving society. This platform is used for the following purposes: collecting vast amounts of data securely and efficiently by leveraging cutting-edge ICT technology, studying and analyzing the data using cloud AI, managing and distributing high-precision 3D maps required for autonomous driving and securely updating software installed on vehicles via OTA updates.

Please view the video introducing Fujitsu's Mobility IoT 2020 including the Mobility IoT Platform.