Panasonic Develops a Driverless Automated Valet Parking System and a Large Screen AR-HUD

Osaka, Tokyo – Society is increasingly seeking to build a safer, more comfortable automotive society through advancing automated driving and reducing traffic accidents. To achieve this goal, Panasonic has developed a driverless automated valet parking system and an AR-HUD (Augmented Reality Head-Up Display). The driverless automated valet parking system achieves fully automated driving (SAE Level 4) in limited areas^*, while the AR-HUD offers safe, comfortable navigation so that the user can concentrate on driving. These technologies will be installed in Toyota Motor Corporation’s concept car, LQ.

• *SAE Level 4：Highly automatic operation according to the definition announced by Society of Automatic Engineers (SAE) at J3016 (the system performs fully automatic operation in a limited area).

Overview of the developed technologies

(1) Driverless automated valet parking system

This system is achieving driverless automated valet parking without dedicated expensive sensors. It accurately identifies the position of own vehicle by using multiple conventional in-vehicle cameras, sonars, and radar sensors as well as a 2-D road map. The system enables safe parking by finding an empty space and detecting pedestrians who enter the roadway by linking to surveillance cameras in a parking lot and a control server. Driverless automated valet parking accomplishes parking in a very narrow space up to a minimum of 20cm clearance between cars, because it is unnecessary to open or close the doors after parking. An in-vehicle emergency braking system that detects people by using deep learning helps to prevent accidents.

This system is expected to offer the following advantages：

• 1. Achieving driverless automated valet parking system, without dedicated expensive sensors. Reduce traffic accident in a parking lot at low cost and reduce significantly stress related to parking (e.g., finding a parking space, walking a long distance from a parking lot, dealing with adverse weather condition)
• 2. Effectively utilize land by enabling the use of distant parking lots and small parking spaces

(2) Large screen AR-HUD

A large image that gives a sense of depth is superimposed on the actual space in front of the driver’s seat. Based on the information from the vehicle, the AR-HUD indicates route information and obstacle alerts just like physical markings on the road. It presents a high-quality image on a large screen with minimal distortion by using the company’s proprietary optical technologies–refined through the development of audio-visual products–to reduce misalignment between the actual space and the image.

This system is expected to offer the following advantages：

• 1. Assist with safe driving by reducing the movements and focus readjustments of the driver’s eyes by presenting information intuitively
• 2. Reduce the misalignment caused by vibration between the actual space and the displayed image by using proprietary vibration compensation technology, achieving navigation that is easier to see and understand

Panasonic is committed to contributing to achieving safe, comfortable mobility using advanced technologies.

Technologies used for the driverless automated valet parking system

Recently, ADAS^*1 have become more widely used to reduce vehicle accidents caused by drivers’ mistakes. The driverless automated valet parking system achieves safe automated driving in a parking lot by combining in-vehicle ADAS sensing technologies (such as cameras, sonars, and radar sensors) with the infrastructure sensing technologies for security (such as surveillance cameras).

A vehicle with the automated valet parking function installed is linked to an infrastructure-sensing-based surveillance function via a local control server that is set up in each parking lot. The driver can give the vehicle remote instructions to enter or exit the parking lot via a controller such as a smartphone or tablet.

The three main technologies below have been developed for the driverless automated valet parking system.

1. In-vehicle emergency braking system that uses deep learning to detect people

The deep learning network structure was developed with in-vehicle ECU^*2 applications in mind. It accurately recognizes pedestrians with minimal calculations.

2. Technology that estimates one’s own vehicle position by using cameras to capture images around the vehicle

This technology allows the vehicle to drive itself to the parking space instructed by the server. This is realized by identifying one’s own vehicle position based on a comparison between a 2-D road map of the parking lot and road information generated from images captured by multiple cameras installed in the vehicle. Highly accurate vehicle positioning also allows for parking in a narrow space which clearance is 20cm, making it possible to utilize land effectively.

3. Technology that links to infrastructure surveillance cameras with the control server

Infrastructure surveillance cameras monitor the parking lot for empty spaces and pedestrians. They ensure the safe, smooth guidance of the vehicle to the parking space. Pedestrian detection using deep learning is also used by infrastructure surveillance cameras. Because the positions of surveillance cameras are fixed, background subtraction based on fixed point observation is also utilized to help increase the sensing performance. The control server link technology allows the vehicle to avoid pedestrians who may enter into the driving path. In addition to the in-vehicle emergency braking system, this technology safely stops a vehicle through the wireless transfer which is designed as an appropriate wireless communication area and prior traffic control, by sending braking instructions from the control server.

• *1. ADAS：Advanced Driver Assistance Systems
• *2. ECU：Electronic Control Unit

Technologies used for the large screen AR-HUD

The two main technologies below have been developed for the large screen AR-HUD.

1. High-quality large screen display technology that gives a sense of depth with the display distance from 7 m to 41 m

A high-quality, large screen display with minimal distortion is achieved by utilizing Panasonic’s optical technologies, which have been refined through the development of audio-visual products (including the development, processing, and production of digital cameras, surveillance cameras, projectors, and TV sets). The large screen display is equivalent to 230 inches at the center of the display distance (about 24 m). (The actual view is different.)

2. Highly accurate superimposition technology

The vibration compensation technology incorporates Panasonic’s proprietary image stabilization technology, which is used for products such as digital cameras. It reduces misalignment between the actual space and the displayed image that is caused due to vehicle vibration.

The HUD superimposes information on the actual view seen through the windshield. Information is projected in the driver’s field of view ahead. The HUD indicates alerts (such as lanes and signs) and route information in a manner that is easier to see and understand than before.