Scientists have introduced the very first augmented reality display that uses point cloud data to project holograms into a driver's field of view, allowing them to see an overlay of information on their windshield without taking their eyes off the road.

The researchers created the images with a three-dimensional laser scanner as well as light detection and ranging, or LiDAR, data. The team collected raw LiDAR data from public roads in London, converting it to computer-generated holograms that can alert drivers to objects outside their gaze. For example, a stop sign hidden behind a tree is not immediately visible to a driver, but this new system can make drivers aware of it before they violate traffic laws or endanger other roadway users. The team's study was published on April 21 in Optics Express.

Car accidents claim the lives of 1.35 million people around the world, every year. To put that number in perspective, the lifetime odds of a person in the U.S. dying in a motor vehicle crash is one in 107, making it just fourth on the list of preventable deaths after suicide, opioid drugs and falls.

The vast majority of vehicular accidents and deaths could be fairly easily avoided — 94% of car crashes around the world resulted from human error. And nowadays, drivers are easily distracted by smartphones or electronic GPS systems, according to the paper's lead and corresponding author, Jana Skirnewskaja, in an interview with The Academic Times. Even if the driver looks down at a screen for a millisecond to find directions or see who is calling, the likelihood of a crash can increase.

One existing technology offers a safer way for drivers to see hidden dangers or glance at a map while operating a vehicle. Head-up displays, also known as HUDs, project a transparent image in the driver's line of sight. These HUDs were first used in fighter jets after World War II and get their name from pilots being able to look forward through the windshield, instead of down at the instruments.

Today, HUDs are regularly used by planes and military aircraft but are not often seen in a commercial setting. Skirnewskaja wanted to change that. She worked for automotive suppliers and companies that manufactured driver assistance systems before starting a doctoral degree at the University of Cambridge; the challenges she saw firsthand in the industry inspired her to go back to academia.

"More than half of the proposed solutions would not be implemented by the industry in the near future, [so] my team establishes collaborations between academics and industry to develop real-world applications," Skirnewskaja said. The research team behind this study combines many fields of expertise, from geography to optics to electrical engineering. "It's so valuable to work together in large groups and discover the hidden potential of existing technology," added Skirnewskaja.

The team's computer-generated holograms show an accurate projection of the real-world environment in a car's front window, allowing the driver to see two- or three-dimensional images in their field of view. To the driver, "Holograms were observed floating as a ghost image at a variable focal distance," the authors reported in the paper.

Two of the most important challenges in executing HUDs are choosing the correct distance and choosing the correct position — too large a viewing area and the display becomes invasive, but too far out of the field of view and the driver still has to shift their vision from the road. Typically, images are projected within an eye box in the driver's view, which is a cube 15 centimeters square in volume, but Skirnewskaja and her colleagues employed a larger eye box to improve visibility. "We enlarged the objects, which appear about 30 centimeters in height to make the visual more natural to the driver," she said.

The team's display aligns the size of holographic objects with real-life objects to harmonize augmented reality with the physical world. This natural approach means that dangers can be visualized immediately and accidents can possibly be avoided. In addition to saving lives, the display could also save a significant amount of money. One study estimates that between 2015 and 2030, injuries on the road will come at a cost of $1.8 trillion to the world economy.

The team is continuing to develop its augmented reality display to keep drivers safer on the road. "We don't have the infrastructure for real-time projections yet, but in the future, we will improve moving objects" in the driver's field of view, Skirnewskaja said. "Hopefully, very soon we can present research with dynamic objects like humans crossing the road," she added. She hopes that mobile holographic systems may be able to provide real-time data to smartphone users in the future.

Skirnewskaja noted that HUDs could potentially help the U.K. meet its goal of having more inclusive transportation by 2030. The displays may be able to help visually impaired drivers maneuver through busy city streets, and public vehicles equipped with HUD systems can better provide for people with disabilities. "I already expect that LiDAR will expand in the next few years to help public vehicles navigate urban environments," she said. 

The study, "LiDAR-derived digital holograms for automotive head-up displays," published April 21 in Optics Express, was authored by Jana Skirnewskaja and Timothy D. Wilkinson, University of Cambridge; Yunuen Montelongo, Centro de Investigaciones en Óptica and University of Oxford; and Phil Wilkes, University College London.