Technology helps self-driving cars learn from their “memory”-ScienceDaily

Researchers at Cornell University have developed ways to help self-driving cars create “memories” of previous experiences and use them in future navigation.

A car using an artificial neural network is in a certain state of seeing the world for the first time, even though it has no memory of the past and has driven a specific road many times.

Researchers have created three simultaneous papers with the goal of overcoming this limitation. The two were announced in the minutes of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2022) in New Orleans, June 19-24.

“The basic question is, can we learn from repetitive traversals,” said Kirian Weinberger, a senior author who is a professor of computer science. “For example, a car can mistake a strangely shaped tree for a pedestrian when the laser scanner looks at it from a distance, but when it gets close enough, the category of objects becomes clear, so the second time. Hopefully the car has learned to recognize it correctly, even in the fog and snow, with the exact same tree as it passes by. “

The group, led by PhD student Carlos Diaz-Ruiz, has driven a car equipped with a LiDAR (Light Detection and Ranging) sensor 40 times in 18 months along a 15 km loop in and around Isaka. , Edited the dataset. Traversal captures different environments (highways, cities, campuses), weather conditions (sunny, rain, snow), and time of day. The resulting dataset contains over 600,000 scenes.

“We deliberately reveal bad weather, one of the key challenges for self-driving cars,” says Diaz-Ruiz. “If the streets are covered with snow, humans can rely on memory, but without memory, neural networks are at a great disadvantage.”

HINDSIGHT is an approach that uses a neural network to compute the descriptor of an object as the car passes by. Next, is the group SQuaSH? These descriptions, called the (Spatial-Quantized Sparse History) feature, are compressed and stored in a virtual map, such as “memory,” stored in the human brain.

The next time the self-driving car crosses the same location, you can query the local SQuaSH database for all LiDAR points along the route and “remember” what you learned last time. The database is continually updated and shared between vehicles, enriching the information available to perform recognition.

“This information can be added as a feature to LiDAR-based 3D object detectors,” said PhD student Yurong You. “Both detectors and SQuaSH representations can be jointly trained without additional monitoring or human annotation. This is time consuming and labor intensive.”

HINDSIGHT is the predecessor of MODEST (Mobile Object Detection Using Ephemera and Self-Training), an additional study conducted by the team, and goes a step further to allow cars to learn the entire perceptual pipeline from scratch. increase.

HINDSIGHT assumes that artificial neural networks have already been trained to expand with the ability to detect objects and create memory, while MODEST assumes that artificial neural networks in the vehicle are completely exposed to objects and streets. I assume it has never been done. By traversing the same route multiple times, you can see which parts of the environment are stationary and which parts are moving. Slowly learn what constitutes other traffic participants and what is safe to ignore.

The algorithm can reliably detect these objects. You can reliably detect this object even on roads that were not part of the first repeated traversal.

Researchers have found that this approach significantly reduces the cost of developing self-driving cars (which still relies heavily on costly human annotated data) and how to navigate the most used locations. I hope that learning will improve the efficiency of self-driving cars.

Story source:

material Provided by Cornell University.. Original written by Tom Fleischman, courtesy of Cornell Chronicle. Note: Content can be edited in style and length. Technology helps self-driving cars learn from their “memory”-ScienceDaily

Back to top button