New research shows driverless car software is significantly more accurate in adults and Caucasians than in children and Caucasians.
Autonomous car systems have a bias problem, according to a new study from Kings College London. The study examined eight AI-powered pedestrian detection systems used for autonomous vehicle research. The researchers ran more than 8,000 images through the software and found that the self-driving car systems were almost 20% better at recognizing adults than children in traffic and more than 7.5% better at recognizing fair-skinned people than black people . The AI was even worse at detecting dark-skinned people in low light and in dark environments, making the technology even more unsafe at night. Crossing the street with autonomous traffic could become more dangerous for children and people with dark skin.
The study didn’t test the exact same software used by driverless car companies that already have their products on the roads. Still, the software is contributing to growing safety concerns as the cars become more common. This month, the California state government gave Waymo and Cruise permission to operate 24-hour driverless taxi cabs in San Francisco. There have been accidents and the problems are piling up.
Waymo defends itself against the accusation that, after all, not only camera images are used, but several sensors to capture the environment. Waymo is specifically training its autonomous vehicle technology to classify people and respond to human behavior, and is working to ensure its datasets are representative. According to the researchers, a significant part of the technology’s problems in children and black people stems from bias in the data used to train the AI, as it contains more adults and light-skinned people.
Algorithms reflect the biases in data sets and in the thinking of the people who create them. A common example is facial recognition software, which consistently shows lower accuracy on the faces of women, black people, and Asian people. So far, these concerns have not stopped the enthusiastic acceptance of this type of AI technology. The face recognition is already responsible for putting innocent black people in jail.