Facial-recognition algorithms from Los Angeles startup TrueFace are good enough that the US Air Force uses them to speed security checks at base entrances. But CEO Shaun Moore says he’s facing a new question: How good is TrueFace’s technology when people are wearing face masks?
“It’s something we don’t know yet because it’s not been deployed in that environment,” Moore says. His engineers are testing their technology on masked faces and are hurriedly gathering images of masked faces to tune their machine-learning algorithms for pandemic times.
Facial recognition has become more widespread and accurate in recent years, as an artificial intelligence technology called deep learning made computers much better at interpreting images. Governments and private companies use facial recognition to identify people at workplaces, schools, and airports, among other places, although some algorithms perform less well on women and people with darker skin tones. Now the facial-recognition industry is trying to adapt to a world where many people keep their faces covered to avoid spreading disease.
Facial-recognition experts say that algorithms are generally less accurate when a face is obscured, whether by an obstacle, a camera angle, or a mask, because there’s less information available to make comparisons. “When you have fewer than 100,000 people in the database, you will not feel the difference,” says Alexander Khanin, CEO and cofounder of VisionLabs, a startup based in Amsterdam. With 1 million people, he says, accuracy will be noticeably reduced and the system may need adjustment, depending on how it’s being used.
Some vendors and users of facial recognition say the technology works well enough on masked faces. “We can identify a person wearing a balaclava, or a medical mask and a hat covering the forehead,” says Artem Kuharenko, founder of NtechLab, a Russian company whose technology is deployed on 150,000 cameras in Moscow. He says that the company has experience with face masks through contracts in southeast Asia, where masks are worn to curb colds and flu. US Customs and Border Protection, which uses facial recognition on travelers boarding international flights at US airports, says its technology can identify masked faces.
But Anil Jain, a professor at Michigan State University who works on facial recognition and biometrics, says such claims can’t be easily verified. “Companies can quote internal numbers, but we don’t have a trusted database or evaluation to check that yet,” says. “There’s no third-party validation.”
A US government lab at the National Institute of Standards and Technology that functions as the world’s arbiter on the accuracy of facial-recognition algorithms hopes to provide that external validation—but is being held up by the same pandemic that prompted the project.
Patrick Grother, a computer scientist who leads NIST’s facial-recognition testing program, says his group is preparing tests to quantify how accurately algorithms identify people wearing masks. NIST plans to digitally add masks to its existing stockpile of photos and test algorithms previously submitted to a test that involves checking whether one photo matches another, similar to the job of a border guard checking passports. Later, it will invite companies to submit new algorithms tuned for face masks. But Grother says the timing of the project is uncertain, because NIST has reduced staffing due to the Covid-19 crisis.
Chinese and Russian companies tend to dominate NIST’s widely watched leaderboards for facial-recognition accuracy. Lighter privacy rules and wider acceptance of surveillance make it easier for those companies to gather the data and operational experience needed to improve facial-recognition algorithms. This year, companies from China and Russia were first to claim their products are ready for a world of half-covered faces.