Biometric technology involving facial recognition could soon be used to determine the moods of people in public and alert security about angry or suspicious expressions, according to research out of Concordia University.
A researcher at the Montreal-based university has developed a computer image processing system capable of classifying human facial expressions and identifying what emotion a person is conveying.
Such a system is ready to be deployed in highly trafficked public areas, says Prabir Bhattacharya, a professor at Concordia’s Institute for Information Systems Engineering.
The technology hasn’t been put into the field yet but could potentially be used in a homeland security role, says Bhattacharya, who also holds a Canada Research Chair in Information Systems.
“It could be used in a public setting like a train station or an airport and security could keep a closer eye on people who seem suspicious,” he says. “Whether certain points like the eyebrows or the lips are expanding or contracting, you can know what sort of emotion they are conveying.”
Can you determine what moods are expressed here? Bhattacharya’s facial recognition technology can.
The system analyzes a facial expression by first measuring the distance between the eyes. Based on that, it is able to map out other regions of the face and set a template. Then, it can process different markers that give away a person’s mood.
By focusing on specific groups of muscles near the eyes, nose and mouth, the system determines mood without requiring a full facial profile. That means less data is needed to determine a profile than other types of facial recognition systems.
Although there are 18 important facial features for determining mood detailed in the research, only seven are necessary.
“[Those are] enough to accurately determine what emotion someone is conveying without getting too complicated,” Bhattacharya says.
Opinion is divided on the value of such “mood detection” technology.
Knowing someone’s mood isn’t really needed when it comes to facial recognition systems, notes Jebb Nucci, vice-president of RFID for Montreal-based RFID ProSolutions. The security vendor installs a combination RFID swipe card matched with facial recognition technology to provide companies with a secure access method.
“We just need to know that a person is the one who should have that swipe card,” he says. “I don’t see why I would ever need to know someone’s mood.”
ProSolutions’ system takes a photo of an authorized person and creates a mathematical algorithm based on different areas of the face. That algorithm is stored on a person’s swipe card. It is then transmitted via antennae to a sensor on a door, and a photo is taken at the same time. If the code on the card matches the one generated by the algorithm in the photo, the person is granted access to the room.
It’s a solution companies use when they have rooms they want highly secured. A range of industries make use of it, according to Nucci. The vendor’s clients include retail, engineering, pharmaceutical, aerospace, hospitals and even colleges.
“If an office has a place with some high-value laptops and they’re having problems with people getting in there and taking things, this is a great solution,” Nucci says.
Authentication could use some extra information about mood in certain situations, suggests Jeff Wacker, an IT futurist with Plano, Texas-based EDS Corp., now an HP company.
“If you’re doing something that requires authentication and are very nervous about it, maybe you should ask that person for further authentication,” he says.
The technology has also been well-proven for surveillance methods.
Although “there could be 1,001 reasons for frowning at the airport,” Wacker says detecting mood can be far more useful to strengthening security than just seeing a face. The concept has been proven by the Israeli security forces who are trained in psychological methods to detect the mood and intent of people in crowds.
Bhattacharya echoes this view.
Take the example of a person entering a bank, he says. A photo could be taken of that person entering the bank and their mood known before they are inside.
“That information could be passed on to the teller and the security guard so they know what to expect,” he says.
Bhattacharya says the technology — developed with the help of a graduate student — is highly accurate. The system has been tested against two large data sets of photos that feature pictures of people in public places both outside and inside. One data set was provided by Carnegie Mellon University in Pittsburg PA. Another set was a collection of photos of the faces of Japanese women.
Japanese women don’t often express their emotions through facial expressions, Bhattacharya says. “So if your algorithm can detect them conveying emotion, then you know it is robust.”
While facial recognition technology isn’t new, using it effectively and accurately can be a challenge, one which the technology out of Concordia seeks to meet by offering accuracy with a restricted data set.
“This is able to do it with a sparser information set than other things I’ve seen, which is good,” Wacker says.
Nucci from ProSolutions says if the technology is capable of improving his ability to identify people he would consider it.
“If a person is really happy and has a stretched, smiley face, we could maybe expand our array to make sure they get in,” he says. “It would be good if we could expand our visual array to allow access.”
But clients rarely complain about their moods affecting their ability to access a room, he adds.
The research results out of Concordia were recently published in Classification of Human Facial Expressions: A Prospective Application of Image Processing and Machine Learning.
