“Gaming has always been in our radar”, says Affectiva’s co-founder.

Affectiva, a Walthaman based startup born from MIT Media Lab, will release a piece of software so powerful it can deduce many emotions by scanning your face, yet so light it only needs a webcam and can run inside a mobile app.

By analyzing your facial expressions in real time, this emotion-sensing tech can pinpoint your feelings (from simple smiles and frowns to subtler clues of anxiety or sadness), keep track of them and many other stats, and use them for videogame calculations.

What this means is, with this software you can develop, among many other things:


I’m just touching the tip of the iceberg, of course.

When the started organizing their upcoming hackaton for local coders to test their new facial-reading technology, the campaign was developed with the idea of attracting a gender-balanced audience (and they succeeded, half of the attendees will be women). What no one saw coming, though, was a partnership with video game developers was their plan all along, too.

According to Rana el Kaliouby, Affectiva’s co-founder, “games tend to bring out very strong emotions, so we decided that as a company we would enter that in a big way”.

Nevermind, a first-person psychological thriller developed by Flying Molusk, will be the first game to showcase this technology – the more frightened you are, the harder the game will be. This is in tandem with their already-implemented heart rate monitor, another way tech is incorporating emotions in gaming.


This update to Nevermind will only need the webcam in your machine to track your facial expressions and present you with harder challenges the more stressed out you are.

The game’s goal, though, is to force you to keep calmed as you stroll through seemingly draconian tasks – after all, the only way to complete the game is to control yourself. “It’s a stress management tool disguised as a game,” says Erin Reynolds, Flying Mollusk’s founder. By reflecting and rewarding your ability to control your negative emotions, Reynolds hopes mastering the game will lead to practical uses in the real world, say, when you are under heavy traffic or going to a stressful meeting.

Of course, this facial recognition technology is already being licensed for all kinds of purposes, from marketing and advertisement to helping robotic partners and AIs interact with people (eg: helping nurse robots to take better care of patients, or smart houses to accommodate light, music volume and such dynamically).

But gaming is a a very important aspect to Affectiva, and as such, they have released their free API and an Unity SDK for developers to start working ASAP with the technology. The good thing is, since it requires no more hardware than a regular web/phone cam, the only limit is the developer’s imagination on how to implement the data, and the company promotes its joint use with other “Biofeedback” tech like Nest’s thermostat, Amazon’s echo speaker and Pavlok’s shock wristband (right now used to control bad habits).

Remember there’s also Kinect and VR (with all their unusual gadgets, NSFW) and those super realist virtual universes like Tron or the Matrix stop looking like “could happen” to rather be “hey, the cards are dealt”.

Btw, sign ups to the hackaton are still open, the event is March 4th and Affectiva really wants you to take emotion gaming to its fullest.

Courtesy of BetaBoston.

Leave a Reply

Your email address will not be published. Required fields are marked *