emotion technology - emotions and virtual reality - vr emotion technology

Why Emotion Technology Is Important

In Article, Artificial Intelligence (AI), Health and Well Being, Virtual Reality by ScottLeave a Comment

About this article:

Emotions dictate how we perceive the world, the decisions we make and the actions we take. In this article I look at the importance of being able to measure human emotional response, and a company, emteq labs, that uses AI, and sensory fusion in Virtual Reality to both measure and manage human emotional response.

Company Info & Tech Check:

emteq labs

  • U.K.
  • ~10
  • 1.5M
  • Tech4Good (Mental Health)
  • GameChanger (AI,VR)
  • Digital
  • Connectivity

Can Technology Interpret Emotions Via Our

Tiny Involuntary Expressions

Ask anyone in my family about my ability to play a prank on them, and they will each tell you of my, apparently, very obvious "tell".

It seems that, not matter how hard I try to wind up a member of my family, my face always lets me down, with a microscopic tell-tale movement. This fleeting twitch, becomes amplified the more I am aware of it, and the more I try to control it.

However, now I know this (and I'm hoping my family aren't reading this), before I start the build up to any prank, I try to "flood the system" with artificial over-stimulus to try and hide the tiny, involuntary response my nervous systems seems to be adamant on betraying me with.

Emotions are complex things, that can be expressed and interpreted differently all over the world. Whilst a study from California identified 27 self-reported emotional types, these can mostly be grouped into the six (almost) universally recognised high-level categories of sadness, anger, surprise, fear, disgust and happiness.

Since the late '70s, researchers have been  documenting how emotions relate to facial movements in a framework known as the Facial Action Coding System (FACS). Explaining FACS, one of the lead researchers Dr Paul Ekman describes it as "a comprehensive, anatomically based system for describing all visually discernible facial movement".

The principles behind FACS are used broadly today. From interrogators, politicians, and researchers, to casinos, creators of entertainment and retailers. Whilst convenient, this type of measurement only scratches the surface of our emotional response. In fact, Dr Ekman's own definition highlights this, "discernible facial movement".

Complexities of Emotions

emteq labs virtual reality and emotions - emotions in vr - emotion research tools - multi-modal sensory information

Our faces are capable of conveying more than 7,000 varieties of expressions, yet our true emotional response to any given stimulus is a combination of several physiological changes or variations, including heart rate, blood flow, pupil dilation, eye movement, skin conductivity, breath, brain activity and muscle activation.

In other words, our faces (actually, our heads) provide an ideal vantage point to observe emotion. Perhaps, in light of Darwin's work on emotion, it could be argued that emotions have in someway played a direct role in Homo Sapien's evolutionary path..?

Either way, to get a true understanding of emotions, it would appear that you need to observe more than facial twitches. (My family, take note!)

But first of all, why is emotional response something I am interested in writing about on a technology website?

Emotions and Technology

There are many reasons why technologists would be interested in emotions.

There are essentially three parts to an emotion; subjective (how we experience it), physiological (how our bodies react) and expressive (how we behave).

Emotions can therefore impact our decision making, our well-being, and the well-being of others, and with that, emotionally aware technology has the potential (like most technologies) for both good and nefarious use.

On one hand, there are more commercially or security focused use cases. For example:

  • Game designers that could create more engaging, rewarding and personalised gaming experiences.
  • Artificial Intelligence could be trained to more deeply understand, and therefore, better respond to human emotion.
  • Retailers could provide more intuitive experiences, delivering things the shopper really likes, with less friction.
  • Governments could better understand the mood of their citizens.
  • Improved security interventions, preventions and reactions.

On the other hand, technological understanding of emotional response stands to play a significant role in the management of many problems that seem to only be accelerating in our societies.

  • Helping rehabilitate trauma victims.
  • Helping combat the rising tide of mental health issues.
  • Overcoming fears and anxieties.
  • Improved tele-health.
  • Improving digital human to human communications.

As mentioned in a previous article, 1 in 4 people in the UK will suffer from mental health related illnesses. Further, in the EU 3.7% of deaths in 2016 were accountable to mental health, and in 2018 13.5% of all hospital beds in the EU are allocated to psychiatric patients. And I feel, that given the challenges of our "new normal", the strain on our collective mental health will only increase.

So let's meet a company that has developed technology that is paving the way to a better understanding of the human emotional condition, emteq labs.

AI, Sensory Fusion and VR to

Understand Human Emotions

Our emotional state and response can impact our physical and mental well-being, our decision making capabilities and our interactions with the world around us. Therefore human emotional response plays a critical part in everything we do and experience, yet it is still remarkably difficult to personalise the understanding in a way that can help alleviate suffering or improve well-being for individuals, at scale.

It can be difficult enough to build an understanding of emotional action/reaction even for highly experienced professionals in controlled environments.

Now imagine how much harder it is when the environment is not so controlled, or the analysis needs to happen remotely, or at home.

Virtual Reality a Mental Health Tool

By hijacking two of our most dominant senses, visual and auditory, VR has the power to fool the mind in very powerful ways. It can transport people to new worlds, or place them in specific scenarios.

The NHS reported a study that appeared to show how VR could help people conquer their fear of heights. Of the 100 participants, those who were subjected to the VR therapy showed almost a halving of their fear when scored on the Heights Interpretation Questionnaire (HIQ).

In this study, the participants were placed into a controlled virtual environment and instructed on how best to manage their psychophysiological response - in other words, their emotional response.

Virtual Reality is showing a great deal of promise in helping manage mental health issues, however there is currently a lack of sophistication that makes it hard to control the amount of therapy the subject receives, as well as the specific targeting of that therapy. Additionally, relying on verbal feedback or facial analysis, the current VR approaches lack accuracy.

Anxiety issues are now a massive problem globally. Even before the era of COVID19 it was estimated that 300 million people worldwide suffer from clinical anxiety disorders. It’s clear that the combination of exposure therapy using VR simulations and objective measurement through biometrics offers one of the most powerful tools to improve outcomes and even enable self-treatment, a mission that we in emteq labs have set out to deliver on.

Like almost any other type of medical treatment,  from paracetamol to oncology, it is important you administer the correct treatment, at the right time in the correct dose.

Virtual Reality therapies are not different. For those that treating psychological or psychophysiological conditions, the measurement of the individual's emotional response is critical for the fine tuning and adaptation of the therapy.

Understanding Emotions in Virtual Reality -

emteq labs

Emotions are a combination of multiple bodily responses, so if you want to understand emotions, you need to be measuring as many of those responses as you can.

emteq labs have pioneered a technology that uses multiple types of sensors embedded into a virtual reality headset to simultaneously collect data that can be aligned to the virtual reality experience.

The solution combines Facial Electromyogrpahy (fEMG) sensors which can detect even the smallest facial muscle activations that would be imperceptible to the naked eye, with heart rate, skin conductivity, eye tracking and motion sensors.

emteq labs' headset to measure emotional response in VR - emotions in Virtual Reality
emteq labs' Virtual Reality Headset to Measure Emotion
Using Artificial Intelligence, the emteq solution learns the relationship between the multiple input data sources and aligns them to the specific visual and audio stimulation that was present at that time.
Through the combined understanding of micro facial expression, eye movements, heart rate and changes in skin conductivity, emteq labs can determine the level of valence and arousal at any point in the virtual reality expereince
  • Valence – a measurement of positive or negative emotion
  • Arousal – a measurement of physiological activation, engagement, excitement

This novel approach to emotional understanding can benefit both researchers and users.

From a research point of view, it provides a solution that is a close as possible to a real-world scenario, without the need to re-create potentially difficult or dangerous situations. This is also known as ecological validity.

emteq labs' headset to measure emotional response in VR - emotions in Virtual Reality

Another way of thinking about the importance of ecological validity is this - If you were researching the impact of alcohol on the ability to drive, you wouldn't ask your participant down a few vodka shots then get behind the wheel to see what happens. No, you'd want a controlled, safe environment that gets as close as possible to the real thing.

Real-time adaptation

Aided by real-time feedback of the users' emotional response to the current virtual scenario, the experience can be dynamically modified to adapt to the response of the user.

This could mean the experience could be made more, or less, challenging or stressful depending on the level of response, allowing a treatment programme to dynamically progress as the user's response improves.

In Summary

The ability of emteq labs to measure human emotional response in a virtual environment offers some tremendous potential.

In today's modern world, more and more people need support with their own mental health challenges. Yet access to the professional support the so much need is becoming harder and harder, due to shortages in expertise but also difficulty in physically attending clinics in the first place.

A portable solution like this could be sent to users to self-administer in their own homes, without the risk of over exposure and with the ability for trained professionals to monitor progress and adjust treatment remotely.

From an entertainment point of view, the level of excitement or challenge in a virtual reality experience could now be tailored to meet the needs and capabilities of the individual.

White Paper

To find out more about the emteq labs solution, enter your email address below to download their white paper, or head over to their website here.

To get in touch directly via email click here.

Find Out More

Enter your email address to download the emteq labs white paper
Download
emteq labs emotional intelligence white paper

Leave a Comment