HOW CAN MY BIOMETRIC DATA BE USED?
After participating in my survey of VR users, one participant asked, “How can my biometric data be used?” I realized there’s no single article that has collected the future applications and implications of biometrics in XR. While current headsets aren’t capable of collecting this data, it’s important to consider exactly how predictive your biometrics can be.
First, I’ll review what the dilation of your pupils can tell about you. Next, I’ll review seven other biometric data streams and how powerful they all are when combined. Hold on tight.
Your inner thoughts are no longer private
In 2012 Scientific American published an article called Eye-Opener: Why Do Pupils Dilate in Response to Emotional States? and this is how it opens:
What do an orgasm, a multiplication problem and a photo of a dead body have in common? Each induces a slight, irrepressible expansion of the pupils in our eyes.
For more than a century scientists have known that our eyes’ pupils respond to more than changes in light. They also betray mental and emotional commotion. In fact, pupil dilation correlates with arousal so consistently that researchers use pupil size, or pupillometry, to investigate a wide range of psychological phenomena. And they do this without knowing exactly why our eyes behave this way.
“Nobody really knows for sure what these changes do,” says Stuart Steinhauer, director of the Biometrics Research Lab at the University of Pittsburgh School of Medicine. He views the dilations as a by-product of the nervous system processing important information.
It’s a great article from Joss Fong and I recommend reading the entire thing, but here’s why it’s relevant to XR. Pupillometry has been used to assess everything on this list:
That means that an XR headset that can measure changes in your pupil can collect data on your interior mental states. That list contains items like racial bias and mental illness that people are likely not accurately reporting about themselves at all times. Either because they can’t access their true state or they want to hide their actual feelings from others. With pupillometry data, it becomes harder to hide things that others could find socially undesirable.
The Scientific American article goes on to review efforts to predict a person’s sexual orientation using pupillometry:
Researchers at Cornell University recently showed that sexual orientation correlated with pupil dilation to erotic videos of their preferred gender, but only on average and only for male subjects. Although pupillometry shows promise as a noninvasive measure of sexual response, they concluded, “not every participant’s sexual orientation was correctly classified” and “an observable amount of variability in pupil dilation was unrelated to the participant’s sexual orientation.”
Is pupillometry a perfect predictor of sexual orientation, mental illness, and racial bias? No, but how comfortable are you having third parties have access to your information?
Pupil dilation is just ONE biometric feed - let’s add in SEVEN more
Behavior and attitude prediction power increases when multiple data streams are combined. Here are seven more biometric data features that could be collected simultaneously:
- Eye tracking — what has visual salience to you? what path does your gaze take?
- Facial tracking is able to predict your feelings. These seven emotions are highly correlated with certain muscles in the face: anger, surprise, fear, joy, sadness, contempt, disgust.
- Galvanic Skin Response — How intensely do you feel an emotion?
- EEG (Electroencephalography)-How aversive or repetitive is a particular task? How challenging is a particular cognitive task? How attentive or distracted is a person?
- EMG (Electromyography) — how tense are your muscles? It can detect micro-expressions so it would be hard for people to fake reactions.
- ECG (Electrocardiography)-does your pulse increase in response to a stimulus?
This list of biometric data comes from Kent Bye’s Voices of VR podcast 517 interview with behavioral neuroscientist Dr. John Burkhardt on BIOMETRIC DATA STREAMS & THE UNKNOWN ETHICAL THRESHOLD OF PREDICTING & CONTROLLING BEHAVIOR. Let’s examine some of the applications of having multiple feeds of biometric data.
Do you want your boss to know when you’re faking?
Suppose you use VR for work meetings in the future. Your boss could learn if you were laughing at his joke out of politeness or if you thought it was actually funny. If this information is gathered, analyzed, and presented in the future, then you won’t be able to fool the people around you.
The convergence of the data streams of facial recognition and galvanic skin response becomes a predictor to know what people are actually feeling.
When EEG is combined with eye tracking, you can learn if people are actually cognitively processing the items they are looking at. Many companies would salivate to have that knowledge about you.
The combination of all of these data streams will also likely lead to the creation of new predictors of what will influence a person to click or buy. From Kent Bye: “The fields of advertising and brainwashing often borrow from each other’s research so the line between marketing / advertising and what is explicitly control” can become blurred quickly.
While existing VR / AR devices aren’t tracking all of this information, we should all assume that as sensors become smaller and devices become lighter, it’s all going to be possible.
Most of your decisions occur outside of your conscious awareness
Decades of social science research documents that people are not rational or logical. Humans live mostly on autopilot. Hypothetically, a company could collect all of this personal data and what will you get in return? Personalized content recommendations. Is that a worthwhile tradeoff
Existing privacy policies are slim compared to the piles of academic research about the potential for behavioral prediction and manipulation. And that’s just privacy. What about the security concerns where no companies can guarantee that your data won’t be hacked and sold?
What are you going to do to protect your own data?
Will consumers XR devices have biometric sensors? And will consumers be able to disable the sensors inside of the devices they purchase? Please start asking these questions and more for the benefit of all users.
If you want to learn more about the people’s attitudes towards privacy in VR, look here. And if you want to read the full survey results about privacy, harassment, and desire for sociality, click here.
