Delft Mental Health Computing Lab

Social interactions in virtual reality exposure therapy: A proof-of-concept pilot study

Willem-Paul Brinkman — Thu, 14 Jan 2016 10:58:01 +0000

Research on virtual reality exposure therapy (VRET) has demonstrated good treatment efficacy with regards to several anxiety disorders. Yet, there is lack of knowledge about the value of integrating interaction between clients and virtual humans in VRET. Such interaction might prove effective in treating psychological complaints that involve social interactions, such as social anxiety.

A VRET system specifically designed to expose clients with social anxiety disorder to anxiety provoking social situations was applied to 16 and 18 individuals with high and low levels of social anxiety, respectively. Participants engaged in two exposure sessions in several free speech dialogues with virtual humans while being monitored by a therapist.

Participants with high levels of social anxiety reported significantly lower levels of social anxiety three months after exposure to two virtual reality interaction sessions than before treatment (p < 0.01). In the group with low levels of social anxiety, no significant change of social anxiety was reported between pre-treatment and follow-up. Additionally, participants in both groups reported higher self-efficacy three months after treatment than before treatment (ps ≤ 0.001).

These findings indicate that virtual reality technology that incorporates social interactions may be successfully applied for therapeutic purposes.

Morina, N., Brinkman, W. P., Hartanto, D., Kampmann, I. L., & Emmelkamp, P. M. (2015). Social interactions in virtual reality exposure therapy: A proof-of-concept pilot study. Technology and Health Care, (Preprint), 1-9. For paper see pre-print

Meta-analysis: presence and anxiety are positively correlated in virtual reality exposure therapy

Willem-Paul Brinkman — Wed, 07 May 2014 06:36:13 +0000

Considerable research focuses on enhancing the sense of being present in a virtual environment. But why, you might ask? Is this only for fun or does it also serve a therapeutic purpose? The latter seems to be the case. For virtual reality exposure therapy used to treat patients with anxiety disorders (e.g. fear of height, fear of flying, and social phobia), the feeling of presence correlates positively with the amount of anxiety experienced, a key ingredient for this type of therapy.

This conclusion is the main outcome of our meta-analysis published in PLoS ONE. In this study we included 33 articles that collected self-reported presence and anxiety data when a total of 1196 individuals were exposed in virtual reality environments specially selected to elicit anxiety. The aim of the anxiety elicitation in the virtual world is to enable patients get habituated to the thing they fear.

Besides an average correlation with a medium effect size, we also found a number of moderating factors that could influence the strength of this relationship. For example, we found variation between the types of anxiety disorders. Fear of animal studies reported large effect sizes whereas social anxiety studies reported no to small effect sizes. We also found the relationship to be stronger for patients than for non-patients. The meta-analysis also indicated that technology could be a moderating factor. Higher levels of immersive technology coincide with higher correlations between presence and anxiety. For example, the relationship was stronger in studies that used trackers with six degrees of freedom instead of three degrees of freedom. Also, increasing the field of view of displays coincided with a stronger relationship.

This put forward the dominance hypothesis of immersive technology. It suggests that advanced immersive technology dominates the relationship as it reduces the impact of other ‘noise’ factors such as individuals’ characteristics. For example, in the study we saw that the mediator factor of being a patient or not, disappeared in studies that used trackers with higher degree of freedom or displays with a larger field of view. In other words, where these more immersive technologies were used the feeling of presence was more able to explain whether an individual reported anxiety in the virtual environment.

In general the meta-analysis outcomes seems to warrant research in presence improvement, for example more immersive technology. For those interested, the whole study is freely available online.

Ling Y, Nefs HT, Morina N, Heynderickx I, Brinkman W-P (2014) A Meta-Analysis on the Relationship between Self-Reported Presence and Anxiety in Virtual Reality Exposure Therapy for Anxiety Disorders. PLoS ONE 9(5): e96144. doi:10.1371/journal.pone.0096144

Criticism affects job interview stress in virtual reality

Willem-Paul Brinkman — Sat, 26 Apr 2014 17:02:54 +0000

Why does one interview feel like an energy boost, while another one feels like a complete disaster? Or more importantly, what happens to you in a job interview and why? As it turns out, what the interviewer is saying has a significant impact on you. Especially, the way questions are formulated and the replies you get on your answers.

In a recently published study in PLoS ONE, we observed how people respond in a job interview. We asked people to imagine a scenario where they had applied for a position at an IT company, and were now invited for an job interview with an HR staff member. The actual interviews we did in virtual reality. Set within a virtual office room, the virtual HR staff member asked a series of questions and listened to the answers a person gave. We wanted to study people's emotional reaction while being engaged in an interview where they receive a mixture of criticism and compliments. Of course, these people knew that this interview would not result in an actual job offer and therefore had nothing to gain or lose. Still, would this social interaction, stripped from any future consequences, affect people?

Figure 1: Virtual job interview with female (left) and male (right) interviewer.

A key element of a job interview is the notion of being judged or evaluated by someone. This can elicit social anxiety. The question that we therefore wanted to answer is whether the amount of criticism or compliments had a direct impact on people behavior and emotion. We manipulated the ratio of positive or negative questions and replies to applicant's answers. Take for example this positive dialogue interaction: HR staff "What do you know about our company?". Participant : "This company has many brilliant products, from internet service to advanced mobile operating systems, that's why I would love to work here if I can get hired. Because this what I'm good at!". HR Staff: "Well...very good! It seems that you know a lot about this company!". Compare this to a more negative example: HR staff: "Are you sure you know something about this company?". Participant: "Hmmm yes, more or less. Your search engine is world famous". HR staff: "Hmm... [doubting tone] You should have at least visited our website before coming here!". In the experiment we changed the ratio from 0% negative and 100% positive toned questions and replies to 100% negative and 0% positive toned questions and replies. Every 4 minutes we changed the ratio in steps of 25%.

Figure 2: The effect of dialogues stressor on the participants’ heart rate.

When confronted with more criticism, people reported more discomfort and their heart rate increased. When examining their answers, we found that their answers also became shorter. This can be seen as an avoidance strategy of individuals, an attempt to disengage from the social interaction. People also felt less dominant, whereas they regarded the HR staff member as more dominant. And likewise their attitude towards the interview became more negative.

People that reported more social interaction anxiety prior to experiment, were even more affected by the manipulation. Besides the pure scientific curiosity, this also shows the potential application of these findings. People with social anxiety disorder have a strong fear of being judged negatively by others or being embarrassed in social situations. Exposing people to these social situations in virtual reality is being studied as a potential treatment. Manipulating the mixture of compliments and criticism gives therapists the ability to control anxiety eliciting stimuli. Besides psychotherapy, we anticipate psychological stress testing, job interview training or games also to benefit from this control if they need to include conversations that could elicit various degree of stress.

Sense of presence and anxiety during virtual social interactions between a human and virtual humans

Willem-Paul Brinkman — Sun, 06 Apr 2014 07:06:42 +0000

Virtual reality exposure therapy (VRET) has been shown to be effective in treatment of anxiety disorders. Yet, there is lack of research on the extent to which interaction between the individual and virtual humans can be successfully implanted to increase levels of anxiety for therapeutic purposes. This proof-of-concept pilot study aimed at examining levels of the sense of presence and anxiety during exposure to virtual environments involving social interaction with virtual humans and using different virtual reality displays. A non-clinical sample of 38 participants was randomly assigned to either a head-mounted display (HMD) with motion tracker and sterescopic view condition or a one-screen projection-based virtual reality display condition. Participants in both conditions engaged in free speech dialogues with virtual humans controlled by research assistants. It was hypothesized that exposure to virtual social interactions will elicit moderate levels of sense of presence and anxiety in both groups. Further it was expected that participants in the HMD condition will report higher scores of sense of presence and anxiety than participants in the one-screen projection-based display condition. Results revealed that in both conditions virtual social interactions were associated with moderate levels of sense of presence and anxiety. Additionally, participants in the HMD condition reported significantly higher levels of presence than those in the one-screen projection-based display condition (p = .001). However, contrary to the expectations neither the average level of anxiety nor the highest level of anxiety during exposure to social virtual environments differed between the groups (p = .97 and p = .75, respectively). The findings suggest that virtual social interactions can be successfully applied in VRET to enhance sense of presence and anxiety. Furthermore, our results indicate that one-screen projection-based displays can successfully activate levels of anxiety in social virtual environments. The outcome can prove helpful in using low-cost projection-based virtual reality environments for treating individuals with social phobia.

Morina N, Brinkman W, Hartanto D, Emmelkamp PM. (2014) Sense of presence and anxiety during virtual social interactions between a human and virtual humans. PeerJ 2:e337. online

Controlling social stress in virtual reality environments

Willem-Paul Brinkman — Sun, 30 Mar 2014 08:01:47 +0000

Virtual reality exposure therapy has been proposed as a viable alternative in the treatment of anxiety disorders, including social anxiety disorder. Therapists could benefit from extensive control of anxiety eliciting stimuli during virtual exposure. Two stimuli controls are studied in this study: the social dialogue situation, and the dialogue feedback responses (negative or positive) between a human and a virtual character. In the first study, 16 participants were exposed in three virtual reality scenarios: a neutral virtual world, blind date scenario, and job interview scenario. Results showed a significant difference between the three virtual scenarios in the level of self-reported anxiety and heart rate. In the second study, 24 participants were exposed to a job interview scenario in a virtual environment where the ratio between negative and positive dialogue feedback responses of a virtual character was systematically varied on-the-fly. Results yielded that within a dialogue the more positive dialogue feedback resulted in less self-reported anxiety, lower heart rate, and longer answers, while more negative dialogue feedback of the virtual character resulted in the opposite. The correlations between on the one hand the dialogue stressor ratio and on the other hand the means of SUD score, heart rate and audio length in the eight dialogue conditions showed a strong relationship: r(6) = 0.91, p = 0.002; r(6) = 0.76, p = 0.028 and r(6) = −0.94, p = 0.001 respectively. Furthermore, more anticipatory anxiety reported before exposure was found to coincide with more self-reported anxiety, and shorter answers during the virtual exposure. These results demonstrate that social dialogues in a virtual environment can be effectively manipulated for therapeutic purposes.

Hartanto D, Kampmann IL, Morina N, Emmelkamp PGM, Neerincx MA, Brinkman WP (2014) Controlling Social Stress in Virtual Reality Environments. PLoS ONE 9(3) : e92804. doi: 10.1371/journal.pone.0092804 online

Conversations with a virtual human: Synthetic emotions and human responses

Willem-Paul Brinkman — Sun, 23 Feb 2014 08:23:13 +0000

To test whether synthetic emotions expressed by a virtual human elicit positive or negative emotions in a human conversation partner and affect satisfaction towards the conversation, an experiment was conducted where the emotions of a virtual human were manipulated during both the listening and speaking phase of the dialogue. Twenty-four participants were recruited and were asked to have a real conversation with the virtual human on six different topics. For each topic the virtual human’s emotions in the listening and speaking phase were different, including positive, neutral and negative emotions. The results support our hypotheses that (1) negative compared to positive synthetic emotions expressed by a virtual human can elicit a more negative emotional state in a human conversation partner, (2) synthetic emotions expressed in the speaking phase have more impact on a human conversation partner than emotions expressed in the listening phase, (3) humans with less speaking confidence also experience a conversation with a virtual human as less positive, and (4) random positive or negative emotions of a virtual human have a negative effect on the satisfaction with the conversation. These findings have practical implications for the treatment of social anxiety as they allow therapists to control the anxiety evoking stimuli, i.e., the expressed emotion of a virtual human in a virtual reality exposure environment of a simulated conversation. In addition, these findings may be useful to other virtual applications that include conversations with a virtual human.

Qu C, Brinkman W, Ling Y, Wiggers P, Heynderickx I, (2014). Conversations with a virtual human: Synthetic emotions and human responses, Computers in Human Behavior, 34, 58-68. online

An expressive virtual audience with flexible behavioral styles

Willem-Paul Brinkman — Sat, 22 Feb 2014 16:37:40 +0000

Currently, expressive virtual humans are used in psychological research, training, and psychotherapy. However, the behavior of these virtual humans is usually scripted and therefore cannot be modified freely at run time. To address this, we created a virtual audience with parameterized behavioral styles. This paper presents a parameterized audience model based on probabilistic models abstracted from the observation of real human audiences (n = 16). The audience's behavioral style is controlled by model parameters that define virtual humans' moods, attitudes, and personalities. Employing these parameters as predictors, the audience model significantly predicts audience behavior. To investigate if people can recognize the designed behavioral styles generated by this model, 12 audience styles were evaluated by two groups of participants. One group (n = 22) was asked to describe the virtual audience freely, and the other group (n = 22) was asked to rate the audiences on eight dimensions. The results indicated that people could recognize different audience attitudes and even perceive the different degrees of certain audience attitudes. In conclusion, the audience model can generate expressive behavior to show different attitudes by modulating model parameters.

Kang N, Brinkman WP, Riemsdijk BM, Neerincx MA, (2014). An expressive virtual audience with flexible behavioral styles, IEEE Transactions on Affective Computing. doi:10.1109/TAFFC.2013.2297104 online

The effect of perspective on presence and space perception

Willem-Paul Brinkman — Sun, 10 Nov 2013 14:42:58 +0000

In this paper we report two experiments in which the effect of perspective projection on presence and space perception was investigated. In Experiment 1, participants were asked to score a presence questionnaire when looking at a virtual classroom. We manipulated the vantage point, the viewing mode (binocular versus monocular viewing), the display device/screen size (projector versus TV) and the center of projection. At the end of each session of Experiment 1, participants were asked to set their preferred center of projection such that the image seemed most natural to them. In Experiment 2, participants were asked to draw a floor plan of the virtual classroom. The results show that field of view, viewing mode, the center of projection and display all significantly affect presence and the perceived layout of the virtual environment. We found a significant linear relationship between presence and perceived layout of the virtual classroom, and between the preferred center of projection and perceived layout. The results indicate that the way in which virtual worlds are presented is critical for the level of experienced presence. The results also suggest that people ignore veridicality and they experience a higher level of presence while viewing elongated virtual environments compared to viewing the original intended shape.

Ling Y, Nefs HT, Brinkman W-P, Qu C, Heynderickx I (2013) The Effect of Perspective on Presence and Space Perception. PLoS ONE 8(11): e78513. doi:10.1371/journal.pone.0078513
online

Virtual reality experiments linking social environment and psychosis: A pilot study

Willem-Paul Brinkman — Sun, 06 Oct 2013 12:53:38 +0000

Initial studies with healthy subjects and individuals with high risk for psychosis have suggested that virtual reality (VR) environments may be used to investigate social and psychological mechanisms of psychosis. One small study reported that VR can safely be used in individuals with current persecutory delusions. The present pilot study investigated the feasibility and potential negative side effects of exposure to different virtual social risk environments in patients with first episode psychosis and in healthy controls. Seventeen patients with first episode psychosis (FEP) and 24 healthy control subjects (HC) participated in four virtual experiments during which they walked for 3.5–4 minutes in a virtual café, looking for avatars with digits on their clothing. The level of paranoid thoughts, as well as psychological, physiological, and behavioral correlates of paranoid thoughts, were measured in different virtual social risk environments, manipulating two factors: population density and ethnicity of avatars. FEP and HC frequently had paranoid thoughts about avatars. Paranoia in the real world correlated strongly with paranoid thoughts about avatars in virtual environments (Spearman's ρ=0.67 and 0.54 in FEP and HC respectively, p<0.01). FEP kept a smaller distance to avatars than HC. In FEP, but not in HC, galvanic skin response was significantly stronger in virtual environments with avatars of other ethnicity than in the own ethnicity condition. These results suggest that VR is an acceptable and sufficiently realistic method to use in patients with first episode psychosis. VR research may help to increase our understanding of the social and psychological mechanisms of psychosis and to develop new treatment applications.

Veling, W., Brinkman, W. P., Dorrestijn, E., & van der Gaag, M. (2013). Virtual Reality Experiments Linking Social Environment and Psychosis: A Pilot Study. Cyberpsychology, Behavior, and Social Networking. online

The effect of priming pictures and videos on a question-answer dialog scenario in a virtual environment

Willem-Paul Brinkman — Sun, 06 Oct 2013 12:42:41 +0000

Having a free-speech conversation with avatars in a virtual environment can be desirable in virtual reality applications, such as virtual therapy and serious games. However, recognizing and processing free speech seems too ambitious to realize with the current technology. As an alternative, pre-scripted conversations with keyword detection can handle a number of goal-oriented situations, as well as some scenarios in which the conversation content is of secondary importance. This is, for example, the case in virtual exposure therapy for the treatment of people with social phobia, where conversation is for exposure and anxiety arousal only. A drawback of pre-scripted dialog is the limited scope of the user's answers. The system cannot handle a user's response that does not match the pre-defined content, other than by providing a default reply. A new method, which uses priming material to restrict the possibility of the user's response, is proposed in this paper to solve this problem. Two studies were conducted to investigate whether people can be guided to mention specific keywords with video and/or picture primings. Study 1 was a two-by-two experiment in which participants (n = 20) were asked to answer a number of open questions. Prior to the session, participants watched priming videos or unrelated videos. During the session, they could see priming pictures or unrelated pictures on a whiteboard behind the person who asked the questions. The results showed that participants tended to mention more keywords both with priming videos and pictures. Study 2 shared the same experimental setting but was carried out in virtual reality instead of in the real world. Participants (n = 20) were asked to answer questions of an avatar when they were exposed to priming material, before and/or during the conversation session. The same results were found: the surrounding media content had a guidance effect. Furthermore, when priming pictures appeared in the environment, people sometimes forgot to mention the content they typically would mention.

Qu, C., Brinkman, W. P., Wiggers, P., & Heynderickx, I. (2013). The effect of priming pictures and videos on a question-answer dialog scenario in a virtual environment. Presence: Teleoperators and Virtual Environments, 22(2), 91-109. pdf online