Can we use games to control our mood?

Video games are very complex experiences. They involve paying attention to virtual objects, vehicles and characters; all working together to offer both a challenge and a narrative. As a result they provoke a wide range of emotions. Games can make us anxious and scared, but also relaxed. They can help change boredom into excitement, and sadness into happiness. They seem to be able to change how we feel. But how reliable are they? Can we use games to switch our emotions at will? Can we use them to control our mood?

One area of research that can help us understand the relationship between video games and emotions is Mood Management Theory. According to it, people tend to use media to improve how they feel. They can do this, for example, by seeking more positive and arousing content. Media can also distract people from the emotions, thoughts and memories that are making them feel bad. Research on this area has found some support for the potential of video games for mood repair. Let’s analyze this studies more closely.

Evidence suggest that there’s a positive relationship between how many actions a player executes in a game and how good he or she feels. For example: Playing a Wii boxing game with motion controllers was followed by a reduction in negative mood, which didn’t happen for those who played a flash boxing game with less actions available and through the use of a keyboard (Chen & Raney, 2009). This seems to suggest that titles involving additional and more realistic user actions are better for the reduction of negative mood. But there are additional differences between both games (like graphics and sound quality) that could explain the users’ change in mood. Another experiment, however, found more conclusive results. People who watched a warplane simulator (Lock-on: Modern Combat) in autopilot showed less improvement in affect than those who had to control the speed and direction of the plane (with a flight stick and a throttle), which also showed a bigger increase on affect than those who additionally had to control landing gears and flaps as well as air, wheel and parachute brakes (with a keyboard and a mouse) (Bowman, 2012). This indicates that, to promote positive affect, some demand for the intervention of the user is better than none. But when the task becomes too demanding, it starts having negative effect on mood. So if you need to feel better after receiving some bad news, playing a game would probably be more effective than just watching an un-narrated gameplay video on YouTube. However, this wouldn’t necessarily apply to reviews or “Let’s play” videos. After all, these type of content includes expert analysis as well as jokes made by successful entertainers, increasing their chances of influencing the user’s mood. Also, we wouldn’t recommend playing games with unusually complex controls and interfaces like “Dragon Ball Z: Budokai Tenkaichi” or “Master of Orion”, unless you already know how to play them pretty well.

Playing a confusing game might not be the best way to improve your mood.

A recent study could help us understand why interactivity tends to make people feel better. This experiment found that playing Pac-Man was better at reducing depressive mood than watching a clip of the game or just waiting (Rieger, Frischlich, Wulf, Bente, & Kneer, 2014). Interestingly, it was also better at reducing involvement towards a previously seen sad movie clip (Rieger, et al. 2014). This result is congruent with (but it doesn’t prove) the idea that interactivity improves mood by distracting the user from negative content. So if you find yourself experiencing a deep feeling of sadness (and social support isn’t available at the moment), try playing a game that can capture your attention for long periods of time. Whether it is a puzzle or an online racing game, it might help you avoid negative thoughts or memories that would only have made you feel worse.

Research has also found support for a positive effect of interactivity on arousal. People that chose to spend a break playing a video game instead of just waiting, experienced a higher increase in self-reported arousal (Reinecke & Trepte, 2008). It’s possible though that those who choose the game had better expectations towards the improvement of their mood, so a placebo effect can’t be ruled out as an explanation. Nevertheless, another study found more solid evidence of the relationship. This experiment found that playing Pac-Man was better than watching a clip of the game at increasing self-reported arousal but not electro-dermal activity (a physiological measure of arousal) (Rieger, Frischlich, Wulf, Bente, & Kneer, 2014). This suggest that interactivity can make users’ feel more energetic, but won’t necessarily trigger the physiological responses associated with it. Interestingly, studies on movies have found that sad content is an effective way to increase arousal, and that it works better than comedy (Rieger, Reinecke, Kneer, Frischlich, & Bente, 2013; Rieger, Bowman, Frischlich, & Bente, 2014). So playing an emotional interactive experience like the ones provided by “Final Fantasy IV” or “Shadow of the Colossus” could be a good way to make an under-stimulated individual feel more excited. However, if someone is feeling both bored and depressed, the use of sad content could make things worse. In this case, games with less emotional storylines like “Mario RPG” or more action oriented like “Marvel vs. Capcom” or “FIFA” would be a better choice.

Emotionally intense content seems to make people more excited.

As we have seen, there’s enough evidence to believe that interactivity has a positive effect on people’s mood. But this doesn’t necessarily mean video games are always reliable as a source of mood repair. For example: While Pac-Man has been shown to decrease depressive feelings, it hasn’t been found as effective in the promotion of a happy mood (Rieger, Frischlich, Wulf, Bente, & Kneer, 2014). On the other hand, titles like Hitman: Blood Money (a third person stealth game), Call of Duty 2 (a war-themed first person shooter) or Madden: 2007 (an American football game) aren’t particularly useful for the reduction of sadness or hostility (Ferguson & Rueda, 2010). One way to explain this is that other game variables are affecting how the users feel. Maybe Pac-Man is challenging enough to distract people from negative memories, but the animation and sounds aren’t enjoyable enough to promote positive affect. And while Hitman and Call of Duty are known for their action packed missions, they also tend to include sad music and storylines that we wouldn’t recommend for players feeling depressed.

Some games can be surprisingly ineffective at reducing negative mood.

One of the lessons we can obtain from these studies is that providing different and interesting opportunities to interact with the game is a good way to keep the players happy and excited. Megaman X, for example, achieved this by letting the players jump from walls as well as charge their weapon. In Tenchu: Stealth Assassins, the user could hide behind a wall to avoid detection, turning an otherwise passive element like the environment into an important tool. Actions doesn’t even have to be important for the in-game objectives. Creators of Grand Theft Auto V, for example, included simple features like being able to switch between radio stations, turn the lights of the car on and off or even make rude gestures to other drivers as a way to prevent the players from becoming too bored during long road trips. The demand for user intervention should be limited though. As we have seen, overwhelming controls can have a negative effect on mood. Nevertheless, we should remember that this didn’t prevent the first person shooter “Arma II” to gain a considerable amount of followers. This reminds us of how important it is to know your target audience, the experience they want and what they are willing to overlook in order to obtain it.

Simple actions, like playing with the lights of a car, prevent users from
switching their attention away from the game.

A final advice would be to avoid any situation that takes the control away from the user. A good example are quick-time events. This technique (introduced in the Shenmue series and included in recent games like the new Tomb Raider) consists of triggering a complex set of actions (like a martial arts move) by pressing a single button at the right time. Quick-time events are used to insert cinematic scenes during gameplay without completely eliminating the intervention of the user. One way to improve this technique would be to use a more natural approach, like pressing right or left to avoid a falling object instead of just pushing the “action” button. This would help smooth the transition between real-time action and “interactive” cut-scenes.

Video games seem to be a good way to change how you feel. Experienced players probably even know what titles and genres match their mood repairing needs. But these effects are probably only temporal and do not substitute social support or professional counseling. Games are not a long term solution. At best, they can help you take a break from angry thoughts, depressive feelings or even traumatic events. They might give you the space you need to see things more clearly. To set a course of action and start working on your problem.

References:

Bowman, N. D., & Tamborini, R. (2012). Task demand and mood repair: The intervention potential of computer games. New Media & Society, 14(8), 1339-1357. doi: 10.1177/1461444812450426

Chen, Y., & Raney, A. A. (2009, May). Mood management and highly interactive video games: An experimental examination of Wii playing on mood change and enjoyment. Paper presented at the annual meeting of the International Communication Association, Chicago, IL.

Ferguson, C. J., & Rueda, S. M. (2010). The hitman study. European Psychologist, 15, 99–108. doi: 10.1027/1016-9040/a000010

Reinecke, L. & Trepte, S. (2008). In a working mood? The effects of mood management processes on subsequent cognitive performance. Journal of Media Psychology: Theories, Methods and Applications, 20(1), 3-14. doi: 10.1027/1864-1105.20.1.3*

Rieger, D., Bowman, N. D., Frischlich, L., & Bente, G. (2014, May). “I’m pumped, but I don’t feel like it!” The differential effects of affect and arousal regulation on mood repair and recovery. Paper to be presented at the annual meeting of the International Communication Association, Seattle.

Rieger, D., Frischlich, L., Wulf, T., Bente, G., & Kneer, J. (2014). Eating Ghosts: The Underlying mechanisms of mood repair via interactive and noninteractive media. Psychology of Popular Media Culture.

Rieger, D., Reinecke, L., Kneer, J., Frischlich, L., & Bente, G. (2013, June). Media induced recovery: The effects of positive versus negative media stimuli on recovery experience, cognitive performance and vitality. Paper presented at the 63th Annual International Communication Association Conference. London: UK.

Are skilled players more likely to experience flow?

What is the relationship between skill, challenge and flow?

One of the most attractive aspects of video games is the possibility to experience flow. This happens when a user engages in an activity that fits his set of skills while still maintaining a certain level of challenge. The individual then becomes so focused in the task that the outside world slowly starts to disappear. Flow can make people feel better about themselves, have fun and forget about their problems. This makes it one of the most desirable objectives for both gamers and developers. But what attributes are expected to produce flow?

Most experts think there is a strong relationship between how skilled a player is, the amount of challenge he or she faces and the experience of flow. But finding what levels of challenge and skill can produce flow has not been that easy. For example: Since flow is associated with reward, then higher levels of difficulty should have a positive effect on flow, right? Well, not exactly. A recent experiment found no significant effects of difficulty on flow in a Tower Defense game (Schmierbach, Chung, Wu & Kim, 2012). Additionally, two experiments involving Wii games showed that the level of perceived challenge didn’t affect flow on “Trauma Center: New Blood” (a surgical simulator that uses haptic feedback on hand-held controllers) or “Need for Speed” (a racing game played, in this experiment, with a steering wheel) (Jin, 2011). It did, however, made the surgical instruments feel more real (physical presence) and increased the sensation of being inside a race (spatial presence), which had a significant effect on flow (Jin, 2011). This seems to suggest that challenge can have an indirect effect on flow. The high number of threats a player has to face in Ninja Gaiden, for example, could increase the importance of in-game events, making everything feel more real. This would make victories more rewarding but could also increase engagement, making the players forget about their problems.

Ninja Gaiden’s difficulty could have increased both presence and flow.

Cognitive skills, on the other hand, seem to have a rather unstable relationship with flow. A study on first person shooters, for example, revealed that the ability to hit a static target improved flow for players using a gamepad, but not for those using a motion controller (Bowman & Boyan, 2008). This suggests that the capacity of a skill to promote flow can be affected by different aspects of gameplay, like the type of interface being used. One possible explanation could be that when the characteristics of a game change, different abilities are required. In other words, the right set of abilities is more important than having many skills. This would increase performance and, therefore, the feeling of competence; both variables that have been associated with higher levels of flow (Bowman & Boyan, 2008; and Jin, 2012, respectively). The idea is congruent with an experiment in which players who rated better their ability to play a racing game achieved higher levels of flow while playing it (Jin, 2011). So, an ability for planning and leading organized attacks might improve your chances of experiencing flow in “World of Warcraft” and “League of Legends”, but it might not be that useful in a more chaotic environment like an online “Call of Duty” match.

An ability that produces flow in a racing game could be useless in a surgical simulator.

How much challenge or skill is needed for flow to occur? A series of studies on Wii titles showed that, more or less, low levels of challenge or skill tend to produce low levels of flow (Jin, 2011; 2012). This is especially clear in the case of “Need for Speed” (Jin, 2011) and “Need for Speed: Nitro” (Jin, 2012). This suggests that, at least in the case of racing games, a minimum level of demand or ability is required to experience flow. However, an experiment that measured the playfulness trait on people playing “Wii Fit” showed an interesting result. When playfulness was low, the effect of low challenge and skill on flow was, as expected, usually low (Jin, 2012). But when participants were highly playful, low levels of challenge and skill produced the highest levels of flow (Jin, 2012). This probably happens because the absence of challenge or skill means there will be little punishment for errors or reward for victories. This means that there will be less negative consequences if the player engages in free exploration, something that tends to attract highly playful people. So, if you start playing a Real Time Strategy games like Red Alert or StarCraft for the first time, try going through the tutorial campaign first. This will help you improve your skill, increasing your chances of experiencing flow later on. Unless you are the type of person that enjoys finding out the different features of a new game first; like listening the character’s voices or watching the different types of deaths.

Forgetting the mission and exploring the surroundings can be fun too.

Researchers believe that balance between challenge and skill should produce the highest levels of flow. This makes sense as a challenge superior to the player’s abilities would end up promoting frustration, and a level of difficulty inferior to the user’s skills would make the task boring. The hypothesis is supported by part of the evidence: Balance between skill and challenge has been associated with flow for computer games like “Pac-Man” (Engeser & Rheinberg, 2008) and “Bloons Tower Defense 4” (Schmierbach, Chung, Wu, & Kim, 2012), as well as for Wii titles like “Need for Speed” (Jin, 2011), “Need for Speed: Nitro” and “Mad World (a gory beat-em-up) (Jin, 2012). This didn’t happen, however, in the case of “Trauma Center: Second Opinion” and “Wii Fit” (Jin, 2012). One way to explain this is that specific game attributes made the experience of failure less negative or the positive feedback obtained from victories more rewarding, preventing a game that is too difficult or too easy to become less enjoyable. Maybe the absence of virtual enemies in “Trauma Center” made the player care less about losing and “Wii Fit” users were more interested in exploring the capabilities of the Kinect than scoring the highest score. In any case, evidence suggest that there is no sure way to provoke flow, but balance between challenge and skill is definitely a good start.

Balance between challenge and skill seems to be less important in games like “Wii Fit”.

It’s seems difficult to predict when or even if flow will happen. But that doesn’t mean there isn’t something we can do to increase our chances of either enter the state or provoke it. The first lesson we can obtain from the reviewed studies is that some skill is better than no skill. And while no player starts a game with a complete understanding of the mechanics, going through a tutorial or keeping the “hints” option activated could help users make a faster transition from “Wow, easy there!” to “Ok, let’s do this!”. The second lesson is that the relationship between challenge and skill seems to play an important role in the occurrence of flow. This means that it’s particularly important for developers to provide different difficulty levels the user can choose from, as well as carefully verify if the challenge increases with each stage just enough to match the player’s newly acquired skills. The final lesson we must remember is this: The circumstances of play can change what does and what doesn’t promote flow. Different game types and user preferences can, as we have seen, prompt unexpected situations, like a highly playful individual entering a deep state of flow just by ignoring all the objectives and do what he or she wants.

Flow is a complex phenomenon. Probably as complex as different types of games and players are out there. Finding a formula that consistently triggers intense and prolonged flow states in every single player might be impossible. But thanks to the work of both researchers and developers, we can expect flow to become a more accessible type of experience with every new generation of games.

References:

Bowman, N. D., & Boyan, A. C. (2008, May). Cognitive skill as a predictor of flow and presence in naturally mapped video games. Paper presented at the 58th annual convention of the International Communication Association, Montreal, Canada.

Engeser, S., & Rheinberg, F. (2008). Flow, performance and moderators of challenge-skill balance. Motivation and Emotion, 32(3), 158-172. doi:10.1007/s11031-008-9102-4

Jin, S. A. (2011). I feel present. Therefore, I experience flow: A structural equation modeling approach to flow and presence in video games. Journal of Broadcasting & Electronic Media, 55(1), 114-136. doi: 10.1080/08838151.2011.546248

Jin, S. A. (2012). Toward integrative models of flow: Effects of performance, skill, challenge, playfulness, and presence on flow in video games. Journal of Broadcasting & Electronic Media, 56(2), 169-186. doi: 10.1080/08838151.2012.678516

Schmierbach, M., Chung, M., Wu, M. & Kim, K., (2012, May). No one likes to lose: Game difficulty, motivation, immersion and enjoyment. Paper presented at the annual meeting of the International Communication Association, Phoenix, AZ.

The Power of Video Game Music

Music has become an essential part of the video game experience. Developers and composers have managed to make stories more compelling and challenges more attractive with the help of carefully designed soundtracks. Games like Ninja Gaiden for the NES or successful series like Metal Gear or Bioshock probably wouldn´t be the same without the beautiful music especially prepared for them. However, we still know very little about the how and why music affect us the way it does. Why does some songs seem playful and other serious? Why some music make us hurry while other tends to relax us? And if music is capable of affecting how we feel, could it also makes us better players?

Let’s start by talking about the relationship between music and emotions. Video game music is frequently associated with positive affect. The highly motivational theme we hear while waiting for a Battlefield map to load or the playful tune we listen to in the wizard town of Mysidia in Final Fantasy IV are good examples. But music can also be used to provoke negative feelings. The presence of music, for example, was shown to increase stress levels in a group of people playing Quake III (Hébert, Renée, Dionne-Fournelle, Crete & Lupien, 2005). It was also found to make players less relaxed and increase how dangerous everything felt in The Lord of the Rings: The Two Towers game (Lipscomb & Zehnder, 2004). This suggest that listening to certain types of music can make players feel more afraid of negative in-game events, like being shot by the enemy or falling off a cliff. This capacity to elicit negative affect isn’t necessarily a bad thing. In Resident Evil 2, for example, the fear of a surprise encounter with an enemy forces players to pay close attention to any suspicious movement or sound, increasing both engagement and emotional involvement. Also, the constant threat of dying and losing all the unsaved progress turns into a profound feeling of relief when a typewriter (a tool used to save the player’s game) is eventually found. So, even when music promotes negative affect, this could indirectly lead to more immersive and rewarding experiences.

Skyrim’s outdoor theme switches to a more action oriented tune in the presence of an enemy.

Developers seem to believe that fast paced music can increase the fear and anxiety provoked by an approaching threat. Super Mario Bros. titles, for example, tend to accelerate the speed of music when time is running out. Castlevania III: Dracula’s Curse, on the other hand, uses a high tempo song (appropriately named “Pressure”) in a level where the player must hurry to escape constantly rising water levels. But how does this work? One possibility is that people use the speed of music as a way to measure time. The faster it goes, the more it feels like additional time has passed. This could also make us feel like we are moving slowly. The idea seems to be congruent with an experiment in which fast paced music made players drive faster in a racing game (Cassidy & MacDonald, 2010). So, if you are running away from an enemy, then chances are a fast song will make you feel like you are about to lose your life.

Castlevania III added a high-tempo song to a flooding level to increase the feeling of an approaching threat.

While altering tempo seems to be a very reliable way to affect gameplay, media producers also depend heavily on the use of highly arousing music. This is especially obvious in the trailers for summer blockbusters, were intense compositions are used to transmit a feeling that something great and important is happening (just watch the theatrical trailer for Sam Raimi’s Spiderman 2). Video games are no exception. Arousing music tends to accompany the most important events in a level or game. Scenes like the Battle of Stalingrad in the original Call of Duty wouldn’t have transmitted such strong emotions if it wasn’t for the epic but sad theme that played in the background. However, this type of songs might not go with every type of game. People playing Project Gotham Racing 3, for example, found high arousal music less appropriate and enjoyable as well as more distracting than low arousal songs (Cassidy & MacDonald, 2010). It also made them reach higher speeds and achieve shorter lap times, as well as hit more obstacles (Cassidy & MacDonald, 2010). This is congruent with the idea that highly arousing music motivates individuals to act with more energy and speed, but also more recklessly. So if you want start a carefully planned campaign in a Real Time Strategy game, we don’t recommend listening to John Williams´ “Duel of Fates”.

Highly arousing music made players go faster but also more reckless in Project Gotham Racing 3.

As we have seen there are different ways to influence the player’s emotions and behavior through music. Video game companies know this and invest a considerable amount of resources in designing the best possible soundtrack. This involves carefully matching different styles of songs to specific types of games or scenes: A heavy metal song may go well with violent action scenes as well as a theme of revenge, but it may feel out of place in a stealth mission or a scene depicting the dead of a beloved character. Still, some players think games should give users the option to listen to their own music. This would allow them to build a more personalized experience, choosing less distracting and more arousing songs, eventually increasing both engagement and emotional involvement. In the previous experiment, for example, listening to self-selected music was not only preferred but also improved the participants ability to avoid obstacles randomly placed in the racing track (Cassidy and MacDonald, 2010). Also, incongruence between soundtrack and audiovisual content seems to have a very limited effect in immersion, the feeling of being inside the content and how realistic the events depicted seem; at least in the case of movies (Petey, Campanella, Rubenking, Buncher & Gress, 2010). Still, developers won’t necessarily appreciate the idea of having less control over the final feel of their product and additional evidence would be required before most of them adopt this approach.

GTA: San Andreas and WWE 13 allow the the user to add their own music, increasing replay value.

Research on the relationship between music and the user experience could have an important effect in the future of video games. Especially considering the current interest in the development of soundtracks that adapt to the user’s feelings and behavior. But instead of using this newly acquired knowledge to provoke a more intense experience, developers could just employ it to obtain more precise effects. After all, triggering a very specific sets of emotions or reduce the levels of distraction to improve performance could be enough to make players perceive a significant improvement in video game quality. Findings could also be applied to different fields like the use of music in work-place to reduce stress without affecting performance. In the end research about video game soundtracks could end up affecting not only how we play at home, but also how we spend our time in the office.

References:

Cassidy, G. G. & MacDonald, R. A. R. (2010). The effects of music on time perception and performance of a driving game. Scandinavian Journal of Psychology, 51(6), 255-464. doi: 10.1111/j.1467-9450.2010.00830.x.

Hébert, S., Renée, B., Dionne-Fournelle, O., Crete, M., & Lupien, J. (2005). Physiological stress response to video-game playing: The contribution of built-in music. Life Sciences, 76(20), 2371-2380

Lipscomb S. D. & Zehnder, S. M. (2004). Immersion in the virtual environment: The effect of a musical score on the video gaming experience. Journal of Physiological Anthropology and Applied Human Science, 23(6), 337-343.

Petey, G., Campanella, C., Rubenking, B., Buncher, M., & Gress, E. (2010). Telepresence, soundscapes and technological expectation: Putting the observer into the equation. Virtual Reality, 14(1), 15-24. doi: 10.1007/s10055-009-0148-8

Video game technology. Part 3 of 3: Immersion through sound.

Can audio really make movies and games more immersive and fun?

What is the best video game you have ever played? Don’t answer yet. What the first thing that came to your mind when you read that question? Was it a specific level in a first person shooter? A character in a role playing game? In any case you first thought was probably associated with visual images. A gun fight from a first person perspective, a character jumping between platforms or a flash of light after an explosion. Every time we think about video games we tend to think in terms of visuals. Nevertheless, sound also plays an important role in our gaming experiences. Now, you are probably thinking about the rewarding sound you heard after picking up an item or finishing an enemy. Maybe you are just remembering how emotionally moving was the soundtrack of the last game you played. And while sounds and music are not the first thing we associated with movies or video games, it is still a very important part of the experience. They can tell if there are enemies around us and even communicate the emotional tone of a scene. But can sound really affect how immersed we feel or how much we enjoy a movie or video game? Is it really that important?

Silent hill reduced visibility and forced the player to rely on sound effects.

Sounds are an essential feature of our environment. They gives us information about the objects and events around us. They can even tell us about what is happening out of our field of view. It’s no surprise then that the presence of sound has been shown to increase the feeling of being inside an interactive virtual environment (presence) (Larsson, Västfjäll, Olsson & Kleiner, 2007). It has also been associated with higher levels of involvement towards the sensory stimulus, story and characters (immersion), the positive feeling of focusing on a cognitively demanding task (flow) and reduced tension while playing a first person shooter (Nacke, Grimshaw & Lindley, 2010). In this last experiment, however, sound didn’t provoke any significant changes in the physiological indicators of emotional valence (facial muscles activity) and arousal (electro-dermal activity) (Nacke, et al. 2010). This means that either the change in emotions wasn’t strong enough to affect the physiology of players, or that the presence of sound increased their expectations, provoking them to overrate how immersed and less tense they felt. This makes the results of the second study somehow inconclusive, but there’s still an important amount of evidence of how audio can affect the experience of media.

Audio can also provide a considerable amount of information about the things and events that provoked them. The sound an object makes when falling to the ground can give us clues about what it’s made of, how heavy it is or how fast it was going. Placing speakers in different positions around the user allows him or her to use the sound as a source of spatial information. For example: If a sound comes from a speaker placed to the left of the user, he or she will feel like it’s coming from this side of the virtual environment. Sounds can also provide information about the position of its source and the acoustics of the environment, something that tends to make them easier to identify as well as feel more realistic (Larsson, Västfjäll, Olsson & Kleiner, 2007). This has also been shown to increase the feeling of being inside the content (Larsson, et al. 2007). After all, people tend to use sounds to discover the position of objects as well as the shape of their surroundings in real life.

Outdoors and enclosed spaces sound different, even in video games.

If increasing the number of audio channels can help the user better identify the position of sound sources, then audio systems that use various speakers in different positions should be able to provoke a higher sense of presence and immersion, right? According to research, using five audio channels instead of two didn’t increase the feeling of being inside the content or how realistic the events felt in the case of a movie (Freeman & Lessiter, 2001), but it did when the media was a first person shooter (Skalski, Whitbred & Lindmark, 2009). The additional number of channels also made both, the experience of the movie and the video game, more enjoyable (Freeman & Lessiter, 2001 and Skalski, et al. 2009, respectively). This suggest that additional audio channels only increase presence and realism when the user is able to interact with the content, but that this isn’t necessary in the case of enjoyment. Maybe the additional audio channels are able to make the movie experience more interesting. But in video games, where the user’s actions can affect what happens next, the information coming from the additional audio channels becomes more important. The sounds now reveal the position of potential threats and rewards, increasing levels of attention towards this type of stimuli. In this circumstances the player is able to better appreciate the details and experience more intense emotions, perceiving the content as more realistic. Also, the additional levels of engagement will eventually make him or her forget about the technology providing the experience, producing a feeling of being inside the content.

Sound direction is more relevant in games, where it can reveal the position of an enemy.

Adding bass to the audio channels while watching a movie seems to increase how clear and exciting the audio seems, but also how uncomfortably loud it may feel (Freeman & Lessiter, 2001). This might have happened because people tend to associated strong vibrations with loud noises, like a sound system turned up to its maximum volume. Interestingly, the additional bass channel also made the visual stimuli more exciting (Freeman & Lessiter, 2001). This could be attributed to a halo effect, where the additional audio quality made the subjects overstate how well other stimuli felt.

The bass channel was also found to increase the feeling of being inside the movie as well as involvement and the believability of the events depicted on it (Freeman & Lessiter, 2001). It’s possible that the viewers not only heard the bass sounds but also felt the strong vibrations associated with them. The additional tactile information could have helped the viewers experience how it would have felt to be a part of the portrayed events, increasing both presence and perceived realism.

Strong bass sounds can make the viewer feel like the floor is moving,
increasing realism and presence.

Regardless of the apparently obvious benefits of using a speaker-based surround system, some people still prefer to wear headphones. One reason to choose them could be that they help isolate the user from external noise. This would make it easier for him or her to focus on the content, increasing engagement and maybe even involvement. However, research on the effects of using headphones or speakers in the experience of a movie is still inconclusive; with one study showing an increase in engagement and involvement when using headphones (Campanella, Pettey, Guha, & Rubenking, 2010) and another one finding no significant differences between each condition (Pettey, Campanella, Rubenking, Buncher & Gress, 2010). Nevertheless, both studies found that using headphones or speakers didn’t produce any significant differences in the feeling of being inside the content or how realistic the depicted events seemed (Campanella, et al. 2010; Pettey, et al. 2010). So maybe, in the end, it’s all about comfort.

What is better for playing video games then: Headphones or speakers? The previous study indicates that there seems to be no significant differences in the case of movies. Still, we should remember that, as previous research has demonstrated, the fact that an audio variable doesn’t affect the experience of watching a movie doesn’t necessarily means that the same will happen with video games (e. g. Freeman & Lessiter, 2001 and Skalski, Whitbread & Lindmark 2009). Maybe the relationship between the isolation provided by headphones and immersion becomes stronger when the user is able to interact with the media. Still, additional studies are necessary to clarify this last question.

While improvements in the field of images and animation has been more obvious, significant advances have happened too in respect to audio technology. Built-in speakers are still available, but users now have the option to invest several hundreds of dollars in complex surround systems and highly advanced headphones. However, additional studies are required before we can assure consumers that a specific piece of hardware will make them feel more immersed, at least in the case of video games. In the meantime just ask yourself this question: Does a device helps you focus on the game? Does it make you more involved in the story and characters? Did you found yourself suddenly feeling like if you were inside the virtual environment? If the answer is yes, then you probably made the right choice.

References:

Campanella, C., Pettey, G., Guha, T., & Rubenking, B. E. (2010). Sound out small screens and telepresence. The impact of audio, screen size and pace. Journal of Media Psychology, 22(3), 125-137. doi: 10.1027/1864-1105/a000017

Freeman, J. & Lessiter, J. (2001, August). Here, there and everywhere: The effects of multichannel audio on presence. Proceedings of the 1001 International Conference on Auditory Display, Espoo, Finland.

Larsson, P., Västfjäll, D. Olsson, P. & Kleiner, M. (2007, October). When what you hear is what you see: Presence and auditory-visual integration in virtual environments. Proceedings of the 10th Annual International Workshop on Presence, Barcelona, Spain.

Nacke, L. E., Grimshaw, M. N. & Lindley, C. A. (2010). More than a feeling: Measurement of sonic user experience and psychophysiology in a first person shooter game. Interacting with Computers, 22(5), 336-343. doi: 10.1016/j.intcom.2010.04.005

Pettey, G., Campanella, C., Rubenking, B., Buncher, M., & Gress, E. (2010). Telepresence, soundscapes and technological expectation: Putting the observer into the equation. Virtual Reality, 14(1), 15-24. doi: 10.1007/s10055-009-0148-8

Skalski, P., Whitbred, R., & Lindmark, P. (2009, November). Image vs. sound: a comparison of formal feature effects on presence, video game enjoyment, and player performance. Paper presented at the 12th annual international workshop on presence, Los Angeles, CA.

Video game technology. Part 2 of 3: The HD Experience.

 
claimtoken-534595084c47d
One of the most recent advances in entertainment technology has been the switch from standard resolution to High Definition. The change has affected not only displays but also the complexity of the media produced. Most TV shows, movies and video games are now made in this format. The capacity to offer higher resolution images even became a powerful weapon in the last version of the console wars. The technology has been so well received by the public that Ultra HD displays are already being sold and media producers are spending time and money to make more content available in this resolution. But putting the public demands aside, is High Definition really capable of improving the user experience? Does it have more complex effects than just making things looks prettier?

First let’s analyze what High Definition is and what it can (probably) do. Image resolution indicates the number of pixels (or squares) available to build an image. The higher the resolution, the more details an image can have. This could make the content easier to identify and more interesting, increasing the attention of the user. The type of attribute being noticed could also affect different aspects of the experience. More defined borders, better lighting and smoother movements, for example, could make the objects, characters and environments look more realistic. On the other hand, if the content affected is emotionally relevant (like the facial expressions of a character in a movie or the position of the enemies in a first person shooter), it could make the user more involved.

The reason we focus on this variables is because they seem to be strongly related with spatial presence (the feeling of being inside the content) and immersion. A higher level of attention towards the content usually implies a reduction in the importance given to external stimuli. This can gradually make the user less aware of the technology providing the experience and, eventually, make him or her feel like they are part of the content. Additionally, engagement, perceived realism and involvement are frequently included in questionnaires designed to measure immersion. Also, both spatial presence and immersion play a central role in the studies about High Definition we were able to find.

"Papers, Please" was able to provoke strong emotional responses using only a couple of pixels.

Let’s begin with two experiments about the effects of resolution on viewer’s experience. In 2005, a group of participants watching a short documentary in HD (1080 lines of resolution) perceived the sensorial stimuli as more believable and identified non-verbal behavior more easily than those who saw it on NTSC (with only 480 lines) (Bracken, 2005). The next year researchers asked the same thing to another group of individuals, but they showed a newscast instead of a documentary. Interestingly, this time HD didn’t make the content look more realistic (Bracken, 2006). However, it’s effect on perception of non-verbal behavior remained (Bracken, 2006). One explanation could be that documentaries tend to show slow paced scenes and smooth transitions, leaving the viewer enough time to appreciate the details designed to increase realism. Newscast, on the other hand, tend to use shorter clips and fast paced transitions with the intention to capture the viewer’s attention as soon as possible, reducing their chances to notice some of this features. High Definition also made the news anchors look more dynamic, qualified and trustworthy (Bracken, 2006). As we previously stated, it’s possible that improved image quality made specific details more noticeable for the audience. In this case, relatively stable attributes intended to increase the anchor’s believability (like their clothing, facial expressions and hand gestures). In any case, results suggest that increased image resolution can make details intended to promote realism and communicate non-verbal behavior easier to identify.

Participants who saw the documentary on HD showed higher levels of self-reported spatial presence (Bracken, 2005). This is consistent with the idea that improved image quality can increase the user’s levels of attention, which eventually can lead him or her to forget about the mediating technology. However, watching the documentary in HD didn’t make the viewers feel more immersed (Bracken, 2005). In order to explain this we should point out that the mentioned study measured immersion by asking participants how engaged and involved they felt, as well as how realistic the content seemed to them. As we have seen, image quality was able to increase both realism and (probably) engagement. However, involvement is a different issue. For a user to feel involved, he or she must find the content important. One way to achieve this is through emotionally relevant stimuli, like a sad story on the news, seeing a character we care about suffer in a movie or playing the last level of video game with only one life left. While increased image quality could have made this type of content more noticeable, chances are it wasn’t present in the video used by the experimenters: A short, dialogue-free documentary showing scenes of everyday life in japan. So, while the enhanced images could have increased both engagement and realism, there was no reason for involvement to do so; reducing the probability of self-reported immersion to experience a significant change. Nevertheless, when the visual stimuli was a newscast (a type of content that tends to be designed to provoke an emotional response), watching it on HD did manage to increase the level of immersion experienced by the viewers (Bracken, 2006). So, maybe high definition can make emotionally relevant content (if available) easier to identify, increasing involvement and, therefore, immersion.

Is high definition making emotional content like facial expressions easier to identify?

The previous studies suggest that increasing resolution in a video can provoke a significant effect on the viewer’s experience. But what would happen in the case of video games, were the interaction between player and virtual environment opens the possibility for much stronger psychological effects?

To answer this question Skalski, Whitbred and Lindmark (2009) asked a group of undergraduate students to play Tom Clancy’s Ghost Recon: Advanced Warfighter for the Xbox 360 on HD (1080 lines) or standard definition (480 lines). Results showed that improved resolution didn’t make sensorial stimuli feel more realistic or in-game events seem more possible to occur in real life (Skalski, et al. 2009). It didn’t increased enjoyment either (Skalski, et al.). One explanation could be that increasing resolution wasn’t enough to make the features designed to increase realism (like particles flying around or precise animations showing enemies falling to the ground) more noticeable. Additionally, it’s possible that perceived realism plays a more important role in the experience of games that try to depict real-life events (in this case, a soldier in the battlefield). So, maybe the inability for resolution to increase realism was in part what prevented enjoyment to improve.

Participants playing Tom Clancy’s Ghost Recon: Advanced Warfighter for the Xbox 360 on HD didn’t show higher self-reported engagement or spatial presence than those who played it on NTSC (Skalski, Whitbred and Lindmark, 2009). This suggest that the increase in resolution didn’t make the content significantly more interesting. The result is also congruent with the idea that increased levels of attention are required for the experience of non-mediation (forgetting that what we are perceiving is coming from technology and not reality) to occur. In a similar experiment, subjects playing Perfect Dark Zero for the Xbox on HD didn’t experienced more spatial presence either (Bracken & Skalski, 2009). Evidence from both studies seem to indicate that the relationship between image resolution and the feeling of being inside a first person shooter is surprisingly weak. Still, experiments with other types of games could show different results.

HD can surely increase the sense of presence in first person shooters, right? Think again.

The study using Perfect Dark Zero had the advantage of measuring self-reported immersion, finding higher levels for the group that played the game in HD (Bracken & Skalski, 2009). Maybe increased resolution made easier for the HD group to identify in-game objects like pick-ups or enemies. This would have made the players more worried about both rewards and mistakes, increasing their involvement and, eventually, making them feel more immersed.

Resolution has been an important aspect of technological purchases for fans of TV shows, movies and video games. They want to see their media with the best image quality available. And while most of them are convinced that their experience as users has improved, it’s most likely that the effects of image resolution go far beyond just making things looks pretty. They involve complex and strong psychological effects. Still, additional studies are necessary to better understand this relationship. And while there aren’t much papers on the effects of High Definition, the development of new visual technologies like Ultra High Definition and Virtual Reality will probably recapture the attention of scientists interested in how to improve the user experience.

Thanks for reading! Visit us in two weeks for the final part of our series on Video Game Technology, when we analyze the effects of sounds and music in the player’s experience.

References:

Bracken, C. C. (2005). Presence and image quality: the case of high-definition television. Media Psychology, 7(2), 191-205. doi: 10.1207/S1532785XMEP0702_4

Bracken, C. C. (2006). Perceived source credibility of local television news: the impact of television form and presence. Journal of Broadcasting & Electronic Media, 50(4), 723-741. doi: 10.1207/s15506878jobem5004_9

Bracken, C. C., & Skalski, P. (2009). Telepresence and video games: the impact of image quality. PsychNology Journal, 7(1), 101-112. Retrieved from: https://www.researchgate.net/profile/Cheryl_Bracken/publication/220168895_Telepresence_and_Video_Games_The_Impact_of_Image_Quality/file/3deec52e14793e7ab5.pdf

Skalski, P., Whitbred, R., & Lindmark, P. (2009, November). Image vs. sound: a comparison of formal feature effects on presence, video game enjoyment, and player performance. Paper presented at the 12th annual international workshop on presence, Los Angeles, CA.

Video game technology. Part 1 of 3: New isn’t always better.

There is something special about bringing home the latest gaming system. The shiny box, the pristine plastic. Everything looks and even smells new. You start connecting the system and can’t stop but think about how awesome it is going to be. How clear the images are going to look, how fluid the animation is going to be, how natural the controls are going to feel. And all of this because you are convinced that the heavy and very expensive piece of hardware you hold in your hands is actually going to provide that. Better graphics, sounds and controls, right? Well, with each new console released the improvements have become a little less obvious. The video games of the last generation specially, while they look great, don’t give the impression a giant leap into the future. But how much does new technologies tend to improve gameplay? Do more recent games provide a more immersive, intense and fun experience? Is new always better?

One way to test the effect of technological advances is to see if more recent games and systems provoke higher physiological responses. In a 2007 study, a group of university students playing a Zombie shooter or a first-person “breakout” type of game from either the mid-90’s or the early 2000’s showed no significant differences on their skin conductance (an indicator of arousal) (Ivory & Kalyanaraman, 2007). In another study, participants playing a different baseball or Mortal Kombat game on a Super Nintendo, a Nintendo 64 or a Playstation 2 showed no significant differences on their heart-rates (an indicator of emotional arousal) (Barlett, Rodeheffer, Baldassaro, Hinkin & Harris, 2008). The same result was obtained when the systems compared were a Nintendo 64, a Playstation 2 and a Nintendo Wii (Bartlett, et al., 2008).

Games only a couple of years more recent weren’t able to affect the player’s arousal.

The results could suggest that technological improvements didn’t affect the experience of playing enough to provoke a change in the user’s physiology. However, there might by other explanations. The term “technological improvements” can make reference to different variables like graphics, sound and controls; each one with a very particular effect on gameplay. For example, graphical improvements may increase the user’s visual attention, while more responsive controls might promote the feeling of being connected to the avatar. Additionally, better technology doesn’t always translate into better games. Super Nintendo titles, for example, tend to show much better graphics than the Nintendo 64 games, even though this console was so powerful it could handle 3D animation. Also, individuals could respond differently to technological improvements. Some players might be fascinated by the detailed 2D graphics of The Legend of Zelda: A Link to the Past, while others might admire the way Ocarina of Time recreated the fantasy world of Hyrule in 3D. So, maybe different variables had opposed but significant effects on arousal, giving the appearance that it remained unaffected. For example: Graphical improvements could have increased skin conductance and heart-rates significantly, but the bad implementation of the new audio technology could have end up reducing physiological arousal in the same proportion, cancelling each other’s effects out. And even if the arousal levels would have shown important changes, this could might as well have been a sign frustration, not enjoyment.

More powerful hardware means better games? Tell that to Superman.

Even when the impact of technology wasn’t (apparently) strong enough to provoke a significant change in the physiology of players, it still managed to affect some aspects of their experience. Playing a computer game from the early 2000’s, for example, provoked significantly higher levels of self-reported excitement than playing one from the mid-90´s (Ivory & Kalyanaraman, 2007). It also increased the feeling that the game was real (presence) and how much the players paid attention and worried about the game (involvement) (Ivory & Kalyanaraman, 2007). Maybe the better quality of graphics and sounds increased how much attention the player paid to the game and reduced the importance given to external stimuli, promoting involvement and presence. In the second experiment comparing consoles, playing with a Nintendo 64 was perceived as significantly more difficult, and the Nintendo Wii as more fun (Barlett, et al., 2008). One explanation could be that the Nintendo 64 controller, while looking similar to the current generation of gamepads, responded much poorly in comparison. And while the Playstation 2 controller worked fine, the Wiimote took advantage of player’s previous experience, making it easier to use and more enjoyable.

Another limitation of these studies is that they weren’t able to properly isolate technology as a dependent variable. In other words, in order to attribute any changes in users to technology, this had to be the only difference between the groups studied. However, participants didn’t played different versions of the same game across time periods or consoles. Instead, they played different games of the same genre or series. This means that any of the observed changes could also be attributed to differences between the titles. Maybe the more modern Zombie shooter game had a better story or more jump-scares, making the users more scared without the use of better technology.

Sometimes a clever use of the existing hardware is all you need to create innovative and fun games.

Future studies should try to isolate specific sub-components of the “technological advances” variable. Researchers could compare how players respond to the same game in different resolutions. They could also analyze how different generations of the same controller impact the gameplay of a specific title. By studying each individual part independently, experts could obtain much more reliable information about the effect of technology on the gaming experience. It would also make results much easier to apply for developers. Instead of just trying to use the most recent technology, they could focus their resources on specific attributes, like frames per second or button sensitivity, and their proven effects on the user.

Technological advances can help developers create better gaming experiences. It expands the limits of what designers and programmers can do with graphics, sound and interfaces. It also allows them to try new techniques to improve enjoyment, like using more realistic facial movements to make the characters more relatable or recording binaural audio to make some sounds feel like they are coming from somewhere inside the player’s house. However, specific aspects of video game technology have to be identified in order to do discover its particular effects as well as to take advantage of them. An approach that, unsurprisingly, is being used more frequently in recent studies.

Thanks for reading! Please, visit us in two weeks when The Hero Archetype studies the effect of high resolution images on gameplay.

References:

Barlett, C., Rodeheffer, C. D., Baldassaro, R., Hinkin, M. P., & Harris R. J. (2008). The effect of advances in video game technology and content on aggressive cognitions, hostility, and heart rate. Media Psychology, 11(4), 540-565. doi: 10.1080/15213260802492018

Ivory, J. D. & Kalyanaraman, S. (2007). The effects of technological advancement and violent content in video games on player’s feelings of presence, involvement, physiological arousal, and aggression. Journal of Communication, 57(3), 532-555. doi: 10.1111/j.1460-2466.2007.00356.x

Are realistic controls better than gamepads?

Before we start, answer me this question: What do you look for in a controller? Does it has to be pretty or just comfortable? Does it has to be easy to use or you don’t mind spending some time until you master it? It has to behave as you expected it, right? That’s obvious. But, what else? Does it has to make you feel like your experiencing something real? Does it has to look like the object the game is trying to simulate (like a steering wheel or a gun)? Many players feel video game controls should feel right, be easy to use, behave as expected and make the player feel like they are in control; a series of attributes that have come to be known as controller naturalness.

Do you need your controller to look like a rifle in order to fully enjoy a First Person Shooter?

Companies have tried different ways to make their controllers feel natural. They have made sure there’s a correspondence between the controller’s directions and in-game actions (pressing the left arrow key should make the character move or look left, not up). They have developed cameras capable of recognizing precise body movements (like Microsfot’s Kinect). They have built motion sensitive controllers that can replicate real world actions (like swinging it to make a virtual bat hit a ball). And they have also designed realistic representations of real world devices, like steering wheels or sniper rifles (Skalski, Tamborini, Shelton, Buncher & Lindmark, 2011).

But do they work? According to some experiments, when playing sports games that involve swinging a long object (like golf or baseball), motion controllers tend to feel more natural than gamepads (Skalski, et al., 2011; McGloin, Farrar & Krcmar, 2011). Also, playing a racing game with a steering wheel is perceived as more natural than playing with a keyboard, a joystick (Skalski, et al., 2011) or a gamepad (Skalski, et al., 2011; Schmierbach, Limperos & Woolley, 2012). Overall, it seems that controllers that facilitate the execution of movements that resembled real world actions (like moving a steering wheel to make a car turn) tend to feel more natural. An attribute that could also allow the use of previously acquired knowledge or skill on the subject (Tamborini & Skalski, 2006). Players could have learned about baseball by watching a match on TV or actually playing the game, and then apply this information to improve their performance.

Motion controllers allow the execution of movements that resemble the actions portrayed in the game.

While these studies suggest that realistic controls elicit a more natural feeling, this might not always be the case. Many variables can affect control naturalness. A new controller will always feel a bit awkward at first, a sensation that will disappear with repeated use. Some may just prefer a specific type of interface, depending on the game. For example, some players may love to play sports games with motion controllers, but prefer the good old mouse and keyboard in the case of first person shooters. Also, real world experience might not always benefit video game performance. Experienced drivers might find some driving games overly simplistic, even if they include realistic steering wheels, gear stick and a set of pedals.

Why should companies worry about developing more natural controls? For starters, it seems to have some positive effects on the gameplay experience. In some of the mentioned studies, for example, controller naturalness increased how real graphics and sounds were perceived (McGloin, et al., 2011), as well as the feeling of being inside the game (Skalski, et al. 2011; McGloin, et al., 2011 & Schmierbach, et al., 2012). Some authors believe this happens because the use of information from the outside world makes the player feel like he’s participating in a real event (McGloin, et al., 2011). Others think it’s related with how natural controls make the player feel like their physical actions and selves are connected with the virtual environment (Schmierbach, et al., 2012), giving the impression that the in-game events are actually happening to him or her. Another possibility is that, as they provide a faster and better learning of the controls, natural interfaces reduce the time it takes for the controller to become less noticeable (Skalski, et al., 2011). Something necessary for the player to experience immersion.

Are those my hands or the avatar’s on the steering wheel?

Perceived control naturalness has also been found to increase enjoyment (Skalski, et al., 2011; McGloin, et al., 2011). This isn’t unexpected as natural controllers have been shown to increase the balance between challenge and skill (Schmierbach, et al., 2012), which tends to make games more fun. Maybe being able to use their previous experiences allowed the players to learn the game controls faster and better, improving their performance. Control naturalness also had a positive effect on transportation, an indicator of increased affective, cognitive and mental imagery involvement (Schmierbach, et al., 2012). It’s possible that a better sense of control made the players think and worry more about the in-game events, making the experience more entertaining.

Natural controls can improve the experience of playing a game that simulates real world events. But what about games that represent fantastic actions, like flying or throwing energy beams? Some authors think natural controls can’t be applied to fantastic activities (e. g., Poole, 2000). With no real world counterparts, how would developers know what to replicate? Also, different players might have a very different idea of how each movement has to be executed. Even if designers manage to overcome this obstacle, would it feel real? Maybe controller naturalness in fantasy games depends on how closely the animation follows the player’s particular idea of how the movement should be executed. Moving your hand forward to shoot a Hadouken in Street Fighter probably won’t feel as good as having to put our hands together, placing them next to your waist, opening them, waiting for the attack to charge and then slowly moving them forward until they face your enemy.

The way natural interfaces interact with knowledge and skill could also be used to improve training programs. Some devices have already been developed for future surgeons to practice complex operations in virtual environments without any risk. The idea could also be applied to less serious scenarios, like sports. Golf and baseball players might benefit from the development of controls and games specially designed for professional training. However, the capacity of this type of systems to improve learning might decrease in the case of experts, as they would be able to notice even the slightest of the differences between the simulation and the real world procedure.

Simulators with natural interfaces are being tested as a new way to train future surgeons.

Natural interfaces can benefit gameplay. However, this doesn’t mean that more realistic controllers would necessarily feel more natural. The level in which a player has adapted to a particular type of interface or its compatibility with a specific video game genre also influences the perceived naturalness. Either way, the capacity of controllers to enhance gameplay depends more on the user experience and preference than on any particular feature of the device. Before choosing your next controller just think about this: Which one feels more comfortable and makes you feel in control? Which one just feels right?

References:

McGloin, R., Farrar, K. M., & Krcmar, M. (2011). The impact of controller naturalness on spatial presence, gamer enjoyment, and perceived realism in a tennis simulation video game. Presence: Teleoperators and Virtual Environments, 20 (4), 309-324. doi: 10.1162/PRES_a_00053

Poole, S. (2000). Trigger Happy. New York: Arcade Publishing.

Skalski, R., Tamborini, R., Shelton, A. Buncher, M. & Lindmark, P. (2011). Mapping the road to fun: Natural video game controllers, presence, and game enjoyment. New Media & Society, 13 (2), 224-242. doi: 10.1177/1461444810370949

Schmierbach, M., Limperos, A. M., & Woolley, J. K. (2012). Feeling the need for (personalized) speed: How natural controls and customization contribute to enjoyment of a racing game through enhanced immersion. Cyberpsychology, Behavior and Social Networking, 15 (7), 364-369. doi: 10.1089/cyber.2012.0025

Tamborini, R., & Skalski, P. (2006). The role of presence in the experience of electronic games. In P., Vorderer, & J., Bryant, (eds). Playing video games: Motives, responses, and consequences (pp. 225-240). Mahwah, New Jersey: Lawrence Erlbaum.

Are Multiplayer Games More Immersive?

Imagine you are at home and decide to play a video game. Let’s say you choose a first person shooter or a survival horror. You grab a gamepad, sit comfortably in front of the TV and launch the game. At the beginning it’s hard to focus. You are still thinking about friends and work. But, after a couple of minutes, things starts to change. You start paying more attention to the game details. Each movement and sound coming from the TV become more important than anything happening in the real world. And while at first you were struggling to remember the controls, now you hardly even notice the gamepad in your hands. Gradually, the objectives of the game become more important, the dangers more scary, the victories more rewarding. Your heart beat increases and your hands start to sweat. The outside world is virtually non-existent. For a moment, it’s like you are inside the game. You are immersed, and it’s awesome!

Amensia: The Dark Descent provides one of the most immersive experiences available.

The experience of immersion is among the most valued by both players and developers. Over the years, it has motivated a considerable amount of studies. As a result, there are now several theories that try to explain why it happens. Some think it’s because the increased focus on the virtual stimuli makes the technology providing the experience less noticeable. In other words, the player become less aware that the images are coming from the TV or that he is using a gamepad to control the avatar. Others believe it has something to do with how the constant stream of data allows the player to build representation of the virtual world so complex, that it starts being confused with the real world.

Research has also allowed experts to distinguish from different types of immersive experiences: Flow (in which the user’s cognitive resources are highly concentrated in the virtual task), emotional involvement (when the game events manage to elicit intense affective reactions), character involvement (a high level of attachment developed towards the characters) and spatial presence (the feeling of being inside the game).

Games like Kaboom! and Resident Evil 2 tend to provoke different types of immersion.

Spatial presence is probably the most known type of immersive experiences. Thanks to scientists, now we know a lot about how graphics, sound and controllers affect it. However, the effect of social variables have been frequently excluded from these studies. There is little information about how the presence of another human affects spatial immersion in either cooperative or competitive matches. This even though millions of dollars are spent every year to further improve the quality of modern multiplayer games.

Apparently, the nature of the opponent can significantly affect how we see and experience a game. Playing against a human instead of a computer, for example, made players expect a game to be more challenging (Ravaja, et al. 2006), as well as experience more positive and intense emotions (Ravaja, et al. 2006; Ravaja, 2009). This might have had something to do with how the introduction of a human opponent made what happened in the game more socially relevant. Victories and defeats could now be seen as signs of being better or worse than the opponent. This would have made the game more important (as it had a bigger effect on self-esteem), exciting (as the player was more afraid of losing or missing the opportunity to win) and (if the challenge wasn´t overwhelming) rewarding. This is congruent with a recent neurological study in which thinking the opponent was human apparently made victories more rewarding (Kätsyri, Hari, Ravaja & Nummenmaa, 2012).

The importance of human opponents resides in their capacity to make the game socially relevant.

Playing against a human opponent also increased the level of attention and the feeling of being inside the game (Ravaja, et al. 2006; Ravaja, 2009). Maybe the additional importance given to the game made the players pay more attention to it, reducing the relevance of any real world stimuli. This would have gradually made them less aware of the mediating technology (the hardware and software providing the experience), giving the impression that the game was real.

Another explanation could be that increased emotional arousal on its own caused the effect. As we know, video games try to elicit feelings that are congruent with the depicted scenario. Players are supposed to be afraid of their avatar’s getting shot and experience relief after successfully overcoming a difficult jump. It’s possible then that the experience of more intense emotions strongly related to the situation portrayed (whether it was a football match or a futuristic battlefield) made the game feel more realistic.

The level in which the player knew the opponent had a similar effect. Playing against a friend, for example, elicited more positive and intense emotions than playing against a stranger (Ravaja, et al. 2006; Ravaja, 2009). It was also accompanied by an increased level of engagement (Ravaja, et al. 2006; Ravaja, 2009). One possible explanation is that comparisons with a friend had a higher impact on self-esteem. This would have made the game more important for the players, forcing them to pay more attention.

Few things engage the player’s attention like racing against a friend in Super Mario Kart.

It’s important to notice that the positive effect friendship had on spatial presence disappeared when the opponent was located in a different room (Ravaja, 2009). This suggests that emotional arousal and engagement influenced immersion only when the other player was physically present. It’s possible that being able to see the opponent’s reactions made the player experience emotions more related to competition, increasing it’s effect on engagement and/or immersion. Maybe watching his friend laugh at him made the player care less about the comfort of the chair and more about winning the next round.

Being in the presence of the opponent was also accompanied by significantly more positive emotions (Ravaja, 2009). The result is interesting as being able to see the opponent’s reactions would have exposed the player to both negative and positive feedback (like the enemy´s facial expressions of mockery and frustration). Maybe the increased effect the game had over self-esteem just made it more exciting. A feature considered desirable in a game.

Local multiplayer games allow, for better or worse, to see the reactions of the opponent.

There are many reasons to study how spatial immersion works in multiplayer games. It could help us better understand, for example, how important social comparison is for emotional arousal and engagement. It could also help improve the quality of multiplayer based games, whether they are online, split-screen or turn-based; competitive or cooperative. Either way, it all depends on how attractive the subject is for players and game designers, and how interesting it becomes for experts in the field of human computer interaction.

References:

Kätsyri, J., Hari, R., Ravaja, N., & Nummenmaa, L. (2012). The opponent matters: Elevated fMRI reward responses to winning against a human versus a computer opponent during interactive video game playing. Cerebral Cortex, 3(12):2829-2839. doi: 10.1093/cercor/bhs259

Ravaja, N. (2009). The psychophysiology of digital gaming: The effect of a non co-located opponent. Media Psychology, 12 (3), 268-294. doi: 10.1080/15213260903052240

Ravaja, N., Saari, T., Turpeinen, M., Laarni, J., Salminen, M., & Kivikangas, M., (2006). Spatial presence and emotions during video game playing: Does it matter with whom you play? Presence: Teleoperators and Virtual Environments, 15 (4), 381-392. doi: 10.1162/pres.15.4.381

Game Transfer Phenomena: When your brain just won’t stop playing.

Do any of these experiences sound familiar:



a)   You see a map and immediately start searching for natural resources for your army.

b)   A wooden bat reminds you of zombie killing, not baseball.

c)   You see a pickaxe on a hardware store and it surprises you how un-pixelated it is.

d)   Every oxygen tank or barrel of fuel you see prompts you to aim and shoot, expecting a huge explosion.

e)   All of the above.



These descriptions make reference to an apparent common occurrence between frequent video game players. They involve alterations of perceptions, thoughts or behavior triggered by real life stimuli that somehow resembles video game objects or situations. This is what Angelica Ortiz de Gortari (2012) describes as Game Transfer Phenomena. Although we all probably have either experienced or heard about this type of experiences, there isn’t much available information about them. This prompted Ortiz de Gortari (2011) to start collecting video game player’s reports about this type of events. Her research helped discover different types of GTP experiences and the results are quite interesting.

In the area of visual perception, players described experiencing distortions of real world elements based on video game content. These included seeing objects as if they were pixelated, with color outlines or even lagging. In this circumstances, a stimuli associated with gameplay experience seems to be what triggers the phenomena. For example: A player reported seeing the Mass Effect dialogue wheel every time someone talked to him. Sometimes the appearing elements had a fixated position in relation to real objects, like health bars floating above people’s heads (Ortiz de Gortari, & Griffiths, 2012).

This neighborhood is starting to look really “blocky”.

Interestingly enough, stimuli from one sensorial modality occasionally triggered a GTP experience from another one. For example: Seeing the letter Y in yellow (the color of the Y button in the 360 controller) or listening to music on the car only to watch the road transform into moving frets (Ortiz de Gortari, & Griffiths, 2012). This type of relationship between different sensorial modalities is similar to what happens in synesthesia, were people report perceiving a sensory stimuli together with one from another sense (like seeing colors while listening to sounds).

While some visualizations occurred when trying to sleep, many players reported them during the day, as soon as they closed their eyes (Ortiz de Gortari, & Griffiths, 2012). This may be a sign that the lack of competing stimuli plays an important role on the generation of GTP experiences. Visualizations also occurred hours or even days after having played (Ortiz de Gortari, & Griffiths, 2012).

Auditory experiences were present too. Players reported hearing video game sounds (like the achievement sound of the Xbox), sometimes in constant repetition. They also described body sensations associated with the activities executed in the game, like feeling the movements performed in “Armored Core” while trying to sleep (Ortiz de Gortari, & Griffiths, 2012).

For some players, real world stimuli triggered intrusive thoughts. These involved interpreting the world in terms of video game experience (confusing an airplane with an UAV from Call of Duty), making illogical conclusions based on in-game situations (thinking territorial expansions were needed for real food to run out) or expecting video game related events (like waiting for the Spy from Team Fortress 2 to attack from behind). It’s also frequent to think about applying video game tools to real life, like using the gravity gun or grappling hook to reach for objects. In some cases players temporarily dissociated with reality and performed actions as if they were in the game. For example: A player described how he walked in the street towards a bike with the intention of using it, before realizing he wasn’t playing GTA anymore (Ortiz de Gortari, & Griffiths, 2012).

I want to eat something but I’m busy. I could really use a portal gun now.

One way to explain all these experiences is through classical conditioning. In this basic learning phenomena a stimuli that didn’t produce an automatic response is repeatedly presented before a stimuli that does. This will eventually allow the paired stimuli to provoke the reflex on its own. For example: If we repeatedly make a clicking sound just before blowing on someone’s eye, the click alone will soon be able to make that person blink. Video games constantly expose us to paired stimulus. More specifically, pairs of stimulus and actions: Every time we want to say something to another online player we have to press a specific key. As a result, the player learns to respond (with speed and precision) to the in-game stimuli (Ortiz de Gortari, & Griffiths, 2012). This might be why occasionally, when a player wants to say something in the real world, they reach for a non-existing keyboard.

GTP can sometimes take the form of very intrusive and upsetting experiences. A participant, for example, reported going to sleep only to see the minecraft grid as an overlay to his room. He couldn’t fall sleep until the furniture was moved into the right place. Additional research could help identify what gameplay characteristics tend to produce these negative effects, benefiting both players and developers (Ortiz de Gortari, & Griffiths, 2012).

Another negative aspect of GTP experiences is that they are easy to confuse with mental disorders. Intrusive and persistent experiences, for example, could be mistaken with obsessive symptoms. An additional source of confusion could be the unpleasant after-effects (headaches, disorientation and nausea) a group of players reported immediately after playing (Ortiz de Gortari, & Griffiths, 2012). These could be taken as signs of an abstinence syndrome, potentially increasing the belief that video games can be addictive.

GTP Adventures allow players to share their experiences using cartoons.

Further studies are required to help professionals distinguish between Game Transfer Phenomena and mental disorders. But there are more positive reasons to promote research on this field. Advances in the study of Game Transfer Phenomena could expand our knowledge about the basic mechanisms of human cognition (Ortiz de Gortari, & Griffiths, 2012). It could also help us understand why in-game learning manifests only in particular circumstances. A discovery that could have a major impact in the development of reliable video game based learning and therapeutic programs.

Note:

Angélica Ortiz de Gortari has an online blog were she further explains the different types of GTP experiences and releases news about her ongoing research. She has also started “GTP Adventures”, an online platform were you can describe your own experience by using cartoons. You can find both sites on the following links:

Game Transfer Phenomena blog

GTP Adventures

References:

Ortiz de Gortari, A. B., Aronsson, K., & Griffiths, M. (2011). Game Transfer Phenomena in Video Game Playing: A Qualitative Interview Study. In: International Journal of Cyber Behavior, Psychology and Learning. 1(3), 15-33.

Ortiz de Gortari, A. B., & Griffiths, M. D. (2012). An Introduction to Game Transfer Phenomena in Video Game Playing. In: J. I. Gackenbach (Ed.), Video Game Play and Consciousness. NY: Nova Publisher.