There’s a screencap of Call of Duty that has circled the internet for a long time now:

But what’s now a meme in fact underlies a deeper problem in how we represent actions in games. It’s a well-discussed problem: sometimes it’s difficult to choose an input scheme that matches well with the verbs we want to give the player control over, especially when input is so standardized. After all, we can’t create a new controller for every new game.

“Press F to pay respects” isn’t just a case of a lazy, detached control scheme. It’s an example of a problem that the developers couldn’t figure out how to solve: how do you keep the player engaged during story sequences in a game where all the usual verbs are combat-related? How do you get the player to pay respects when their interface with the game world is a gun? How do you even represent paying respects without speech and facial expression?

I don’t have the answers, but I do want to explore a broader topic surrounding this. How do we create games where the player’s actions do more than just control the character; how do we create games where the player’s actions are an active part of the narrative itself?

We’re past the question of whether games are art now. I don’t think it makes sense to continue asking that question; I think it’s fairly obvious that at the very least games have the potential to be artistic. The next question is: how do games differ from other forms of art?

It’s a similar question to one that film had to answer in its early days. It’s one that all forms of media have to answer in their own way. So I want to really dig deep into the question and attempt to understand some of the fundamental capabilities that games have, especially in contrast with other forms of art, and with the understanding that my view is just part of the answer.

Before beginning, I also want to clearly lay out my intentions for this. The question of what is art is too nuanced and misunderstood to even attempt to pin down. The question of what makes a meaningful experience is similarly difficult to describe. There are no true answers to these questions, and it would be folly to assert my own views as fundamental and all-encompassing. Rather, I want to describe one method of thinking about one type of meaningful experience: an experience in which the audience of a work of art has an emotional reaction that helps them to relate to the art itself. There are meaningful experiences to be had playing Pong, Call of Duty, and Frog Fractions. Each of those, in their own way, are art. The types of games I’m more focussed on in this essay are single-player experiences with a narrative and a protagonist, and which have some type of character development. With that said, I think these tools I will present are applicable to a wider range of game types, with some modification, though they may not be the best fit for the job.

I’m including a clip here of Alfred Hitchcock talking about the power of montage. Montage, as Hitchcock describes it, “is the assembly of pieces of film which when moved in rapid succession before the eye create an idea.”

You might say that film’s core tool in storytelling is the montage. Together with framing, film has the ability to present an idea through implication. It is uniquely capable of limiting our view to the most minute details, as well as expanding it to the grandiose, and thereby providing a picture of the story which is more than simply characters, setting, and plot. It’s an abstract, difficult-to-grasp idea, and without the expertise of Hitchcock I can’t truly express it in words.

Montage is what differentiates film from theatre. Film and theatre, of course, share a lot in common: they are visual media that move through time. They differ from paintings in photography in the time dimension, and all of these media differ from literature, poetry, and music in that they are visual.

Photography and paintings are uniquely able to capture the real world in a single moment—paintings differ from photography in that they can visually describe abstract ideas. Literature’s core strength, you might say, lies in its ability to tell us directly what the characters are thinking, and poetry and music are capable of expressing emotions entirely through language and sound.

That’s not to say that there isn’t overlap between these media. Literature can, in a sense, contain montage, and it can do as much as poetry can with language. Theatre and film can contain every other type of media I’ve just mentioned.

But when an idea of a story comes to me, I tend to ask the first question of “what is the goal of this story, and what medium is best able to express that theme?” And my thoughts are something like this:

  • To explore the thoughts of a character, choose literature.
  • To explore an intense, interesting plot, choose film or theatre.
  • To capture something visually beautiful, choose visual art.
  • To elicit an emotion, choose poetry or music.

So what’s the goal that makes you choose games? What is the goal of a story which games are most able to capture?

I believe that games are capable, more than any other medium, of getting the audience to sympathize with a character. The audience does not just experience the thoughts of the character: they are able to understand how a character reaches conclusions about their world. Games are the medium of sympathy.

The next question deals with how we reach that conclusion. What tools do games have at their disposal which allow audiences to sympathize with the character? The obvious answer is interactivity, but I want to dig deeper than that. How do we use interactivity in order to elicit sympathy in the player? And I want to stress again that these are my own views; just as two painters can use color to entirely different effects, so too can two game designers use interactivity to entirely different effects.

My own answer is this: the game should enforce a parallel between the player’s actions and reactions and the character’s actions and reactions. The player’s interaction with the game should in some way mimic the character’s interaction with the game world.

There are two sides to this. The first is what I will call mimetic action, (from mimesis, which I do not mean in the sense of Roger Giner-Sorollla’s essay on interactive fiction). What I mean by mimetic action is that the player’s input or interaction with the game should mimic the actions made by the character in the game world.

A more fundamental example of this might be the use of an analog trigger to fire a gun. The action is mimetic; pulling the controller’s trigger feels like pulling the trigger of a real gun, or at least it feels more so like pulling a gun trigger than pressing a button does. Another example might be the act of physically exploring your surroundings to find Pokemon in Pokemon Go. And this can go more abstract, too: is tapping your phone a mimetic action for bumping the bird in Flappy Bird? Perhaps — these jumps happen so quickly and so easily that a tap is the most mimetic action that’s feasible for the game.

A better example might be the six-button fighter. I’ll draw a distinction here between replication, which would be a direct 1-to-1 correspondence between player and character action (ie, punch forward to make the character punch forward); and mimesis, which might be more abstract. The six-button fighter is the latter. I’m referring to the input mechanics of games like Street Fighter, King of Fighters, Skullgirls, etc.. In these games, complicated actions are performed by moving the joystick in certain patterns while pressing a button. Different patterns and different combinations of patterns and buttons result in different character actions.

These controls are notorious because they are difficult to perform and require great timing to pull off. On top of that, these fighting games require you to understand your opponents actions and react to them with such complicated controls. But I would argue that these controls are mimetic because their complexity mimics the complexity of pulling off a cool move in the game world. A replicative action would be found in something like Arms, in which motion controls allow you to physically punch forward in order to make your character do the same.

The second aspect of this mimicking of player interaction and character interaction is what I will call emotional mimesis, or mimetic reaction. What I mean by emotional/reactive mimesis is that the player’s understanding of their current situation should mimic the character’s understanding of their current situation.

This is both easier and harder to describe. I don’t mean simply that we see the character looking happy at the same time that we also feel happy. I mean that the events in the game world which cause the player to feel happy are the same events that cause the character to feel happy–or at least, which we imagine cause the character to feel happy if we cannot directly see the reaction.

A possible example of this can be found in horror games. Little Nightmares presents us with a world which is bigger than the character, and the visual effects help to give that same sense to the player. Staring at the screen, you feel small. Large, creepy monsters sit in rooms you’re about to enter, and when they start chasing you, you feel like running away–which is the only thing that Six, the protagonist, can do. There’s a real sense of adrenaline you get when you have to start running, and you can only imagine that it’s the same feeling that Six has.

But it goes further than that, and I think Little Nightmares actually fails in that regard. A warning of minor spoilers here. Near the beginning, Six enters a room and suddenly becomes hungry. She slows down and must eat. But this is the first time we know that she is hungry. These moments are the only moments in the game in which Six must eat. Had the game forced you to deal with the hunger pangs during other segments, the player might feel some sort of relief when they finally find food. But as it stands, these sections detach the player from the character.

That may be intentional. Part of the beauty of Little Nightmares is finding out more about Six and watching in horror as she transforms into something unsettling. So once again I want to reiterate that this type of mimesis I’m talking about is just one tool among many for making a meaningful experience.

One difficulty here is that this type of mimetic reaction is possible in other media. If Little Nightmares were a film, we would still feel that visceral reaction to the antagonists of the game. So how is mimetic reaction particularly suited towards video games?

Here I would like to present the cult-classic Mother 3, and one section of the game in particular. Once again, spoilers ahead. One of the antagonists, Fassad, offers a product called a Happy Box to the grieving villagers of the Nowhere Islands. Four of the villagers accept. For this section of the game, the player controls a monkey ‘working’ for Fassad. The monkey is tasked with delivering the boxes to the villagers, but can only carry one at a time, slowly—and the player is forced to control the monkey for the whole of it, slowly delivering each package one at a time to the correct place. No matter how quickly the task is completed, the monkey is punished.

Here is a video from a let’s play of the game, which includes a timer. The player even explores other areas and talks to other villagers. When they finish, the timer reads just under 9 minutes—the time limit was 22 minutes—and the monkey is still punished.

Of course, when the monkey is punished, the player feels cheated. You know that you did it as fast as possible. You feel punished yourself. It’s a true mimetic reaction. But more than that, I don’t think this would be replicable in a film or a book. Imagine watching the monkey in a film, slowly delivering each box. The audience could not help but imagine that perhaps he was just unlucky, or could have moved quicker after all, or took a wrong turn. They might see from the film that the task was impossible, and perhaps they could even feel, through montage, how difficult the task was. But having control of the monkey gives the player the option to carry out the task in whatever way they see fit.

The audience does not just watch the monkey move slowly, wondering if there were a faster way of delivering the boxes. There is no question in the audience’s mind: no matter how differently the monkey might carry out the task, it is impossible not to be punished. The player can try every possible route. They can press all the buttons on the controller and attempt to run, but they can’t finish without being punished.

Being face to face with that impossibility–having felt how unfair the task was–that’s the power of mimetic reaction in games. The sympathetic understanding that the player feels with the monkey is strongest because the player experienced the task themself. More than any of the other times that Fassad shocks the monkey, this one cuts deep.

Call of Duty is not a game that’s meant to provide a deep, meaningful narrative on war. It’s an action game for the fun of it. And that’s okay—as I said at the very beginning of this essay, this is just one tool to provide one type of meaningful experience. But it’s something that I think is worth exploring further, and at the very least it’s an idea that I’ll be keeping in my head when designing my own games—at least, the ones that I intend to provide that kind of meaningful experience. Let me know what you think, and tell me about any other games that use mimesis in some way. I’m very interested in hearing what you come up with.