Is videogame a musical instrument ?

/!\This article is the script of my intervention in Game Dols Advance episode 7.

Human beings have always made various and varied noises punctuating their very banal existence. And one day to counter the sound banality of human existence, in his absolute mercy, Shigeru Miyamoto invented video games. And it wasn't such a good idea because it didn't help the fascists become less stupid.
"Okay, that's a nice presentation, Rose, you'll tell me, but how do we make the musical instrument with video games?"
First, what is a musical instrument? I will tell you that it is a tool which, in a context considered musical, has an action on this musical phenomenon. In this definition, everything can be considered as a musical instrument if the musical consideration window is sufficiently opened. Like all the tools, there are several ways to use them, some people play guitar by pinching the strings, rubbing them or hitting them on the resonance box. It's the same for a hammer that can be used both to plant nails or percussion tool, among others. For the video games, it's the same. Each video game is a tool, a different sound tool. It is rather magical to consider the video game as a musical instrument, because each video game is equivalent to a musical instrument with its specificities of sound emergence. But are video games all equally permissive as a sound creation tool?

To define all this, I tried to write a chart of permissivity of video games to be practical musical instruments. Here are some questions that have come to my mind and are not exhaustive, I think.

Can you easily play with music launch triggers? This is the way I find the most funny to use a video game. Often it's just getting in and out of a room, sometimes with a loading time, or a quiet area between two triggers. By extension, one can also wonder, if there is dynamic music, if the music can return to a previous stage.
Can music/sound design/voices be lowered individually? This is an option that we find in a lot of games but not always in recent games. Getting places of silence can be complicated in some games without this option, sometimes music can be removed but not sound design, so you have to figure out with all these options.
Are there any game elements dedicated exclusively to noise production? Typically, elements of the game that only serve that purpose, optional traversable sound areas like sound slab mechanisms like in Minecraft, virtual musical instruments like Lyre in Hades.
Is there a way to make sounds with other people ? It can consist of multiplayer online, but also being several players to hold the controller and make input together without any problem, such as the Music 2000 music jam mode or the Animal Crossing multiplayers.
Is maximum permissivity directly accessible at the beginning of the game? Typically, can I be chaotic right from the start of the game or does it take a long time?
Are the emerging sounds necessarily related to the identity of the game? Some games with a will to anchor in the real have a realistic sound design so tending to be less unique, like war simulators like Call of, but also realistic daily simulators like the Sims or Unpacking. Animal Crossing moves away even though there is a great work to emulate sounds, but the game wants to be more cartoonesque in its way of inhabiting sounds and reactions.

Du coup pour réfléchir à tout ça, j’ai fait plusieurs streams pour étudier quelques jeux et leur permissivité : Angeline Era, Animal Crossing New Horizon, Super Smash Bros Ultimate, Fire Emblem : Three Houses, Music et Pokemon Émeraude.

Let's start with Fire Emblem: Three Houses.
I remembered that the game was accessible and easy sound triggers between the market square and the monastery of Garreg Mar. After passing the tutorial and skip the different phases of dialogues and animations, you can travel between the two places after a good time of play. So we start turning round and it turns out that the stairway separating the areas is a total area of silence that breaks a bit the turn of the music triggers. It's possible, but it's laborious. But, on the other hand, going from silence to an area, going back in silence and then returning to the area will create loops to launch, and we can already play music like that. Here it is :

Video capture of a Rose stream, showing a playthrough of Fire Emblem Three Houses, we see Byleth turn in circles in a defined area to trigger invisible sound triggers.

As a nice touch, we can also specifically lower voices, music and sfx to either play with sound design, voice or just music. Fire Emblem Three Houses is therefore a fairly permissive game in its way of being played as a musical instrument. There is little sound design that does not relate to its video game identity apart from the pace sounds and specific moments of battle.

Then I wanted to try Super Smash Bros Ultimate. My attempts to play with the sounds and the music inside the fighting moments were unproductive. However, it is in the menuing that there is a strong potential to play with. First of all, if you remove the sounds and voices from the game, you can have fun playing with triggers to launch music inside the hub of the game. It's one of the few games to offer different music for the same menu with, in addition to that, a music launch probability selector. That means there's a randomness around the triggers. And that's very funny.

Video capture of a Rose stream, showing a playthrough of Super Smash Bros Ultimate, we see a random song selection menu at the start of the game.

On the other hand, what is really very funny too, is that there is a section of the menu for audio tests, and especially voices. AI don't think I'm the only one who did this. I'll just let you listen to what it can give.

Video capture of a Rose stream, showing a playthrough of Super Smash Bros Ultimate, we see an audio file launch menu, the wii fit trainer section, and Rose moves from voice 2 to voice 3 frantically.

Really, I can do this for a very long time... SSBU is therefore a very permissive game in its way of being used as a musical instrument.

Animal Crossing New Horizons.What do you think? Permissive or not? Well, it's a very unpermissive game. There is almost no menuing and there are no sound parameters, there is almost no silence, music is necessarily imposed on the island except the most remote corners of the beaches. The only areas of silence are the players' houses. The game has very little music easy to interact with except the possibility to put music in a radio that will superimpose on the island's music. We can also play Marie's little synth at the town hall.

Video capture of a Rose stream, showing Animal Crossing New Horizons, we see a frog keyboard where we can define note heights.

However, it is possible in houses to have absolute silence or to overlay several musics and sounds, such as putting in a room a radio that will play random music, a CD player that will play specific music and a television that can also play at certain moments of music.
The game however has two very unique things for it related to a particular feature, it is the random behavior of certain sound objects. Gyroids and musical instruments.
First the gyroids. These are the things that you pick up in the ground, grow and you can pose wherever you want, and react to sounds, especially intra-diegetic music. It is therefore possible to put a variety of gyroids in a room in his home, with a sound station that broadcasts music and listen to the behavior of gyroids. Unfortunately, there will necessarily be the sound of the music and not just the sound flow of the gyroids responding to this sound stimuli. But you can already see a nice random sound arrangement game.

Video capture of a Rose stream, showing Animal Crossing: New Horizons, we can see the villager in her decorated house, a gyroid whistling to the rhythm of music.

Animal Crossing New Horizons also has an opportunity to create concert spaces in homes. Since there is the possibility of making absolute silence, it is possible to put playable musical instruments, and to play in multiplayers. I am also looking for comrades to perform concerts on Twitch. If you have the game and a nintendo online, pok me via the contact section of this website ! When you click on a musical instrument, the instrument emits a random sound in a major range. What's funny, too, is that when you play multiplayer on the internet, a note issued and heard by one player on his version of ACNH will be potentially different from that heard on another switch. Which makes each sound performance different. ACNH is close to a small Cageian dream in its way of offering a musical experience but is not a very practical musical instrument. By itself. Not practical/20.

The day after this first test, we continued with Pokemon Emerald. Well, it was pretty sad. The parameters do not allow to vary the intensities of music and sound design. At the beginning of the game, we have access to about 10 sounds and 3 musics, with quite complicated sound triggers to play with. It's game boy advance, so don't ask too much either. But you can still have fun bumming the walls on the music of the game.

Video capture of a Rose stream, showing Pokemon Emerald, we see the trainer frantically hitting his mother, brother, table, and Rose opens the menu from time to time.

Next was Music. For those who don't know, it's a Playstation 1 game that made it possible to compose music, especially electro, released in 1998. I spent my first composition experiences there and wondered sincerely if a compositional software had any potential to be used as a musical instrument. The manoeuvrability with the controller is particularly terrible, and there are some emulation bugs on the placement of MIDI information for many percussions. And it's really at the last moment when I thought it was useless to dig that I came across the function to move when the music is playing. It gives this thing here and it's particularly funny to play with :

Video capture of a Rose stream, showing Music on PS1, we see the timeline moving forward and backward frantically.

In fact, I would recommend playing Music and Music 2000 if you have the possibility, these are great games to test the basics of composition. There are probably more interesting and accessible tools like Garage Band, but it's a good starting point to understand how to build songs.

We finished on the demo of Angeline Era. Well, it was a little bit of a pretext to show Melos Han Tani and Marina Kittaka's latest work from Analgesic Production, which I have already praised a lot on stream on Game Dolls Advance. People will think I only know Pokemon and this studio (that's kind of true thou). But I was thinking that with all the tools we just saw before, all these different approaches, and the creativity there is in Analgesic Production we'd end up finding something. Since there is possibility to edit the volumes of the SFX and the music, I wanted to test the transition from the hub to the first level and I wasn't disappointed. Because the hub's music continues to evolve between each round loading while the level 1 music always starts in the same way, it causes an interestingever changing loop.

Video capture of a Rose stream, showing the demo of Angeline Era, we see the protagonist coming in and out of the hub towards the forest.

From this experience, I was able to conclude on several ways to play with the music instrument video game.

Play around the music triggers, which consists of crossing certain triggers with the avatar.

Play around the speed of the music, which consists of playing with its avatar to trigger the music more or less quickly. (Here with Music, the avatar becomes the playbar, but one can think of Untitled Goose Game with its dynamic music according to the actions performed)

Play with sound arrangement of a space. This means making available sound elements that can interact together. Like with the giroids of Animal Crossing.

Bump the walls like in Pokemon.

I think I don't have much more to say to conclude except that it was super fun to do and that I think I'm still looking for funny ways to play music via video game. And I invite you to find your practices, and share them, I would love to see what you can do.

[EN]Want to leave a comment ? You can write it in my guestbook !
[FR]Tu veux laisser un commentaire ? Tu peux le faire sur mon livre d'or !