Participants play a simple game to collectively create music on real musical instruments over the internet. Players can join together on a video chat application to see and hear the band of acoustic, electric, and electronic instruments respond in real time.

bandy consists of 5 main parts
- A mobile web app where players play a simple paddle ball game
- Another web application collects all of the players’ game states and creates musical events from them
- Software processes the musical events and distributes them to controllable musical instruments
- A band of commercial and custom acoustic, electric, and electronic musical instruments reacts to the player gestures to create musical compositions
- A live video feed allows players and visitors to see and hear the instruments responding to the players’ actions in real time

Over the past few years I’ve moved geographically further away from my friends and musical collaborators. I was missing making music with them and looking for ways to play music remotely. Our past year spent inside has given me the opportunity and the push to learn some of the new available technologies and see how they apply to playing music together.
Some of my main goals in these explorations were maintaining the sense of connection and the joy of discovery in playing and improvising together, making it simple for less technologically adept musicians to play music together remotely, and to allow anyone, not only trained musicians, to engage in making music together.

The web applications for bandy are written in standard HTML5/CSS/JavaScript. They take advantage of the Firebase hosting platform and realtime database to serve the content and collect the status of each players’ game. I implemented a simple paddle ball game where players contribute to a musical composition by bouncing a ball against colored boxes. Each time a box is hit it generates an event that is sent to a piece of software that acts as a performance controller.
The performance controller uses the Magenta machine learning libraries’ Piano Genie to create musical information based on the events from players’ games. Piano Genie is trained on 1400 pieces of music and simplifies the 88 keys of the piano into only 8 keys. This means that simple gestures can be translated into note sequences. As the game events enter the performance controller Piano Genie translates them into musical events in MIDI format.
The MIDI information from Piano Genie is processed in various ways for a set of controllable instruments to create specific compositions. The band of instruments can be viewed via video conference so players can see and hear the music they are creating in real time and interact with their bandmates. The intent is that they will play using the web browser on their phones so that they can join the video conference on a computer. The games can also be played using any modern browser on most platforms.
The music generated by the system is of a particular aesthetic because of the music the machine learning model is trained on. This is an intentional decision to make the result feel familiar to the expected audience. Since the system can accommodate any number of players this can have a significant effect on the musical result. Limitations are intentionally imposed by the game mechanics allowing multiple players to interact at the same time while keeping the musical result varied and dynamic. The “compositions” are differentiated by the choice of instrumentation and how the musical events are orchestrated within the ensemble. Some of the most exciting moments are when the system breaks out of its western harmonic patterns and modulates seemingly at random or breaks into a streaming cadenza when a player’s ball gets stuck between blocks and creates a flurry of events.

It took about two weeks to create the basic software for bandy. During those two weeks I was also able to create a prototype of a MIDI controlled electric guitar to add to the band. Following development of the software components I’ve been playtesting with different numbers of players. Watching people of different ages, backgrounds, and technical skill levels has led to a few adjustments to bandy. I will be adding a simpler game mode for those less adept at manipulating the phone or those who want to pay more attention to what the instruments are doing. I’m also adjusting the interface to offer more information about the events, like Maker
Music Festival, this will be a part of. At the moment, bandy is intended as a facilitated performance event because of its dependence on the physical instrument setup.


Interactive performances where you can join in and play with bandy will be at the following times

Saturday, May 15th
12 noon PST - Live Zoom Performance
1 PM PST - YouTube LiveStream Performance

Sunday, May 16th
12 noon PST - Live Zoom Performance
1 PM PST - YouTube LiveStream Performance

Information for connecting will be available before the performances through the bandy web app at
http://www.bandy.camp

Zoom Performances can also be accessed through https://www.makermusicfestival.com/
YouTube Live Performances also can be accessed through https://youtu.be/P1xaUXcqj-w