Controlling game music in real time with biosignals

Effective game music is typically adaptive, interactive, or both. Changes in game music are usually influenced by the current state of the game or the actions of the player. To provide another dimension of interactivity, it would be useful to know the affective state of the human player. Biosignals are continuous signals generated by a person that can be measured over time, and have been shown to reflect affective state. This project demonstrates that control signals can be gathered from the player and mapped to musical parameters. Using a heart rate sensor and galvanic skin response sensor built from open source designs, we have used biosignals to control music playback while playing four games from different genres.

A system for controlling game music with biosignals is computationally cheap, and can provide data that is useful to other game systems. The prototype developed for this project is basic, but with further research and development, we believe such a system will greatly improve the immersive experience of video games by involving the player on a new level.