Data Connections Between Ableton Live and Minecraft (MIDI & MORE!)

Since late 2020, I’ve been using Minecraft as an immersive musical instrument. However, my original project, void loop () only made use of in-game sounds. Custom sounds had to be added via a resource pack, and the use of custom samples was limited.

In May 2022, I began looking for ways to expand the expressivity of the game. Through the use of the Mineflayer Javascript library, I’ve managed to make robust data connections in and out of the game. In-game chat can trigger MIDI events, report entity locations, and report basically any other game mechanics.

Please enjoy some videos outlining some of the capabilities of these connections.
The most recent two, mcMIDI.js and mcM4L.js are both projects you can deploy yourself. A full set of instructions can be found in each project’s GitHub README.

I have presented on these projects at the following conferences:

• HOPE (Hackers on Planet Earth) – July 2022
• SEAMUS (Society for Electro-Acoustic Music in the United States) – April 2023
• Connect 2023 (Electronic Digital Instrument Summit at Berklee College of Music) – June 2023