Permalink
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
285 lines (205 sloc) 10.1 KB

Emstrument: Scripting tutorial with FCEUX

Preface:

This tutorial assumes you have some scripting/programming experience, although not necessarily in Lua (which is easy to learn), and some familiarity with the way 8-bit games work.

Step 1:

Find a game to start with. Let's start with the iconic original Super Mario Brothers.

Step 2:

In order to make the game musically interactive, we need to find out where values are stored in the game's memory/RAM. There are some references out there, such as other Lua scripts for FCEUX and RAM maps at romhacking.net. For Super Mario Brothers there is a fairly in-depth RAM map here.

Step 3:

Start with a basic script:

    while (true) do
        FCEU.frameadvance();
    end;

Anything inside the while loop will run once per frame. There are 60 frames per second, so if you add 1 to a variable every frame, it will increase by 60 every second (assuming no lag).

Step 4:

Let's first try to make a sound play when Mario gets a coin. According to the RAM map, the memory address for number of coins is 0x075E. We need to read that value, and store it at the end of the loop so we can tell when it has changed:

    lastcoins = 0;
    
    while (true) do
        coins = memory.readbyte(0x075E);
        
        if (coins ~= lastcoins) then
            -- Number of coins increased
        end;
        
        lastcoins = coins;
        FCEU.frameadvance();
    end;
Step 5:

Now let's actually make the script send a MIDI note when a coin is obtained. First we need to include Emstrument and set it up:

    require('emstrument');
    MIDI.init();

Then in the if block, let's play a note:

        if (coins ~= lastcoins) then
            -- Number of coins increased
            MIDI.noteonwithduration(MIDI.notenumber("G4"), 100, 5);
        end;

This plays the note G4, with velocity 100, for 5/60 second (about 0.083 seconds). At the end of the loop we need to send the MIDI messages

        MIDI.sendmessages();
        FCEU.frameadvance();

Full script so far:

    require('emstrument');
    MIDI.init();
    
    lastcoins = 0;
    
    while (true) do
        coins = memory.readbyte(0x075E);
        
        if (coins ~= lastcoins) then
            -- Number of coins increased
            MIDI.noteonwithduration(MIDI.notenumber("G4"), 100, 5);
        end;
        
        lastcoins = coins;
        MIDI.sendmessages();
        FCEU.frameadvance();
    end;
Step 6:

Now we can run the script and see what happens. Open FCEUX, open the lua script, and then run the ROM. For this excercise you are going to want to disable sound output in FCEUX, as that makes it harder to verify that the sounds are being correctly generated by the script.

What happens next depends which DAW you use. Most will accept the MIDI commands from FCEUX/Emstrument without needing any configuration, as long as there is a track that is active. Bascially, if you plug in a MIDI keyboard and the DAW makes sound when you play the keys, the script will make those same sounds. If for some reason the script doesn't register, make sure EmstrumentMIDISource is detected and enabled in the DAW's settings while the script is running.

Step 7:

Let's make the note more interesting by changing its sound based on Mario's vertical position. First we need to find out where in the RAM to find that value: According to the RAM map 0x00CE is Mario's y position. Let's read that:

        ypos = memory.readbyte(0x00CE)

Unfortunately this value isn't as simple as ground = 0 and top of screen = 256. We need to do some math to translate. First let's observe what the values of y are:

        gui.text(0,8,ypos);

This will print some text on top of the game displaying the current y position.

Full script so far:

    require('emstrument');
    MIDI.init();
    
    lastcoins = 0;
    
    while (true) do
        coins = memory.readbyte(0x075E);
        ypos = memory.readbyte(0x00CE)
        gui.text(0,8,ypos);
        
        if (coins ~= lastcoins) then
            -- Number of coins increased
            MIDI.noteonwithduration(MIDI.notenumber("G4"), 100, 5);
        end;
        
        lastcoins = coins;
        MIDI.sendmessages();
        FCEU.frameadvance();
    end;
Step 8:

We see that Mario on the ground has a y position of 176 and near the top of the screen has a y position of 0. So we want 176 to map to 0 and 0 to map to 127, the min and max values of a MIDI control change (CC). This can be done with the following math:

        (127 / 176) * (176 - ypos)

Then we want to make that an integer by rounding (either down or up), and limit its range to [0,127]:

        math.max(0, math.min(127, math.floor((127 / 176) * (176 - ypos))))

This is messy and we might need to do it again later (although not in this tutorial), so we can make a function to do this automatically:

    function roundandclamp(number)
        return math.max(0, math.min(127, math.floor(number)))
    end;

Now let's plug that value into MIDI.CC():

        MIDI.CC(1, roundandclamp((127 / 176) * (176 - ypos)));

We're sending MIDI CC number 1, which is the mod wheel. That means that whatever happens when you rotate the mod wheel on your MIDI keyboard is what will happen to the sound when Mario's y position changes. A good place to test this is the coin room in world 1-1 (go down the 4th pipe).

You may notice the sound changes a lot when Mario jumps very high (past the text at the top of the screen), due to the y position wrapping back around to 256. Resolving this issue is left as an excercise for the reader.

Full script so far:

    require('emstrument');
    MIDI.init();
    
    function roundandclamp(number)
        return math.max(0, math.min(127, math.floor(number)))
    end;
    
    lastcoins = 0;
    
    while (true) do
        coins = memory.readbyte(0x075E);
        ypos = memory.readbyte(0x00CE)
        MIDI.CC(1, roundandclamp((127 / 176) * (176 - ypos)));
        
        if (coins ~= lastcoins) then
            -- Number of coins increased
            MIDI.noteonwithduration(MIDI.notenumber("G4"), 100, 5);
        end;
        
        lastcoins = coins;
        MIDI.sendmessages();
        FCEU.frameadvance();
    end;
Step 9:

Now let's try to make a different note play when Mario jumps on an enemy. There is no convenient "number of enemies" number in the RAM, so we'll have to look elsewhere to figure that out, namely the score.

        score10s = memory.readbyte(0x07E2);
        score10_2s = memory.readbyte(0x07E1);
        score10_3s = memory.readbyte(0x07E0);
        score10_4s = memory.readbyte(0x07DF);
        score10_5s = memory.readbyte(0x07DE);
        score10_6s = memory.readbyte(0x07DD);
        score = 10*score10s + 100*score10_2s + 1000*score10_3s + 
                10000*score10_4s + 100000*score10_5s + 1000000*score10_6s;

The score, like many early games, is not stored directly as an integer, but is instead stored as one digit per byte in the ram, presumably to make it easy to read and display the score as VRAM tiles. That means we have to read each value and multiply by the appropriate power of 10, and add them all together to get the score.

Anyway, when we jump on an enemy, the score increases by 100. So we need to store the score at the end of every frame:

        lastscore = score;

And then compare the last score with the current score:

        if (score == lastscore + 100) then
            -- score increased by 100
            MIDI.noteonwithduration(MIDI.notenumber("D3"), 100, 10);
        end;

Note that this does not take into account the behavior where jumping on multiple enemies in succession applies a score multiplier. This means that 200 points is ambiguous; it could either mean stomping on a second enemy or getting a coin. Resolving this is left as an excercise for the reader.

Full script so far:

    require('emstrument');
    MIDI.init();
    
    function roundandclamp(number)
        return math.max(0, math.min(127, math.floor(number)))
    end;
    
    lastcoins = 0;
    lastscore = 0;
    
    while (true) do
        score10s = memory.readbyte(0x07E2);
        score10_2s = memory.readbyte(0x07E1);
        score10_3s = memory.readbyte(0x07E0);
        score10_4s = memory.readbyte(0x07DF);
        score10_5s = memory.readbyte(0x07DE);
        score10_6s = memory.readbyte(0x07DD);
        score = 10*score10s + 100*score10_2s + 1000*score10_3s + 
                10000*score10_4s + 100000*score10_5s + 1000000*score10_6s;
        coins = memory.readbyte(0x075E);
        ypos = memory.readbyte(0x00CE)
        MIDI.CC(1, roundandclamp((127 / 176) * (176 - ypos)));
        
        if (coins ~= lastcoins) then
            -- Number of coins increased
            MIDI.noteonwithduration(MIDI.notenumber("G4"), 100, 5);
        end;
        
        if (score == lastscore + 100) then
            -- score increased by 100
            MIDI.noteonwithduration(MIDI.notenumber("D3"), 100, 10);
        end;
        
        lastcoins = coins;
        lastscore = score;
        MIDI.sendmessages();
        FCEU.frameadvance();
    end;
Step 10:

That's it for this short tutorial! Obviously, replacing sound effects isn't the most interesting or musical thing in the world, but by combining different data from the game in conjunction with modern digital music workflows, an infinite number of visual musical possibilities can be created.

Thanks for reading! For further help, the example scripts are all commented and should be fairly easy to follow/copy-paste from. Check out SMB_piano_horizontal.lua and SMB_piano_vertical.lua scripts for more on scripting Super Mario Brothers.