Making own instruments

How sointu works under the hood?

Let's start with a simple example how one would make an instrument in a modular synthesizer, visualized as a graph:

graph LR;
    Envelope-->Multiply;
    Oscillator-->Multiply;
    Multiply-->Delay;
    Delay-->Out;

Loading

Sointu adopts terminology from 4klang and calls each of the nodes of this graph a "unit". For, we will assume all unit inputs and outputs mono signals and worry about stereo signals later. To synthesize such an instrument, sointu uses a stack-based virtual machine, where the byte code is executed once per sample. The instructions for the previous example:

                       // stack: <empty>
envelope               // stack: envelope
oscillator             // stack: oscillator, envelope
mulp                   // stack: envelope*oscillator
delay                  // stack: delay(envelope*oscillator)
out                    // stack: <empty>

Notice "mulp"; there is an another closely related instruction "mul". The p in the end stands for pop: "mulp" pops the top of the stack and replaces the new top of the stack with the product of the two signals, while "mul" just replaces the top of the stack with the product of the two topmost signals.

In sointu, this example instrument looks like this:

Notice the small numbers after each unit: they indicate the number of signals on the stack.

Stereo signals

If you actually implemented the previous instrument, you will discover that it only outputs sound to the left channel. To output a stereo signal, you would use the stereo version of the "out". However, the stereo version of "out" expects the two signals on the stack: left and right. Thus, we also need to convert the mono signal to a stereo signal, converting one signal on top of the stack to two signals. To do that, we use the "pan" unit:

Almost every unit has a mono version and a stereo version. In general, mono versions modify/add/remove single signals and stereo versions modify/add/remove signals in pairs from the stack. For example, the stereo version of "envelope" just pushes the same envelope twice on the stack.

Unit parameters and bytecode

Each of the unit instructions take also parameters (e.g. "envelope" has stereo, attack, decay, sustain, release and gain). Stereo parameter is only a single bit, while most values are between 0 and 128. Unlike in typical assembly languages that CPUs consume, the instructions and the parameters of an instruction are not encoded in a single stream, but the sointu units (instructions) are encoded in two streams:

1. Command stream, which has single bytes, telling the type of next unit
2. Value stream, which contains variable number of bytes per unit, containing the parameters of a unit

In sointu, the stereo bit is encoded in the least significant bit of the command stream, but all other parameters are in the value stream.

The instruction encodings are split into two streams, because this compresses better: for many units, values 0, 64, 128 are good "off/neutral/on" values, so grouping them into one stream compresses best.

In the compiled version of the song, the meaning of the bytes in the command stream depends on the patch: only units that are used at least once are included in the jump table of the virtual machine.