Improving timing for music

[JerwuQu]

There has been much discussion in the past about how the audio interface could be made more versatile.

One improvement is in the frequency domain, where the most elaborate (and in my opinion best) solution is #333.

Another improvement that is still left unanswered is the time domain, which is difficult because WASM-4 operates on a tick-basis that is hard to break without also breaking fundamental design decisions.

This is a problem because notes currently need to be aligned to 60th of a second, which greatly limits the BPM choices if you want to stay accurate in tempo.

Proposal #1: timer(fn, ms)

This would, to the user, be a quite simple API which calls any given function after ms milliseconds.
(I briefly mentioned this in an old issue comment.)

Limiting this to one active timer at once is no problem for this purpose, and would probably greatly simplify implementation.

The FRAMEBUFFER buffer and the GAMEPAD flags would be the same as on the last update, and any changes to FRAMEBUFFER or PALETTE wouldn't change what's currently on screen (though they would remain in the next update when SYSTEM_PRESERVE_FRAMEBUFFER is set), meaning no graphical weirdness could happen.

tone could remain as-is, still operating on a tick-basis, and calling tone again would interrupt any currently playing audio on the given channel.
The timing could either be made to align to ticks, or to be offset to whatever the current offset from the tick "grid" is, it doesn't matter too much since it could be corrected the next tick update if needed.

Practically, this could be implemented on another thread that syncs with the thread running update using a mutex, or on the same thread if doing some math and syncing is kept well in check.

Proposal #2: timer as a called function

Make a seprate update (e.g. timer, sequence, tick) whose interval is configurable through a system flag. This would work similarly to timer above both in implementation and usage, but limited to a single timer in the API specification itself.

Proposal #3: tone queue

Another solution to this could be to make a "command queue" for tone and make the duration argument a 10th or even 100th of a tick to greatly reduce the current temporal error. Weaved in with a System Flag to enable the previously referenced new frequency mode, the flag would also enable this new temporal mode which works on a more precise scale.

Practically, this could work as a fixed buffer with a size of whatever the chosen divisor is (10, 100, etc.), which is then checked whenever a new sample is to be played (similar to what is already done with attackTime, releaseTime, etc.). Whenever a new tick starts, the last item in the queue would be moved to the front set as the currently playing to prevent misuse.

As an API user, this means you could queue up multiple tones in an update that would happen in sequence.

This wouldn't have any performance impact at all on the current WASM-4 runtimes, since they already check a releaseTime on each sample iteration (which with this would instead be a time in the tone queue).

Conclusion

In short:

• Both timer proposals either require threading or use of precise timing, but have a very simple API
• The tone queue proposal requires a more complex tone API with an internal buffer

I'm not sure what's best, but I think it's good to have a discussion about this since having music out of sync is sad :C

[JerwuQu]

Proposal #4: Make a music function (#15)

A new API that is able to do everything tone can currently do, but in a sequence with a customizable BPM.

I have no idea if it's even desirable for this new API to overshadow tone in this manner (since it was said not to do so in a comment). If this is OK after these consideration, I could help come up with a proposal for it.

[YesSeri]

This is a problem because notes currently need to be aligned to 60th of a second, which greatly limits the BPM choices if you want to stay accurate in tempo.

What do you mean by this? When is the 60 bps a problem?

[JerwuQu]

What do you mean by this? When is the 60 bps a problem?

In short, because not many BPMs are well divisible with 60.

More proper explanation:
Say you want to have a BPM of 160, and you have want to have at least 4 notes within a beat (very common).
That would come out to 4 * 160 / 60 = 10.66... notes per second, and 60 / 10.66... = 5.625 ticks between each note.
The ticks between each note will need to get rounded to an integer (since you can't play tones between ticks). This means you end up with an uneven distribution, and depending on how bad it is, things will sound out of tempo.

The best solution right now is to stick to a BPM which divides perfectly.

Here's a quick calculation of currently usable BPMs and their "tick waits" (considering 4 notes per beat):

> for (let i = 20; i > 1; i--) console.log(60 * 60 / 4 / i, i);
45 20
47.36842105263158 19
50 18
52.94117647058823 17
56.25 16
60 15
64.28571428571429 14
69.23076923076923 13
75 12
81.81818181818181 11
90 10
100 9
112.5 8
128.57142857142858 7
150 6
180 5
225 4
300 3
450 2

It's not terrible, it's just not flexible. The issue becomes more apparent when you if you need for example 8 notes per beat, or different grooves.

[JerwuQu]

While this issue still exists, it's not as important as I first phrased it to be, and after having thought about it quite a lot more, I've reached the conclusion that fixing it with any of the proposed solutions would affect the simplicity of the architecture of WASM-4 too much.

The "cleanest" possible solution I can think of would be a "high framerate mode" flag for WASM-4 that switches it to run at a 120hz clock instead of a 60hz one. This would of course increase the fps too. It could be written in such a way that it would work on 60hz devices too (but with a half a frame of lag built in).

For now, I'm in personally against adding an extra clock in addition to the update one, so I'm closing this.