_ __ _ __ ___ / _|_ __ __ _ _ __ ___ _ __
| '_ \| '__/ _ \ |_| '__/ _` | '_ ` _ \| '__|
| |_) | | | __/ _| | | (_| | | | | | | |
| .__/|_| \___|_| |_| \__,_|_| |_| |_|_|
|_|
Predicting Register Encoded Frames Rendered As Musical Results
preframr is an experimental small language model that predicts SID music from sequences of SID chip register calls intercepted in an emulator. This example output was predicted from an overfitted model trained with a small dataset including Goto80's CBM_85.
preframr's predicted register calls are rendered as audio via an emulated SID chip or a real SID chip via the ASID protocol (for example to an Elektron SIDStation or Vessel equipped C64). Because preframr works with low level SID register calls directly it can reproduce subtle SID programming techniques such as combined waveforms and complex envelopes which cannot be represented as standard MIDI files or MIDI tokenizers such as MidiTok. While register calls are less complex than audio samples they are by definition musically expressive as they realize all C64 chipmusic and avoid the need to process audio samples with frameworks such as Magenta. preframr's register call logging could be directly used with other register logging formats such as VGMusic. While preframr currently only supports single-SID music it could support other platforms that periodically generate control messages such low resolution video displays or speech chips.
preframr is intended as a research tool for SID artists, explorers and historians, at a scale large enough to be useful artistically (for example, to assist artists in the creation of their music), but small enough to be within the limits of consumer grade GPUs. This enables an individual to manage their own "foundation" model. Therefore, no pre-trained model is currently provided - you must obtain any and all necessary permission to use SID files for training.
The encoding layer lives in preframr-tokens. It is a white-box SID decompiler: given a tune's .sid (the playroutine) and a .dump.parquet (a register write log captured from an emulator), it runs the playroutine under a 6502 emulator, taps driver RAM, and recovers a per-voice bounded-accumulator (BACC) program — a sparse note-on score plus pitch-invariant instrument definitions and a small initial-state seed. The recovered program regenerates the dump byte-exact (verify_residual), and per-frame modulation (vibrato, portamento, pulse-width, envelopes) is regenerated by the driver VM rather than stored, so the token economy is well under one token per frame.
Only playroutines a backend recognises (currently the Rob Hubbard "Monty"-class drivers) can be decompiled; tunes whose driver no backend matches are skipped.
Each recovered program is serialized (program_to_ids) to a flat stream over a small fixed 33-symbol alphabet (base-16 LEB digits plus a backward-LZ REPEAT marker) — there is no BPE/Unigram step and no learned vocabulary. preframr shifts those ids up by one into model space so id 0 is reserved as PAD, giving a 34-entry vocabulary. The model is a plain autoregressive language model over this stream. At inference a generated id stream is shifted back, decoded (ids_to_program), rendered to per-frame registers (render_program), and clocked through a resid SID chip to audio.
preframr.corpus.Corpus globs *dump.parquet, pairs each dump with its sibling .sid, recovers + serializes + windows it into seq_len blocks (written next to the dump as .blocks.npy), and skips any tune that has no matching driver backend. preframr/parse.py is the CLI that builds and caches these block arrays ahead of training.
It is assumed docker and NVIDIA drivers have already been installed.
$ ./build.sh
$ ./run_tests.sh
parse.py builds .blocks.npy block arrays from (.sid, .dump) pairs, train/trainer.py trains the language model over them, and inference/predict.py generates a continuation from a prompt block and renders it (or the ground-truth tune, if the generation is not yet a decodable program) to a WAV.
Currently, preframr is a stand alone framework that only supports inference from SID files it has parsed, and predicting the results as new sequences and/or audio output. It can of course predict new sequences from sequences it has not been trained on. Work to integrate preframr with DAW tools is planned.
- Tools to import/export frames from other applications (e.g. from and to SID Wizard)
- Real-time integration/workflows with DAWs
- Multiple SID support
- Non-C64/SID platform support (e.g. ZX Spectrum)
- Model and tools for the identification of composer techniques from frames
- Generate non-audio output (e.g. graphical VIC II output)
- goto80 for his many creative achievements, encouragement and openness to exploration
- Lucas Tindall for his machine learning mentoring
- Jim Murphy for his wisdom and support