W tape data structure

For those adventurers wanting to hack at the bits on the uSD card here is a general outline:

Card Structure

nb: All hex numbers (eg 0x100) represent page-indexes, not absolute memory. Every 0x1 is a card location that contains 512bytes of data. 0x1 = 512bytes, 0x10 = 8kB etc

The card is broken into 7 useable segments beginning a static offset into the card. The reason for the offset (SDCARD_TRAINER_LOCN below) is to avoid clobbering the FAT32 filesystem. This means the card is mountable in a computer and indeed the magic number is the location of the start of the tape.bin data you'll see as the only contents when you mount the card.

Starting from this location we create 7 tapes. The first of which is reserved for Izzy (called TRAINER below) and has a custom size (64MB).

The remaining 6 pages are equally sized segments of the uSD. See the SDFS_tape_address() function below for how we internally query the data offset to use at a given time. Note the bit shifting causing addresses to be aligned on 0x10 boundaries - This was found to dramatically improve throughput in practical use.

#define SDCARD_PAGE_TOTAL    0x7FFFFF // 8,388,608pages (4GB card)
#define SDCARD_TAPE_COUNT    7
#define SDCARD_TRAINER_LOCN  (0x446C)
#define SDCARD_TRAINER_SIZE  0x10000
#define SDCARD_TRAINER_END   (SDCARD_TRAINER_LOCN + SDCARD_TRAINER_SIZE)
#define SDCARD_PAGE_PER_TAPE ((int32_t)((SDCARD_PAGE_TOTAL - SDCARD_TRAINER_END) / (SDCARD_TAPE_COUNT-1)))

const uint32_t SDFS_tape_address( int tape_count )
{
    const uint32_t sd_tape_address[SDCARD_TAPE_COUNT] =
        { ((SDCARD_TRAINER_LOCN>>4)+1)<<4 // trainer
        , ((SDCARD_TRAINER_END>>4)+1)<<4
        , (((SDCARD_TRAINER_END +   SDCARD_PAGE_PER_TAPE)>>4)+1)<<4
        , (((SDCARD_TRAINER_END + 2*SDCARD_PAGE_PER_TAPE)>>4)+1)<<4
        , (((SDCARD_TRAINER_END + 3*SDCARD_PAGE_PER_TAPE)>>4)+1)<<4
        , (((SDCARD_TRAINER_END + 4*SDCARD_PAGE_PER_TAPE)>>4)+1)<<4
        , (((SDCARD_TRAINER_END + 5*SDCARD_PAGE_PER_TAPE)>>4)+1)<<4
        };
    return sd_tape_address[tape_count];
}

Tape Structure

Each of the 7 tapes (including the TRAINER) begins with a header as described by the sdfs_layout enum below. Each section will be outlined separately below.

typedef enum sdfs_layout
	{ sdfs_METADATA = 0x0   // select which cue page, current playhead position
	, sdfs_CUES_A   = 0x20  // 32pages = ~1365 cues @12bytes each
	, sdfs_CUES_B   = 0x40
	, sdfs_RESERVED = 0x60  // reserved for future use
	, sdfs_AUDIO    = 0x100
	, sdfs_ENDER    = (SDCARD_PAGE_PER_TAPE-0x10) // paranoia
} sdfs_layout_t;

sdfs_METADATA

offset = 0x0
size   = 0x20 = 16kB

The metadata page is quite sparse. It contains the current state of W/ so startup & jumping between tapes can return you to the most recent context. Below is a symbolic representation of the data -- note this layout is bit-perfect & you have to imagine word-alignment isn't a thing. Stupidly we explicitly copy data to/from the card piecemeal rather than using the below struct and memcpy(). oh well.

typedef struct{ // just a symbolic representation
    uint8_t tape_has_data;
    uint8_t cue_switch;
    float   through;
    float   reserved;
    int16_t cue_index_of_playhead;
    int16_t cue_index_of_loop_start;
    int16_t cue_index_of_loop_end;
    int8_t  is_loop_active;
    int32_t touched_by_angel;
} tape_metadata_t;

tape_has_data will equal 0x66 if the tape has been initialized. Otherwise it will likely be 0x00 but could be 0xFF (or any random byte). This just means we can detect if the card is brand new & needs to have defaults / blank tape inserted. 0x66 = 102. I don't know.

cue_switch is currently unused, but the intention is that it chooses between sdfs_CUES_A and sdfs_CUES_B (see above & below), meaning that a backup of cue data is provided. This bit is intended to be flipped on each write of the cues. There is a risk of data corruption if powering down while a page is actively being written, so this is some overdesign that never worked correctly & hangs around to be fixed one day.

through just saves the monitor mode. it's a float, but is always 0.0 or 1.0

reserved is just that.

The next 4 values capture the runtime state of W/. The first is the nearest cuepoint to the current playhead so we can jump to it on boot/switching tapes. The following two are loop start & end points. These aren't really used and will be re-computed based on the cue_index_of_playhead. is_loop_active is just that: whether looping is currently turned on.

Finally touched_by_angel should really be called last_sd_page_with_valid_data but that's longer and less fun. It is an SDcard page index which is used to keep track of how far into the tape the playhead has moved. This is essential because we don't 'zero' the tape in production (it would take ~1hr per card). Instead pages are blanked as the tape passes them, regardless of read/write mode. touched_by_angel means we know whether this action needs to be done, or if the data is expected to be valid.

sdfs_CUES_A

offset = 0x20
size   = 0x20 = 16kB

This section stores the list of cues the user has added to the tape. They're basically a linked-list but using array indices rather than pointers. This solves the problem of where in memory the cues go (it can change from firmware to firmware), plus it only needs a 16bit int rather than 32bit pointer.

There are 1365 cues stored across these 20 pages. The whole array is read/written directly in-place. Each node looks like the below. ssample is the number of samples into the SD page with sub-sample accuracy. prev and next are indices for the linked-list - the list is guaranteed to be maintained in increasing time-index. The list becomes fragmented over time as the user adds & deletes cues and should be defragmented (unimplemented as yet).

typedef int16_t C_ix_t;
typedef struct cnode{
	float         ssample; // subsample position with page
	int32_t       ts;      // SD page-offset
	C_ix_t        prev;
	C_ix_t        next;
} cnode_t;

sdfs_CUES_B

offset = 0x40
size   = 0x20 = 16kB

Functionally identical to sdfs_CUES_A.

sdfs_RESERVED = 0x60

offset = 0x60
size   = 0xA0 = 80kB

This section is reserved for future use.

sdfs_AUDIO

offset = 0x100
size   = ??? ~650MB

This is where the audio, cv and trigger recordings are saved. The structure is as in the below imaginary structs (we don't use these but unpickle directly into an expanded form).

#define SD_PAGE_SAMPLES 168
#define SD_PAGE_CV      4

typedef struct{ // imaginary data types!
    int12_t cv;
    int4_t trigger;
    int1_t dirty;
} cv_block_t;

typedef struct{ // imaginary!
    int24_t audio[SD_PAGE_SAMPLES]; // int24_t doesn't exist
    cv_block_t cv[SD_PAGE_CV];
} sdfs_AUDIO_page_t;

audio data is stored as 24bit integers, little-endian, 2s compliment.

cv values are stored as 12bit integers, little-endian, 2s compliment. Note the LSB comes first, then you need to and the following byte with 0xF. Deserialize with = (int16_t)(src[0]<<4 | ((src[1] & 0x0F)<<12))>>4.

trigger is the etrig_t enum from wrEvent.h (in wrLib).

Only the second dirty flag is meaningful. 1 means the page data should be valid, 0 means the page is blank. Check it with equivalent of (page.cv[1].dirty == 1).

sdfs_ENDER

offset = (SDCARD_PAGE_PER_TAPE-0x10)
size   = 0x20 = 16kB

Just an empty footer for the tape. This is here in case the playhead somehow escapes the bounds of the tape while it is brought to a halt by the 'brake'.