Table to Binary in Pico-8

tab2bin and bin2tab are functions for converting a table of a known format/shape into binary within Pico-8.

Currently, bin2tab requires 536 tokens and tab2bin requires 913 tokens. There's a lot of code that can be removed to reduce the amount of tokens if they aren't used.

Table Format

n	Any number recognized by Pico-8
xyz	Stored variable name, can replace any n
_	Any element or number
abc	Any string, though unrelated to stored variables

Characters	Function
[_]	Indexed table
{_}	Keyed table
_,_	Separate elements in a table
abc=_	Denotes a key-value pair in a keyed table
#n	Reads the next n bits as a number into the last read value
%	Reads the next bit as a boolean into the last read value
?n	Read the next n bits to read that number of bytes as a string (into last read value)
@xyz	Stores the last read value into variable xyz
!n	Stores n as the last read value
+n	Alters the last read value by adding n
-n	Alters the last read value by subtracting n
>n	Alters the last read value by shifting right n >>
<n	Alters the last read value by shifting left n <<
$abc	Starts a separate provided sub-format or function
(_)	Loops a section equal to the last read value before it starts

In Depth

Reading Bits with #

When Reading bits, # will only read up to 16 bits and will apply them starting from the least significant non-decimal bit (0x1) and go towards the most significant. So #7 would read the bits that overlap 0x7f or 0b01111111.

Last Read Value

The last read value is what is being assigned, altered, stored, and appended during the tab2bin and bin2tab functions. by applying something like #9-256 you can have a full range of numbers from -256 to 255 with 9 bits of information. tab2bin applies this conversion for you.

Appending to table

While the last read value is made or changed, it only gets added to the table(s) generated by tab2bin/bin2tab after the following characters:

• , end of current element
• ] end of indexed table
• } end of keyed table
• ) end of loop

Note that you will not need to place a , after a loop end ()). After a store operation happens, the last read value is set to nil and store operations are skipped if the last read value is nil. If a new values is read into the last read value, the old one will be lost. In addition, the end of a table (] or }) will set that table to be the last read value, so that it can be appended to another table.

Subformats

A table of subformats can be provided that are used at $abc. abc is the key inside the table while the value can either be a string or a function.

The string version has the exact same rules as the tab2bin format.

The function version requires a separate function for tab2bin and bin2tab

-- for tab2bin
function tobin(writer, last_value, stored_values)
    if type(last_value) == "number" then -- double check value is a number
        writer(last_value, 8) -- write last_value as the next 8 bits
        return true
    end
    return false -- return if the value being compressed is valid
end

-- for bin2 tab
function totab(reader, last_value, stored_values)
    return reader(8) -- read next 8 bits and return
end

Examples

-- reads 2 entries
tab = {1,2}
form = "[#8,#8]"
tab2bin(tab, 0x8000, form)
tab2 = bin2tab(0x8000, form)

-- reads a variable amount of numbers, up to 255
tab = {1,2,4,8,16,32,64,128}
form = "[#8(#8)]"

-- reads a table of tables
tab = {{1,2,4},{8,16,32},{64,128,3}}
form = "[#8([#8,#8,#8])]"

-- the format will also work with a fixed loop size
form = "[#8([!3(#8)])]"

-- can store a full 32-bit fixed point number from pico8
form = "[#16>16@dec#16+dec]"

--[[
#16 read in 16 bits (0xffff)
>16 shift those 16 bits to the right (0x0.ffff)
@dec store the 16 bits into 
#16 read in 16 bits (0xffff)
+dec add the lower 16 bits in for the decimal (0xffff.ffff)
]]

tab = {
    x = 10,
    y = 30,
    z = 40
}
form = "{x=#8,y=#8,z=#8}"

tab = {
    {name = "player", health = 5, maxhealth = 10},
    {name = "enemy1", health = 3, maxhealth = 3},
    {name = "enemy2", health = 2, maxhealth = 4},
    -- ...
}
form = "[#8({name=?5,health=#5,maxhealth=#5})]"

tab = {1, 5, -4, 0.5, -0.5},
form = "[#8($num)]",
subform = {num = "#8>1-64"}
tab2bin(tab, 0x8000, form, subform)
))

function tobin(writer, last_value, stored_values)
    if type(last_value) == "number" then
        writer(1, 1)
        writer(last_value, 8)
    elseif type(last_value) == "boolean" then
        writer(0, 1)
        writer(tonum(last_value), 1)
    else
        return false
    end
    return true
end

function totab(reader, last_value, stored_values)
    if reader(1) == 1 then
        return reader(8)
    end
    return reader(1) == 1
end

tab = {1, 2, false, 4, true, false},
form = "[#8($bn)]",
subform = {bn = tobin} -- or {bn = totab} depending on if tab2bin or bin2tab is used

Extras/Ideas

A string format that doesn't take in a length, but instead looks for a terminating character.

A more compressed string format, that instead uses 5 or 6 bits per character, or a format with more steps to better compress the string.

A loop instruction for keyed tables that reads in a string (like ?n) for the key first, then calculates the value. Then rename to indexed loop and keyed loop.

Something that would be nice to have is a format calculator function that takes in a table and returns a string that would best compress that table. This way any the format could be stored inside the pico-memory and the table could be easily compressed and decompressed without even knowing the format. Though this would be very complex and will likely have problems. (So I will not be the one making it if ever.)