Repository navigation
Flag v0.4.0
A release about threads, arenas, speed and failing well: a program uses every core with two words, parallel and wait, memory can be released all at once, with the same lists, text and maps as always, small functions and structs cost what they cost in C, and a program that fails says the chain of calls that led there. Flag still checks what it checked, the types, the errors that can't be dropped and -memory.
New
parallel f(x)runs the call in a new thread and gives aThread[T],wait twaits for it and gives whatfreturned, anddone(t)asks without waiting. The call is chosen like any other, the arguments are copied to the new thread, and the result can be text you own, an error to handle or a temporary, thatwaittakes into the scratch of the thread that waits.parallel f(x)alone on its line lets the thread run on its own, andmainwaits for it when it ends, before its deferred statements- A thread you keep must be read, a function that can fail can't run alone, and
-memorywarns about a thread never waited, waited twice or used afterwait, about a pointer to a local variable that leaves with a thread, also one that runs alone outsidemainor one sent through a channel, about a temporary that itswith scratch():may release while a thread still uses it, and about using what you sent through a channel.sendgives what the value owns, not its views Lock, that works at zero and is taken withwith lock(x.lock):,Channel[T, N], a bounded queue of at least one value that lives where you declare it, withsend,receive,closeandfor v in c, andcores()- Variables at the root of a file,
count: i32ornames: array[&strview, 3] = ["a", "b", "c"], shared by every function and every thread. They start at zero or at a value the compiler knows, in the data of the program, so a table that never changes is not built again each time. One cannot have the name of a function of its module, like a constant Arena, a struct of core that starts empty and gives out pieces of big blocks.arena(a, v)copies a value into it and gives a pointer, likeheap(v),arena(a, "text")gives a&str, andarena(a, []),arena(a, [1, 2]),arena(a, {})andarena(a, {k: v})build a list or a map that lives in the arena, with the type of the place where they land.free(a)releases everything andclear(a)releases everything and keeps the blocks, for the memory of each frame. A small allocation takes 76 instructions in an arena, and 400 withheapandfree- A
&str,&listor&mapthat lives in an arena grows there, a text or a list in its place when it's the last piece of the block, and itsfreeleaves it empty without releasing anything, so a function that frees what it gets works with any list -memoryknows arenas: what lives in one is not a leak, using it afterfreeorclearof its arena warns like a temporary after itswith scratch():, also through a function,freeof a pointer from an arena saysfree cannot release boss, it lives in level, free(level) releases it, and an arena that is never freed is a leak-memorywarns about a copy of anArena, aLockor aChannelin use, also inside a struct, and about a function of yours that takes one by value:this copies level, an Arena that may have given memory: the copy gives out the same blocks, use a pointer to itmemsetwith the C functions of coreinline funcandinline operatorreplace every direct call with the body of the function, whatever its size, also in each instance of a generic one. A recursive function, themainthat starts the program, anexportfunction and a C function cannot beinline, and the error of a recursive one points to the call that leads back- A runtime error writes the chain of calls after its place, one line for each function of yours with the line where it's declared:
in parse_line, main.flg:9. The library is left out, a function called again and again is one line with the count, after 20 lines the rest is counted, and an error in a thread writes the chain of that thread. When two threads fail at once, one report comes out whole - A crash says what happened and writes the chain too:
flag: invalid memory access at 0x0,flag: stack overflow,flag: illegal instructionandflag: arithmetic error, in any thread, also inside a function of C likestrlen. On Linux and macOS a signal handler runs on a stack that each thread keeps for it, and on Windows a vectored exception handler. It asks for no memory and waits for no lock, so a crash insidemallocis told too. The exit code is 128 plus the signal, or the code of the exception on Windows -debugputs DWARF in the program, for gdb and lldb: the variables in their blocks, the types (structs, enums, tagged unions as variant parts, fat pointers, threads, function values), the variables of the files, the contexts as thread-local variables, the lines and the frames, and the names of your structs, enums and aliases as typedefs. A function that takes or returns a struct, a union, an array or a fat pointer is markedDW_CC_nocall, it doesn't pass them like C, so gdb doesn't call it.break panicstops at a runtime error. Nothing is inlined in it. On macOS it runsdsymutiland leavesprogram.dSYMexamples/threads.flg, the Mandelbrot set drawn by a thread for each processor, and primes counted by workers that share a channelexamples/arena.flg, the enemies of a level and the memory of each frame in two arenas
Changed
- A
contextbelongs to each thread: a new thread starts with its contexts at zero and its own scratch memory, and the random numbers ofmathstart again in each one parallelandwaitare keywords-memorynames the fields you wrote,free cannot release level.boss, where it named only the variable,levelprintformats all its values into one line and writes it at once, so the lines of two threads never mix. Aneprintinside the+=of a value now comes out before the line that prints that value, andwrite(f, value)formats the value before the file takes it- Files can be used from any thread: each one has a lock for writing and another for reading, so a thread waiting to read a connection doesn't stop the ones that write. A read,
flush(),exitand a crash leave a file that another thread is writing to that thread, so a thread stuck sending stops nobody else - On Windows a context takes room in the program file, thread data of Windows keeps its zeros there
- A Linux program links with
-lpthread &str,&listand&maptake 32 bytes: a fourth word,arena, says where their memory lives, zero for the heap. The views,&strviewand&slice, stay at 16- The scratch memory of the temporaries is an
Arena - Every function keeps a frame pointer, on x86_64 too, so the chain can be read: a program is about 1 % bigger
- A function whose frame can go past a page of stack touches each page as it lowers the stack, on Linux and macOS as on Windows, so a stack overflow always stops at the guard page and is reported. A frame bigger than the 4 KiB guard of a thread jumped over it, into the stack of another thread
- QBE has twenty more changes, thirty-one in all: thread data on Windows, its address read once per function on Windows and macOS, and the thread data that starts at zero kept out of the file on macOS, small slots cut into pieces that live in registers, a fix for riscv64, six for the chain of calls and the debugger, six that keep a long function linear, and five fixes of QBE itself
- Building the compiler takes 32 % more instructions: most of it is the work QBE and the assembler do on the inlined code, and the rest is the code of threads, arenas and the debugger. The compiler that comes out is faster, it checks the sources of 0.3 with 7 % fewer instructions
Faster
- A direct call to a small function is replaced by its body when the program is written, after
-memory, so the warnings still name your functions. The parameters are still copies, a runtime error keeps its place, the stack doesn't grow, and a function whose calls were all inlined is not written to the program. A function with more than 4 KiB of variables is not inlined on its own, they would take room in the frame of every caller, and into a recursive function, whose every level keeps them, one with more than 32 bytes - Small structs live in registers: QBE cuts a slot of 32 bytes or less whose address stays in the function into its pieces.
acc = acc + scale(V{1.0, 2.0}, 0.5)in a loop of 100 million rounds runs in 0.11 s, it took 0.69 s, as fast as C with-O2 - A struct or array literal that gives its values in order puts its small values, numbers,
booland enums, together in words of 8 bytes and writes each word at once, so a copy reads it from one write and doesn't wait: a loop that appendsV{x, y}of twof32to a list takes half the time, with twoi322.5 times less, andjsonparses 4 % faster - A constant whose value the compiler knows, an array or a struct of literals, of enum variants or of other such constants, is kept once in the read-only data of the program, and every use reads it: a lookup in
TABLE = [3, 1, 4, 1, 5, 9, 2, 6]in a loop is 2.8 times faster - Text literals with the same text share their data, and
printno longer buildsstdouton the stack, so long functions compile much faster: 20000elif, or a switch of 20000 cases, take 5 times fewer instructions - A long function compiles in a time and a memory that grow with its length. QBE keeps in a set of temporaries only its words that are not zero, where the sets of each block had a bit for every temporary of the function, and three of its walks no longer climb the whole dominator tree, as deep as a chain of
elif. 20000elifwith aprinteach compile in 1.6 s and 0.4 GB, they took 8.1 s and 1.1 GB, and on a value known only when the program runs, 20000 compile in 2.8 s and 0.5 GB where 5000 took 4.9 s and 2.2 GB. A temporary defined in several places finds its phis through its definitions, not every instruction of the function, so 6000 copies of a struct in a row compile in 0.37 s
Fixed
- The
==andhashthat the compiler writes for a type, and the data of a text literal, could take the name of a function of the program, likesv, and the program didn't assemble - A constant that names another constant read the local variable with that name where it was used
-memorywarned that a call may free memory it also reads through another argument when both were memory of contexts, and no argument was involved-memorytook a field that comes after an array in a struct for an element of the array, and warned about memory that was fine-memorywarned that memory may be freed already when an element freed in a loop was also kept in a heap block, like the text that each thread of a list gives, freed as you wait them one by one- QBE for riscv64 stopped with
unimplementedwhen a constant or a value on the stack went into a temporary that lives on the stack - On Windows a function of Flag that C calls, like a callback, could change the floats that the C around it kept in
xmm6toxmm15, which Windows has every function keep