Repository navigation
Releases
Flag v0.4.0
A release about threads, arenas, speed and failing well: a program uses every core with two words, parallel and wait, memory can be released all at once, with the same lists, text and maps as always, small functions and structs cost what they cost in C, and a program that fails says the chain of calls that led there. Flag still checks what it checked, the types, the errors that can't be dropped and -memory.
New
parallel f(x)runs the call in a new thread and gives aThread[T],wait twaits for it and gives whatfreturned, anddone(t)asks without waiting. The call is chosen like any other, the arguments are copied to the new thread, and the result can be text you own, an error to handle or a temporary, thatwaittakes into the scratch of the thread that waits.parallel f(x)alone on its line lets the thread run on its own, andmainwaits for it when it ends, before its deferred statements- A thread you keep must be read, a function that can fail can't run alone, and
-memorywarns about a thread never waited, waited twice or used afterwait, about a pointer to a local variable that leaves with a thread, also one that runs alone outsidemainor one sent through a channel, about a temporary that itswith scratch():may release while a thread still uses it, and about using what you sent through a channel.sendgives what the value owns, not its views Lock, that works at zero and is taken withwith lock(x.lock):,Channel[T, N], a bounded queue of at least one value that lives where you declare it, withsend,receive,closeandfor v in c, andcores()- Variables at the root of a file,
count: i32ornames: array[&strview, 3] = ["a", "b", "c"], shared by every function and every thread. They start at zero or at a value the compiler knows, in the data of the program, so a table that never changes is not built again each time. One cannot have the name of a function of its module, like a constant Arena, a struct of core that starts empty and gives out pieces of big blocks.arena(a, v)copies a value into it and gives a pointer, likeheap(v),arena(a, "text")gives a&str, andarena(a, []),arena(a, [1, 2]),arena(a, {})andarena(a, {k: v})build a list or a map that lives in the arena, with the type of the place where they land.free(a)releases everything andclear(a)releases everything and keeps the blocks, for the memory of each frame. A small allocation takes 76 instructions in an arena, and 400 withheapandfree- A
&str,&listor&mapthat lives in an arena grows there, a text or a list in its place when it's the last piece of the block, and itsfreeleaves it empty without releasing anything, so a function that frees what it gets works with any list -memoryknows arenas: what lives in one is not a leak, using it afterfreeorclearof its arena warns like a temporary after itswith scratch():, also through a function,freeof a pointer from an arena saysfree cannot release boss, it lives in level, free(level) releases it, and an arena that is never freed is a leak-memorywarns about a copy of anArena, aLockor aChannelin use, also inside a struct, and about a function of yours that takes one by value:this copies level, an Arena that may have given memory: the copy gives out the same blocks, use a pointer to itmemsetwith the C functions of coreinline funcandinline operatorreplace every direct call with the body of the function, whatever its size, also in each instance of a generic one. A recursive function, themainthat starts the program, anexportfunction and a C function cannot beinline, and the error of a recursive one points to the call that leads back- A runtime error writes the chain of calls after its place, one line for each function of yours with the line where it's declared:
in parse_line, main.flg:9. The library is left out, a function called again and again is one line with the count, after 20 lines the rest is counted, and an error in a thread writes the chain of that thread. When two threads fail at once, one report comes out whole - A crash says what happened and writes the chain too:
flag: invalid memory access at 0x0,flag: stack overflow,flag: illegal instructionandflag: arithmetic error, in any thread, also inside a function of C likestrlen. On Linux and macOS a signal handler runs on a stack that each thread keeps for it, and on Windows a vectored exception handler. It asks for no memory and waits for no lock, so a crash insidemallocis told too. The exit code is 128 plus the signal, or the code of the exception on Windows -debugputs DWARF in the program, for gdb and lldb: the variables in their blocks, the types (structs, enums, tagged unions as variant parts, fat pointers, threads, function values), the variables of the files, the contexts as thread-local variables, the lines and the frames, and the names of your structs, enums and aliases as typedefs. A function that takes or returns a struct, a union, an array or a fat pointer is markedDW_CC_nocall, it doesn't pass them like C, so gdb doesn't call it.break panicstops at a runtime error. Nothing is inlined in it. On macOS it runsdsymutiland leavesprogram.dSYMexamples/threads.flg, the Mandelbrot set drawn by a thread for each processor, and primes counted by workers that share a channelexamples/arena.flg, the enemies of a level and the memory of each frame in two arenas
Changed
- A
contextbelongs to each thread: a new thread starts with its contexts at zero and its own scratch memory, and the random numbers ofmathstart again in each one parallelandwaitare keywords-memorynames the fields you wrote,free cannot release level.boss, where it named only the variable,levelprintformats all its values into one line and writes it at once, so the lines of two threads never mix. Aneprintinside the+=of a value now comes out before the line that prints that value, andwrite(f, value)formats the value before the file takes it- Files can be used from any thread: each one has a lock for writing and another for reading, so a thread waiting to read a connection doesn't stop the ones that write. A read,
flush(),exitand a crash leave a file that another thread is writing to that thread, so a thread stuck sending stops nobody else - On Windows a context takes room in the program file, thread data of Windows keeps its zeros there
- A Linux program links with
-lpthread &str,&listand&maptake 32 bytes: a fourth word,arena, says where their memory lives, zero for the heap. The views,&strviewand&slice, stay at 16- The scratch memory of the temporaries is an
Arena - Every function keeps a frame pointer, on x86_64 too, so the chain can be read: a program is about 1 % bigger
- A function whose frame can go past a page of stack touches each page as it lowers the stack, on Linux and macOS as on Windows, so a stack overflow always stops at the guard page and is reported. A frame bigger than the 4 KiB guard of a thread jumped over it, into the stack of another thread
- QBE has twenty more changes, thirty-one in all: thread data on Windows, its address read once per function on Windows and macOS, and the thread data that starts at zero kept out of the file on macOS, small slots cut into pieces that live in registers, a fix for riscv64, six for the chain of calls and the debugger, six that keep a long function linear, and five fixes of QBE itself
- Building the compiler takes 32 % more instructions: most of it is the work QBE and the assembler do on the inlined code, and the rest is the code of threads, arenas and the debugger. The compiler that comes out is faster, it checks the sources of 0.3 with 7 % fewer instructions
Faster
- A direct call to a small function is replaced by its body when the program is written, after
-memory, so the warnings still name your functions. The parameters are still copies, a runtime error keeps its place, the stack doesn't grow, and a function whose calls were all inlined is not written to the program. A function with more than 4 KiB of variables is not inlined on its own, they would take room in the frame of every caller, and into a recursive function, whose every level keeps them, one with more than 32 bytes - Small structs live in registers: QBE cuts a slot of 32 bytes or less whose address stays in the function into its pieces.
acc = acc + scale(V{1.0, 2.0}, 0.5)in a loop of 100 million rounds runs in 0.11 s, it took 0.69 s, as fast as C with-O2 - A struct or array literal that gives its values in order puts its small values, numbers,
booland enums, together in words of 8 bytes and writes each word at once, so a copy reads it from one write and doesn't wait: a loop that appendsV{x, y}of twof32to a list takes half the time, with twoi322.5 times less, andjsonparses 4 % faster - A constant whose value the compiler knows, an array or a struct of literals, of enum variants or of other such constants, is kept once in the read-only data of the program, and every use reads it: a lookup in
TABLE = [3, 1, 4, 1, 5, 9, 2, 6]in a loop is 2.8 times faster - Text literals with the same text share their data, and
printno longer buildsstdouton the stack, so long functions compile much faster: 20000elif, or a switch of 20000 cases, take 5 times fewer instructions - A long function compiles in a time and a memory that grow with its length. QBE keeps in a set of temporaries only its words that are not zero, where the sets of each block had a bit for every temporary of the function, and three of its walks no longer climb the whole dominator tree, as deep as a chain of
elif. 20000elifwith aprinteach compile in 1.6 s and 0.4 GB, they took 8.1 s and 1.1 GB, and on a value known only when the program runs, 20000 compile in 2.8 s an...
Flag v0.3.0
A release about errors that say what happened and unions you handle with switch, with a standard library that is much faster and has one way to do each thing.
New
switchover a tagged union names the types of its members:case i32:,case A|B:,case f64, null:. In each case the variable has that type, like afteris, and the default gets the types no case named. A switch over aboolwithcase true:andcase false:is complete- Open slices,
x[a:],x[:b]andx[:], on text, lists, slices, arrays and your types with alenfield - Structs and unions without their own
==compare field by field, and get ahashthe same way, so a struct with text fields works as the key of a map m[k] += v, and the other assignments with an operator, on maps and on any type whosex[i]gives a copy. A key that is not there counts as zero, and on a map the key is looked up once- Errors with data,
type error BadLine:with fields, are documented and work everywhere:try,iferror,switch,print Fileis in core:stdin,stdoutandstderr, andopen,create,read,read_line,write,flushandclosefor any file.create(path, append = true)writes at the end without emptying it, andflush()sends what waits in every file- A
Socketofnetreads and writes like a file, withread,read_line,write,flushandclose, andtimeout(s, seconds)makes a slow read or write fail with the codeTIMED_OUTon every system. A server is aServer, its own type, so a file, a server and a connection never go one where the other goes - What the system refuses is an
FsErrorwith the code, what failed and the path, and printing it says all of it:cannot open notes.txt: No such file or directory.netfails with aNetErrorof the same fields:cannot connect to example.com:80: Connection refused.NotJsonsays where and why:not JSON at line 3, column 14: expected ':' after the key. Witherrno()your own C calls fail the same way - A name you declare that core also declares, like
File,Integerorstdout, is yours in your files, and the library keeps its own, shown ascore.Filein messages - A field or an element of a value in a map changes in place, the key found once and added with a zero value when it is missing:
players[name].score += 10,pairs[a][b] += 1. Passed where a pointer goes,m[k]is copied and put back after the call:append(groups[k], v) words(text)splits on spaces, tabs and line ends:for w in words(text): counts[w] += 1append(list, items),clearandreservefor lists,clearfor rings,find_last,pad,view(bytes),free(p)for whatheapgives, theIntegeralias andjson.key(d, value)- A value goes into a union through the member it converts to: a
&strinto&strview|i32, a&list[T]into&slice[T]?, and{}or{1, 2}into a union with one struct, alsom[k] = {} a += bon a&strviewjoins likea = a + b, and aforwalks through a pointer to an array, a list, a slice or textexamples/grep.flg, files line by line with the errors of each step
Faster
- The output is buffered: printing a million lines is 7 to 10 times faster, and
printallocates nothing. It goes out line by line on a terminal for c in textis written by the compiler and is 57 times faster,find39 times, floats print 6 times faster,read_line4 times,split3 times, counting words in a map 2 times, andjson.parse2 times. A line of 80 MB reads in 0.13 s, it took 3.2 s- The compiler checks itself with half the instructions and builds itself with 38 % fewer: the overload a call chooses is remembered, and no code is generated to be thrown away
- Structs, unions and arrays bigger than 128 bytes are copied with
memcpyand passed by address. A struct of hundreds of kilobytes passed by value compiled for minutes, now it's instant - The
==andhashthat the compiler makes are one function per type, so big and nested structs compile at once - A
contexttakes no room in the program file, the tags of its unions are written when the program starts - QBE is linear on big functions: a loop over an enum of 1500 variants compiles in 0.2 s, it took 37 s. Long chains are faster too: 20000
elifcompile in 8 s, they took 38 s - A run of
andor oforis tested in one block after another, so what the left side narrowed is known once: 10000 terms compile in 3.5 s, they took 42 s and 1.8 GB - The compiler uses less stack: blocks, expressions and types nest up to 500 levels (blocks were about 150), and chains of
elif,ifnull,iferroror of one operator are not nesting - A connection sends at once (
TCP_NODELAY)
Changed
f(x[i])with a pointer parameter, whenx[i]gives a copy, puts the copy back withput(x, i, v)after the call, and without thatputit doesn't compile: it changed a copy that nobody saw- An enum without a value starts as the first variant you wrote, also when another enum declared some of its variants before, and two enums with all the same variants are an error:
type Paint = Colorgives one a second name - A case of a switch over a union is a type, never a value or a variable, and a default that can never run is an error
- A deferred call that can fail doesn't compile,
defer _ = f()drops the error on purpose, and awithwhoseclosecan fail is an error - A type with its own
==and nohashcan't be the key of a map, declare itshashtoo - The counter of a range loop and the index of
for i, x incan't be assigned arg_count,arg,write(fd, text)andos.read_line()are gone:args()ofos,write(stdout, x)andread_line(stdin)- Files moved from
fsto core, andread_text_fileisread_file.close(f)gives nothing, sowith file:works, andflush(f)gives the first write that failed since the lastflush. A write that is interrupted or refused for now is tried again stdoutandstderrkeep the order of your calls, and every file sends what waits before the program waits to read and before another program runs. A program whose standard output could not be written ends withflag: cannot write the standard output: ...and the exit code 1sendandreceiveofnetarewriteandread,NetErroris a struct with the code, andCannotRunis gone:os.runfails withFsErroros.runandos.outputtake a&sliceand givei32|FsError, andoutput(command, into)appends to your text and gives the exit code. On Windowsrunof a list refuses.batand.cmd,cmd.exewould read their arguments again- Files, connections and pipes of the library are not passed to the programs you run
getis gone from maps and fixed maps,m[k]is the way.json.textgivesnullwhen the value is not a string,json.quoted(text)is a value for+andprint, andjson.freetakes the document by pointerwrite_file,append_fileandchange_dirgiveFsError?,fs.joinisjoin_path,swaptakes any integer,bitsany integer index andhexany integer, printing the bits of its sizeparse_floattakes only decimal numbers,infandnan, andfixedprints0, not-0- Text is read with one UTF-8 rule: an overlong form, a surrogate or a code above U+10FFFF comes byte by byte in
for c in text,utf8.countcounts those pieces andutf8.validagrees printcomputes all its values before it writes- An error inside a generic of the library points at your call, with a note for the line in the library
- No type can be made from an empty literal (
wrap([])intoarray[$T, 2]),heap([])still works atfor a list, a slice, an array or text can't take an index that can be an integer- A module with a
privfunction keeps that name for itself, so an exact public overload next to aprivgeneric wins - Passing an empty function value stops the program at that call
forover text belongs to the language, like over lists, soiterfor text can't be declared-memorytakes a free through an element as the free of all the elements, so freeing them in a loop or withpopdoesn't warn, and freeing only some of them is not reported- Type names in messages read like code,
func(i32, i32) -> bool, and many messages say more about why
Fixed
m["a"] = showwithshowoverloaded,return {1, 2}into aPoint?, and(usize(0) as func())()skipping the check of an empty function value-memoryfalse positives: unions whose members keep a pointer in the same place, reading a whole file and printing its error, errors with data aftertry,ison a union like&str|FsError, and print, read a file, print- On Windows a connection and a file could get the same number and share what waits to be sent
- Using
netignoredSIGPIPEfor the whole program and the programs it ran is_dirof a folder you cannot read,file_sizeof a FIFO (it waited forever),removeof a link to a folder,walklisting locked folders as files,executable()afterchange_dir, andlist_dirandwalkon macos-x86_64renamereplaces an existing file on Windows too, andoutput(command)keeps every byte on Windowsos.outputcouldn't tell a program that doesn't start from one that prints nothing- An
eprintinside youroperator +=stopped the program inside core, andout_of_memorycould call itself build.shrebuilds QBE when one of its headers changes
What may still surprise you is in Rough edges.
Flag v0.2.0
This release makes the language more stable and adds two new features: function values and pass.
New
- Function values: the name of a function is a value, with the type
func(i32) -> bool. They go in variables, fields, arrays and arguments, and they are called like any function. There are no closures pass, a statement that does nothing, for a block that has to be empty- A generic alias works in a parameter (
func get(x: Opt[$T])), and$Talso comes from inside a function type sort(items, before)sorts with your own order andfind(items, fits)finds the first element that a function accepts{k: v}fills any type that hasput, not only maps- An array can be sliced directly,
values[0:2]
Faster
x[i]on lists, slices and text, andforover arrays, lists and slices, are written by the compiler instead of calling the library. A loop that walks a list by index or withforruns about 7 times faster, and the compiler checks its own code in two thirds of the time- The IL that goes to QBE is smaller: no empty blocks, no jumps to the next line
Changed
- The keywords of the language cannot be used as names anymore
- A union always holds a real tag. Memory that you zero yourself is
nullfor a union that hasnull, and it's not a value for a union that doesn't (a variable, a field or a literal still starts as its first member) - In an assignment the right side runs first, so
l[0] = grow(l)ands += "x" + sdo what they say - A statement that only makes a value (
x == 2) and a variable that keeps an error nobody reads are errors privin a file without a module hides the name from the other files, and the helpers of the library areprivnowasdoesn't turn a value into a union or a union into another of the same size- An operator between two numbers of one type cannot be declared,
==and<must returnbool, andoperator +=is only the one that prints atwith an integer index cannot be declared for lists, slices, arrays and text, noriterfor lists, slices and arrays, andat(list, i)is gone: a pointer to an element isptr(list[i])- Code can nest about 150 blocks deep, and a chain of
elifor of one operator can have about 300 parts. More than that is an error now, where 0.1.0 crashed a bit further on - A type with parameters cannot have a block of fields inside, and
a.Typeneedsuse a - Several things that compiled by accident are errors: a function declared twice, a constant and a function with one name,
x := [],0x_ff,1e5e3, a call that returns nothing used as a value (also on the right ofiferror:x := load() iferror stop()), assigning to a slice or to an element of a temporary,defer x := 1, variants of an enum at different indentations - The counter of a
forover a range with literal ends is not passed as a pointer on its own, writeptr(i). A function that changed it made the loop index an array out of bounds without a check - A
privfunction that fits a call wins over the ones that are notpriv, so a function of yours with the name of a helper of the library no longer replaces it - A
&strgiven to C as acstris never grown: when it has no room for the zero it goes as a temporary copy. Growing it left the other copies of that text pointing to freed memory - The slice of an
array[u8, N]is a&slice[u8], it was text - A module of the library is the file of its name,
use jsonloadslib/json.flg - The signature of a C function is checked where it is declared, and a fat pointer cannot be a parameter of one
- When two overloads tie, the one that takes its arguments through a shape like
&slice[$T]wins over a bare$T json.parsetakes exactly JSON,fs.walkdoesn't follow links to folders, and several messages of runtime errors say more (slice 4:2 is out of bounds, length 6)
Fixed
- More than a hundred bugs: the compiler crashing or hanging on code it didn't expect, programs reading the wrong member of a union,
deferand loops losing track of what a variable holds, text appended to itself, constants that name a field, wrong results infs,json,osand the printing of floats, and many messages that pointed at the wrong place or didn't say what was wrong - On macOS, a function with more than 32 KiB of locals could store a 64-bit number as 32 bits. It was a bug of QBE for Apple ARM64, and it made
-memorynever end on a struct bigger than 4096 bytes - The memory checker: it reports the memory of a
&ring, the free of a temporary and the real place of a free, and it no longer invents warnings in functions with many allocations
What may still surprise you is in Rough edges.
Flag v0.1.0
The first release of Flag
flagcfor Linux x86_64, macOS arm64 and Windows x86_64, a single file with the standard library inside- The standard library: core,
fs,os,json,net,math,utf8,bits,ringandraylib(raylib only for Linux x86_64) -memory, the memory safety checker at compile time- The docs: Overview, Reference and Library