Core 0.3.2

TLDR

• Fixed a lot of DIMA-related and memory-related bugs and added new internal features to greatly reduce double-free and leak errors and general crashes, codegen errors or incorrect runtime behaviour
• Added support for shorthand field assignments like d.x += 5;
• Added support for func-modules as interfaces (wiki)

use Core.print

data Data:
	i32 x;
	i32 y;
	Data(x, y);

func Func requires(Data d):
	def move(i32x2 diff):
		d.(x, y) += diff.(x, y);

	def get_x() -> i32:
		return d.x;

	def get_y() -> i32:
		return d.y;

entity Entity:
	data: Data;
	func: Func;
	Entity(Data);

data Data2:
	i32 z;
	i32 speed;
	Data2(z, speed);

func Func2 requires(Data d, Data2 d2):
	def move2(f32x3 direction):
		f32x3 diff = direction * f32(d2.speed);
		d.(x, y) += i32x2(diff.x, diff.y);
		d2.z += i32(diff.z);

	def get_z() -> i32:
		return d2.z;

entity Entity2 extends(Entity e):
	data: Data2;
	func: Func2;
	Entity2(e, Data2);

def apply_move_operation(mut Func f, i32x2 diff):
	f.move(diff);

def main():
	e1 := Entity(Data(10, 20));
	print($"e1.(x, y) = ({e1.get_x()}, {e1.get_y()})\n");
	apply_move_operation(e1, i32x2(5, 5));
	print($"e1.(x, y) = ({e1.get_x()}, {e1.get_y()})\n");

	e2 := Entity2(Data(10, 20), Data2(30, 3));
	print($"e2.(x, y, z) = ({e2.get_x()}, {e2.get_y()}, {e2.get_z()})\n");
	apply_move_operation(e2, i32x2(5, 5));
	print($"e2.(x, y, z) = ({e2.get_x()}, {e2.get_y()}, {e2.get_z()})\n");

• Added the start_capture, end_capture and end_capture_lines functions to the system Core module to be able to capture everything printed to stdout and stderr between those two calls

use Core.print
use Core.system

def main():
	start_capture();
	for (i, _) in 0..10:
		print($"iteration {i}\n");
	str capture = end_capture();
	print($"capture = \"{capture}\"\n");

• Slicing a string from n..n now is well-defined behaviour and returns an empty string ""
• Tests now are able to capture their output and print it on failure (or in a controlled manner in general)
• Added #test_output_always and #test_output_never annotations
• Added ability to the builtin test runner to capture stdout and made it all pretty
• Added ability to enums to give each tag a value:

enum MyEnum:
    TAG0 = 1, TAG1 = 2, TAG3 = 4, TAG4 = 8;

• Added enum support to FIP
• Added ability to FIP to auto-generate binding files for externally found code (wiki)

use Fip.raylib // auto-generates a raylib bindings file

def main():
    // Use raylib things...

• Added the i8, u16 and i16 primitive integer types
• Added ability that fields of data types now can have default-values, added ability to define const data, both which work on the basis of expression substitution (wiki)

use Core.print

def add(i32 x, i32 y) -> i32:
	print($"{x} + {y} = {x + y}\n");
	return x + y;

data MyData:
	i32 x = 4;
	f32 y = 3.14;
	str s = "Hi there!";
	MyData(x, y, s);

const data Globals:
	i32 x = add(10, 20);

def main():
	md := MyData(_);
	print($"md.(x, y, s) = ({md.x}, {md.y}, \"{md.s}\")\n");

	print($"x = {Globals.x}\n");

output:

md.(x, y, s) = (4, 3.14, "Hi there!")
10 + 20 = 30
x = 30

• Added the opaque type and the null literal (wiki)

use Core.assert

extern def malloc(mut u64 n) -> opaque;
extern def free(mut opaque ptr);

def main():
	opaque ptr = malloc(100);
	assert(ptr != null);
	free(ptr);
	ptr = null;

• Fixed --parallel flag

Changelog

• Unified freeing of types and values in a flint.free function located inside the Memory subclass of the Generator (commit
• Refactored a lot about types and type IDs to get this whole system up and running (commit
• Made the flint.free function call the release functions of DIMA for data types (commit
• Added dima.retain call for every argument passed to function before calling the function with the parameters (commit
• Added a type_id field to the DIMA head and added call to the flint.free function before releasing the data in the dima.release function if it's arc has fallen to 0 (commit)
• Created the flint.clone function to provide proper deep-clone support for all complex (freeable) types (commit)
• Added optional type support to the flint.free and flint.clone functions (commit)
• Added tuple type support to the flint.clone and flint.free functions (commit)
• Added ability to detect temporary garbage strings for string binary operations to remove all memory leaks coming from string binary ops (commit)
• Generalized the clear_garbage function to call the flint.free function on the garbage types (commit)
• Updated garbage collection system for function call generation code, now temporary values of calls are properly freed after the call (commit)
• Added support for shorthand field assignments (d.x += 5;), it's weird that they were not supported until now (commit)
• Added ability of entities to be implicitely castable to func modules if the entity contains the func module it wants to be cast to (commit)
• Added parsing support for func-modules as interfaces (commit)
• Implemented codegen for func-modules as interfaces, but the LLVM assertion about duplicates triggers again... (commit)
• Added the ErrExprCallOnConstInstance error to detect when doing an instance call on an instance (or interface) which is marked as const (commit)
• Added (missing) ability to detect calls on const instances for call statements as well, it was only detected on call expressions until now (commit)
• Moved from a multi-translation-unit pipeline to a single translation-unit pipeline, e.g. a single llvm Module (commit)
• Changed calling convention of DIMA to be purely call-side instead of scattered around (commit)
• Added the pong-3.0 example (commit)
• Refactored the parser to a segmented-scope parser to provide more nuanced auto-free ability to the codegen of Flint (commit)
• Added the start_capture and end_capture functions to the system Core module to be able to capture everything printed to stdout and stderr between those two calls (commit)
• Changed behaviour when both indices of a slicing operation are the same on strings. This is now well-defined behaviour and simply returns an empty string (commit)
• Added ability for all tests to capture the stdout of their respective tests (commit)
• Added the #test_output_always annotation for tests. If tests are not annotated then they only output their captured stdout when they fail (commit)
• Made the new output captured output printing very pretty (commit)
• Added the #test_output_never annotation (commit)
• Completely restructured and refactored the test generation code to reduce code duplication by a lot through a reusable test execution function which is generated if a test binary is built (commit)
• Implemented generate_visible_width_function which generates a function which checks the visible length of a line, by counting bundled unicode or ascii escapes as one "character" since only once "character" will be visible in the terminal output. This fixes issues with printing the output of nested tests. (commit)
• Updated the test runners to the new system, it's now based on a "nested testing" approach where each test compiles and runs another test, and output is hidden if everything was ok (commit)
• Made FIP directory-independant (commit)
• Refactored enums quite a bit to be able to specify the values of each tag in the enum (commit)
• Implemented interoperability of enum types with C (commit)
• Added ability to auto-generate "bindings" files for extern C code (commit)
• Added ability for auto-bindgen to emit type aliases to multi-types instead of actual data types if the data type could be a multi-type instead (commit)
• Added the i8, u16 and i16 primitive types and refactored quite a bit of internals (commit)
• Added ability to auto-generate different types depending on whether we are generating function types or not (commit)
• Added resolvability of the void* type to generate u8[] instead (commit)
• Added ability to parse data field initializers (commit)
• Added ability to define const data as a way for compile-time expression substitution (commit)
• Moved functionality of collapse_tokens_in_line to collapse_tokens_in_slice function to make it reusable (commit)
• Refactored the data parsing pipeline to add parsing data to the two-phase parsing approach. This means that now nearly everything is parsed in two phases. Parsing of default-values for non-const data types now works too (commit)
• Added ability to default-construct data which then inserts the field initializer expressions at the correct places (commit)
• Changed default-construction behaviour that a _ now "fills in" all remaining field initializers with default-values (commit)
• Added parsing support for the opaque type and the null literal and added the OpaqueType type (commit)
• Added ability to cast opaque values to strings (commit)
• Added ability to detect / generate opaque pointers when interacting with C via the FIP (commit)
• Added ability to compare opaque values with the null literal (commit)
• Added ability to track memory leaks of opaque values and added cli options to control the behaviour accordingly (commit)
• Added missing ability to compare opaque values to the null literal using the != operator (commit)
• Made the profiler thread-safe while also keeping a proper profiling result output (commit)
• Added SingleExecutorGuard class to assert that only one thread executes a given code path at any given point (commit)
• Made the namespace_map_mutex shared to allow subsequent reads but writes are still mutexed (commit)
• Removed the aligned and immutable keywords entirely since (i think) they will never actually be used (commit)
• Removed [f] flag from two tests of data regarding default-values since they now pass after the recent feature addition (commit)
• Fixed bug where imports of nonexistent files would crash the compiler (commit)
• Fixed bug where pseudo-variables (variables to keep track of the return-type of a function within that function) led to codegen problems (commit)
• Made a bunch of static constexpr globals in the generator_dima file to not rely on magic numbers any more for the struct GEPs (commit)
• Fixed a few bugs for block_id access and store as it's an i16 but a i64 was stored on it, leading to stack corruptions (commit)
• Fixed bug where the block count in DIMA was not initialized to 1 on creation of the first block (commit)
• Fixed infinite loop bug in the dima.release function caused by a missing postcondition decrement (commit)
• Fixed the calculate_type_alignment function which just calculated an alignment of 8 every time for integer and float types (commit)
• Fixed multiple bugs in the dima.allocate_in_block function (commit)
• Fixed bug where the flint.free function was called by passing in the pointer to the DIMA slot instead passing the pointer to the actual value to it (commit)
• Fixed double-free mistake where function parameters like strings would be freed (commit)
• Fixed bug where the type_size of newly created blocks (dima.create_block) was uninitialized leading to undefined behaviour for allocations (commit)
• Fixed bug where a none literal would crash the codegen when used as an initializer (commit)
• Fixed bug in flint.free function where some branches of freeing variants would not end with a terminator instruction (commit)
• Fixed mistake that func modules are freeable leading to compile errors in the flint.free and flint.clone functions because func-modules as variables are not implemented (yet) (commit)
• Fixed mistake where things like array initializers, string interpolations etc all were cloned when being assigned or declared but these are all "producers" and cloning them makes absolutely no sense, and leads to a lot of memory leaks (commit)
• Fixed bug in flint.free and flint.clone function where the index of arrays would not be incremented on each loop iteration (commit)
• Fixed memory leaks of slicing operations because they were not recognized as "producers" in initializers, assignments and declarations (commit)
• Fixed bug where the clear_garbage function would skip too much garbage leading to leaks (commit)
• Fixed mistake where freeable function parameters in a return statement were returned as-is without being cloned, leading to double-free errors (for example when the function parameter is of type str and is returned directly) (commit)
• Fixed bug where passing data to extern functions would cause memory leaks because the dima.retain function was called before passing the value to the function (commit)
• Fixed mistake where freeing the old value of a variable when assigning a new value to it would lead to a segmentation fault because the pointer was not loaded from the variable (commit)
• Fixed mistakes where array accesses were misinterpreted as array slicing operations leading to double-free bugs (commit)
• Fixed bug in FuncType which resulted in every equals call to always return false (commit)
• Fixed mistake where func and entity variables were passed by-copy to functions instead of by-value which is not correct since you should be able to overwrite passed-in entities and func variables (commit)
• Fixed missing resolving of unknown parent types of entities in cross-file scenarios - closes #6 (commit)
• Updated the time Core module to utilize DIMA. It allocated timestamps on the heap and stuff like that leading to crashes when using the time Core module since it was expected that all functions in it would utilize DIMA under the hood, like the rest of Flint (commit)
• Fixed A LOT of bugs and incorrect looping, incorrect code etc in the dima generator module (commit)
• Fixed bug where continue and break statements in loops would lead to memory leaks (commit)
• Fixed bug where the dima.release function would not write the updated used count back to the block leading to more and more block creations and thus "leaks" since blocks were never destroyed (commit)
• Fixed bug where the main function would not initialize the DIMA heads leading to segmentation faults whenever any DIMA-managed data wanted to be allocated (commit)
• Fixed bug in the end_capture_lines function of the system Core module where if the first line was an empty line then the first line would become the entire output of the capture because of how string slicing works (see issue #7) (commit)
• Fixed bug where the space count for each line would be below the output width of the container leading to not-so-pretty output of the builtin test runner (commit)
• Fixed branching mistake where the performance profiling results were only printed if the test failed (commit)
• Fixed bug where every single file was thought to be "inside" the .fip directory (commit)
• Fixed bug where printing an error which goes to the end of the line would cause an out-of-bounds crash (commit)
• Fixed bug where data fields and function parameter names with the name of keywords would break the generated bindings file (commit)
• Fixed bug where a missing body would lead to a hard crash (commit)
• Fixed mistake where the type name of a struct was not generated if one was present leading to the generation of tuples whenever data type names should have been generated (commit)
• Fixed bug where parameter and return types of extern functions were not alias-resolved leading to incorrect "function not found" errors (commit)
• Fixed bug where the compiler would crash if the very first definition in a file is a data definition (commit)
• Fixed mistake where assigning a value to an opaque variable would cause the "free before assignment" chain of operations, which does not apply for opaque typed variables (commit)
• Fixed mistake where the opaque value was not loaded before being passed to the flint.free function when it goes out of scope (commit)
• Fixed mistake where using an alias as the initializer type would throw an error even though it should be possible and valid (commit)
• Fixed bug in type conversion functions from converting types to extern and from extern types regarding multi-types since all u8xN variants were missing from the code (commit)
• Fixed bug in FIP where the master state was not being properly reset when killing all slaves in FIP::shutdown (commit)
• Fixed mistake where FIP::shutdown would not clear it's global values (commit)
• Fixed LSP crash caused by transitive imports (commit)
• Fixed bug where extern function resolving was done too eagerly, resulting in extern functions not being found as their types were alias types or unknown types (these types are resolved at a later stage) (commit)
• Fixed mistake where lifetimes of the body vector was shorter than needed for the process_function (commit)
• Fixed bug where the --no-generation flag would lead to FIP not being shut down at all, leaving the fip-c process stranded and running on the system (commit)
• Fixed massive performance slowdown from the CumulativeProfiler which even profiled all hot-path functions when it's results should not be printed to the console (commit)
• Fixed bug where cloned call expressions would not get their scope id's updated (commit)
• Fixed bug where casting an enum to a string would lead to a code generation error because of an incorrect key to access the enum node names map (commit)
• Fixed bug where casting an enum to a string where the values have been set manually would lead to a segmentation fault because the values themselves were used as indexing offsets (commit)

Full Changelog: v0.3.1-core...v0.3.2-core