Note

Macroni is in its design phase. There is no implementation yet.

Macroni

Macroni is an operating system (that can also run as a process) with an integrated development environment for a Lisp dialect.

Macroni combines many solutions to problems most computing systems have.

Social Features
Problem	Solution
Untrustworthy computing is ubiquitous.	Macroni is open source.
Plutocrats deprive others of freedom.	Macroni is licensed under the AGPLv3.
Human replacement technology is destroying livelihoods, privacy, autonomy, security, freedom, creativity, and progress.	Macroni promotes intelligence augmentation technology by exposing all software as a toolbox instead of a black box and leaving consequential decisions to humans.
Most governments and computing cartels seem to believe they have the right to observe and interfere with all data storage and communication. Most computing cartels seem to believe they alone own the hardware, software, electricity, and Internet service their customers pay for and the content their customers create. Most computing cartels seem to believe they have the right to extort or forbid others from selling, gifting, or repairing hardware or software.	Macroni’s design respects human rights and sane property rights by helping the person who paid for or was gifted the hardware stay in control of their computer.
Unaccountability causes software to get worse over time.	Macroni exposes the sources of unreliability and inefficiency to cause software to get better over time.

Operating System Features
Problem	Solution
Most attempts to improve upon currently popular operating systems do not attempt to remain compatible with them.	Macroni remains compatible with currently popular operating systems.
Most operating systems use the obsolete and insecure concept of system-wide accounts.	Macroni supports multiple usage models: personal computing – where there are no accounts network services – where each network server may define its own concept of accounts and provide accounts to a different set of users The usage models can coexist. For example, the owner-user can host and play an online game.
Most operating systems require a lot of manual management.	Hardware interfaces and network protocols allowing, Macroni is self-managing: self-configuring self-protecting self-healing self-optimizing This only minimizes manual management. It does not automate consequential decisions because the owner-user could disagree with its decisions.
Most file systems are inefficient to search and do not allow tagging contents to improve search. Most file systems cannot ensure a mutation, which may affect multiple files or directories, occurs completely or not at all (“atomicity”). Most file systems cannot ensure their contents are not corrupt (“consistency”). Most file systems cannot ensure their contents are shared among multiple threads safely (“isolation”). Most file systems cannot ensure a mutation, which may affect multiple files or directories, is stored completely before they report success (“durability”). Most file systems cannot adapt to differences in storage devices (e.g. mechanical drives’ slow seeking, shingled mechanical drives’ slow writing, and flash drives’ finite endurance). Most file systems cannot adapt to changes in the space delegated to them or the storage devices they are spread across (including I/O parallelism). Most file systems cannot compress, deduplicate, tail pack, defragment, and encrypt contents. Most file systems cannot be mounted read-only and store writes elsewhere (e.g. to a RAM drive). This is useful for live media (e.g. system recovery tools). Most file systems cannot perform backups while the file system is in use. Most file systems cannot detect and correct bit rot. Most file systems do not require file system paths to be NFC UTF-8 without a BOM. Most file systems do not forbid file system path components from containing unterminated bidirectional control characters. Most file systems use harmful attributes (e.g. access time and hidden) and obsolete permissions (e.g. readable, writable, and executable by user, group, and others). Most file systems cannot undo accidents by recovering old versions of files or directories.	Macroni’s database file system solves these problems. XMP sidecar files store metadata when exporting to or importing from currently popular file systems.
Software is often monolithic, causing maintenance, satisfaction, and efficiency problems. Some packaging systems can only install or update packages from a single repository, causing a single point of failure for package management and security. Most packaging systems distribute executables, causing repository bloat and making trust difficult. Some packaging systems allow packages to be spoofed or tampered with to spread malware. Manually handling package dependencies is annoying. Most packaging systems allow packages to damage other packages’ or users’ files, (un)install other packages without the owner-user’s consent, and annoy users with prompts (e.g. about licensing, configuration, or feedback) upon (un)installation. Package upgrades can break software.	Macroni’s packaging system solves these problems.
Most operating systems suffer from versioning hell.	Macroni enforces the use of Simplified Semantic Versioning.
Some software only works when it can periodically or constantly connect to a network server. This is incompatible with unreliable network connections, air gaps, and software preservation.	Macroni is not a Service as a Software Substitute. It works offline indefinitely.
Processes crashing or hanging can cause data loss. Restarting processes to update them is annoying.	Macroni uses microrebooting to solve these problems.
Power loss or operating systems crashing or hanging can cause data loss. Some operating systems force, often unexpected, reboots to update themselves, which can cause data loss. Restarting operating systems to update them is annoying.	Macroni uses transparent persistence to solve these problems. Macroni will never force a reboot because the owner-user could disagree with that decision. When the owner-user chooses to reboot, transparent persistence will restore all process’ local state.
Most code is vulnerable to attacks on integrity and confidentiality.	Macroni uses the capability security model between processes. The rest of this problem is solved at the programming language level.
Most code is vulnerable to attacks on availability.	The operating system constructs that consume space or time have been identified and minimized and their availability problems solved.
Most GUIs are resolution dependent, making them scale poorly. Secure user interaction is usually unnatural and annoying. Lack of a common, consistent, rich set of widgets and flat design have caused GUI usability to regress to early 1980s norms.	Macroni’s user interface solves these problems.

Integrated Development Environment Features
Problem	Solution
Manually matching brackets is difficult.	Macroni’s source code editor is a structure editor that can toggle highlighting based on the depth of bracket nesting around the cursor.
Noticing changes damaged code is usually difficult.	Macroni’s source code editor performs analyses to warn about bad practices and defects in the background.
Discovering code is usually difficult. Auditing code is usually difficult.	Macroni’s source code editor can search for code. Macroni supports docstrings. Auditing code is easier when you can search for type definitions and untainting procedures.
Comprehending code is usually difficult.	Macroni’s source code editor can cross-reference code.
Manually refactoring code is difficult.	Macroni’s source code editor provides automated refactoring.
Most development environments suffer from build system hell.	Macroni is self-building. Package building is standardized.
Testing your comprehension of code is usually difficult. When a programming language provides listeners, their transcripts are usually unsuited for use as source code.	Macroni provides listeners (a.k.a. REPLs) for interactively testing code. Macroni’s listeners’ transcripts require minimal editing to use as source code.
Discovering all relevant invariants is usually difficult.	Macroni provides an invariant detector.
Discovering defects by testing is usually difficult.	Macroni provides a property-based tester.
Locating unreliable code is usually difficult.	Macroni’s debugger can display modules sorted by how often they caused crashes, hangs, or corruption. Macroni automatically reports crashing and hanging. A sandbox can report corruption as the reason for destructing a sub-sandbox. A module is blamed for crashing or hanging when the program counter was in it at the time of destruction, and the debugger can display the source locations of those program counters. A module is blamed for corruption when it was loaded directly into a sandbox destructed for that reason, but no source locations can be displayed, so convert corruption to crashing via assertions. The program package responsible for each error report is recorded. This exposes which modules are causing problems for a program. It also exposes malware attempting to defame modules in other packages and makes removing false reports possible.
Comprehending defects is usually difficult.	Macroni’s debugger records the last several steps of execution and their resulting values along with the memory dump. How many to record can be configured by the owner-user.
Locating wasted space and memory leaks is usually difficult.	Macroni provides a space profiler.
Locating wasted time is usually difficult.	Macroni provides a time profiler.
Comprehending defects in regular expressions is usually difficult.	Macroni provides a regular expression debugger.
Specifying a GUI by programming is like painting by typing.	Macroni provides a GUI builder.

Lisp Features
Problem	Solution
Different programming languages have different strengths, so programmers should use the right tool for the job. Unfortunately, different programming languages usually do not link together.	Lisp’s strength is extending itself, including embedding other programming languages in itself, so it is the right tool for every job. It can generate code to define its runtime system or for computing systems too limited to host it. Lisp links with itself.
Discovering defects is usually difficult.	Lisp macros can perform any static or dynamic analysis.
Debugging macros and external code generators is usually difficult.	Lisp macros are no more difficult to debug than functions. Errors report the macro call location, not a location in the generated code.
Programming languages that use implicit grouping often cause unintended evaluation and order of evaluation defects.	Lisp uses only explicit grouping.

Lisp Dialect Features
Problem	Solution
Most programming languages and computing systems are designed for obsolete batch processing.	Macroni is event-driven, ultimately by hardware interrupts.
Some programming languages ignore errors, maximizing the destructiveness of defects and the difficulty of debugging.	Macroni does not ignore errors.
Most programming languages overcomplicate error handling.	Macroni handles defect errors by microrebooting and represents system errors as values.
Most code is vulnerable to attacks on integrity and confidentiality.	Macroni uses the capability security model and taint checking within processes. The rest of this problem is solved at the operating system level.
Most macro systems are dangerous, limited, or verbose.	Macroni uses an implicit renaming macro system that is hygienic by default, procedural, and concise.
Some programming languages use mutable literals, causing copying or shared state when copying was intended.	Macroni uses immutable literals.
Lists waste space and time.	Macroni uses immutable vectors (trees of short arrays) instead of lists. Immutable vectors support structure sharing and efficient random access with less pointer overhead and more locality of reference, and they can be efficiently constructed forward.
Some programming languages use 1-based array indexing, making computing indices error-prone and verbose.	Macroni uses 0-based vector indexing.
Some programming languages do not perform bounds checking, causing buffer overflow vulnerabilities.	Macroni performs bounds checking.
Most Lisp dialects conflate the empty list, the null pointer, and false, causing the semipredicate problem. Some programming languages conflate the null pointer and 0, causing the semipredicate problem.	Macroni distinguishes between '() (the empty immutable vector), #null, 0, and #f.
Most Lisp dialects are case-insensitive, unlike math and most computing systems.	Macroni is case-sensitive, but uppercase in identifiers is intended to only be used for math transcription and interoperability.
Some programming languages do not allow modules to control what they import or export, causing name collisions and exposing implementation details.	Macroni only imports and exports what modules explicitly request with the exception of built-ins. Built-ins are locally implicitly redefined instead of causing name collision defects so that built-ins can be extended without breaking code. There is no way to import or export everything from a module so that modules can be extended without breaking code. Imported and exported constructs can be locally renamed for readability and compatibility.
Global scope causes maintenance problems.	Macroni’s widest scope is module scope.
Some programming languages do not support shadowing, breaking composition.	Macroni supports static scope and inner scopes shadowing outer scopes (i.e. lexical scope).
Dynamic scope does not compose.	Macroni does not support dynamic scope.
Some Lisp dialects use separate namespaces for variables and procedures, making functional programming verbose.	Macroni uses the same namespace for variables and procedures.
Some programming languages’ built-in constructs are unnecessarily inconsistent with user-defined constructs, interrupting the programmer’s flow and causing accidental complexity.	Macroni’s built-in constructs are as consistent as possible with user-defined constructs.
Most programming languages suffer from the expression problem.	Macroni’s object system can separately define types and generic functions while enforcing encapsulation.
Asserting all relevant invariants is usually error-prone and verbose.	Macroni supports design by contract.
Most Lisp dialects coerce the empty list and false to false and all other values to true in Boolean contexts, which is useless. Most Lisp dialects do not coerce false to 0 and true to 1 in arithmetic contexts, which prevents the use of bitwise operations and branchless programming.	In Macroni, the bool<- generic function converts empty data structures, "", #null, zero, and #f to #f and all other values to #t. Values are coerced that way in Boolean contexts. Boolean special forms (and and or) always return a Boolean value. This is useful for bitwise operations, branchless programming, and processing strings and data structures. Macroni promotes #f to 0 and #t to 1.
Some programming languages use Boolean operators that do not short-circuit (stop evaluating subexpressions when the result is known), making programming error-prone, verbose, and inefficient.	Macroni’s Boolean special forms short-circuit.
Some programming languages only provide floating-point numbers, making working with binary files and network protocols error-prone and verbose. Some programming languages do not follow conventions for numeric promotion, making arithmetic verbose. Some programming languages do not support syntax for integers in binary and hexadecimal and floating-point numbers in scientific notation without a decimal point, making arithmetic error-prone.	Macroni provides the full numeric tower. Macroni follows conventions for numeric promotion (e.g. integers are promoted to floating-point numbers when used in arithmetic with them). Macroni supports syntax for integers in binary (e.g. #b-101) and hexadecimal (e.g. #x-F00) and floating-point numbers in scientific notation without a decimal point (e.g. -1e-1).
Most programming languages ignore arithmetic errors.	In Macroni, integer underflow and overflow causes promotion to bignum. Modular fixnum arithmetic is supported via built-in functions but is not the default. In Macroni, integer and floating-point division by zero, floating-point underflow and overflow, and floating-point invalid operations are defects.
Most programming languages ignore dimension and unit errors.	Macroni performs dimensional analysis.
Some programming languages use biased rounding or truncation, causing cumulative error.	Macroni uses unbiased rounding (round half to even) but provides explicit truncation.
Some programming languages define 0⁰ as always being a defect, breaking several branches of math.	Macroni defines 0⁰ as 1 for discrete (integer) exponents and as a defect for continuous (non-integer) exponents, following math conventions.
Most programming languages only provide quotient (a.k.a. truncating division) and remainder (a.k.a. truncating modulo) despite floor division and modulo being more useful. When floor division is provided, sometimes it does not return an integer, making using it error-prone and verbose.	Macroni provides floor division returning an integer and (flooring) modulo.
Some programming languages use mutable strings despite their mutability being useless and it preventing strings from being hashable or internable.	Macroni uses immutable strings.
Some programming languages do not support UTF-8, breaking modern text.	Macroni source code and dynamically constructed characters, strings, and symbols are NFC UTF-8 without a BOM. Characters are extended grapheme clusters. Positions within a string are measured in such characters, but strings are not random access. Macroni source code uses LF line endings. Comments, strings, and symbols cannot contain unterminated bidirectional control characters. File system path components are represented as byte vectors instead of strings because foreign file system path components may not be valid strings in Macroni.
Some Lisp dialects allow whitespace and special characters in symbols, which is only useful for obfuscation.	Macroni’s symbols are UAX #31 identifiers (which disallow whitespace and special characters) with adjustments to allow conventional Lisp symbols, but characters outside the ASCII range in identifiers are intended to only be used for math transcription and interoperability.
Regular expressions are usually difficult for humans to read and for structure editors to edit, and they usually support non-regular features that are inefficient.	Macroni uses something similar to SREs.
Generators use mutation, so they cannot backtrack.	Macroni uses something similar to Scheme streams.
Most programming languages do not represent random access data structures as functions, making using them verbose.	Macroni represents random access data structures as functions.
Most Lisp dialects have poor support for sets and dictionaries.	Macroni provides immutable, mutable, and weak key dictionaries. The syntax for dictionary literals is easy for humans to read and for structure editors to edit (e.g. {("key a" "value a") ("key b" "value b")}). Macroni’s dictionaries are insertion order preserving and order insignificant (i.e. dictionaries with equal contents in different orders are equal). Insertion order is the only order that makes sense when keys are not required to be ordered. Order preservation in dictionaries is often desired. Order significance in dictionaries is rarely desired and often surprising. Dictionary sorting and order significance are easily implemented when dictionaries are insertion order preserving. Macroni’s immutable and mutable dictionaries are indexed by value for immutable keys and by identity for mutable keys. To be clear, they are not indexed by string representation and string keys are compared case-sensitively. Macroni’s weak key dictionaries are indexed by identity for immutable and mutable keys. This prevents vulnerability to malicious definitions of hashing and value equality. Macroni’s dictionaries can be processed as functions, sets, dictionaries, or sequences of vectors containing key-value pairs. Set operations on key-value pairs have their uses. Implementing Macroni’s environments efficiently in itself and the capability security model require these definitions.
State often becomes corrupt.	Macroni minimizes state by only allowing mutation where it is necessary (state must survive across events and revocation and confinement of capabilities must work) or unobservable and useful (temporary local state sometimes improves elegance and efficiency).
Some programming languages separate allocation and initialization, causing uninitialized values in variables and data structures.	Macroni combines allocation and initialization.
Most programming languages use the same form for binding and mutation, causing typoed variable names to go undetected and mutation when binding was intended.	Macroni uses different forms for binding and mutation.
Most Lisp dialects overcomplicate equality.	Macroni defines only two equality predicates: is and =. is tests identity equality (a.k.a. physical equality). Be aware that is cannot be extended, unlike most built-ins, because security requires identity equality to be trustworthy! = tests value equality (a.k.a. structural equality). Be aware that testing the value equality of types incompatible with value equality (e.g. different but equivalent immutable closures) or each other (e.g. most different non-numeric types) will return #f instead of crashing! Macroni’s dictionaries require these definitions.
Tail calls usually waste space.	Macroni performs tail call elimination.
Optional and keyword parameters do not fit the execution model.	Macroni does not support optional or keyword parameters. It does support generic functions with different definitions for different arities and variadic parameters.
Most programming languages do not define the order of evaluation of all subexpressions, making programming error-prone.	Macroni evaluates all non-macro subexpressions once from left to right. To be clear, the functional position is evaluated the same way as a function argument before any arguments. Special forms may skip some subexpressions. Optimizers may use a different order only if the difference would be unobservable. The behavior of macros is defined by their author, but they should evaluate subexpressions once from left to right unless they have good reason to deviate.
Unstructured control flow (not single entry, single exit) usually does not compose, makes ensuring initialization and cleanup code is executed correctly difficult, and makes static analyses difficult for humans and computers.	Macroni supports only structured control flow.
Multiple return values do not fit the execution model.	Macroni uses pattern matching for destructuring instead.
Some programming languages do not use stable sorting procedures, breaking composition (nesting sorting criteria).	Macroni uses stable sorting procedures.
Some programming languages’ pseudorandom number generators make it impossible to regenerate sequences or are insecure.	Macroni provides a fast pseudorandom number generator whose state can be read and written so that it is possible to regenerate sequences. Macroni provides a separate cryptographically secure pseudorandom number generator using entropy pools.
Most optimizers cannot optimize Lisp dialects well.	Macroni’s restrictions on mutation and exposure from modules and types make static analyses easier for humans and computers. Macroni’s optimizer does not use a programming language independent intermediate representation so that it does not lose semantic data.
Optimizers are slow.	Macroni’s module dependencies form a directed acyclic graph. This enables incremental static analyses and incremental compilation. Initially packages are compiled without optimization and with instrumentation. Data inferred about a module is stored with it so that analyses are not reperformed needlessly. Packages can be recompiled with profile-guided aggressive optimization in the background. The improved code can be hot-swapped in. Packages can include profiling data so that other installations can be efficient immediately. They cannot include other inferred data because that would be insecure (e.g. it could incorrectly eliminate dynamic bounds and type checks).
Most garbage collectors cause pauses.	Macroni’s garbage collector is incremental.
Most garbage collectors are slow.	Macroni’s garbage collector is fast for several reasons: Macroni avoids allocating on the heap when possible. It is quiescent when not allocating on or reading the heap, and it only mutates a variable during most heap reads. It uses a semi-space algorithm, so it only traverses reachable objects, unlike mark-and-sweep. It uses a bump allocator, the fastest kind of allocator. It self-optimizes for locality of reference. Destructing a sandbox deallocates its memory without performing garbage collection.
Most garbage collectors do not scale.	Macroni’s sandboxes’ heaps are separate. Garbage collection between them is potentially parallel.

Macroni is influenced by several other projects.