Jun 25, 2017
Release 0.9.0
NEWS

With 1287 commits since 0.8.4 this release is the biggest one in the
history of Viua so far.
And not only in terms of commit count, but also functionality.

CLOSURES

Closures were reworkerd.
The `enclose*` family of instructions was renamed to `capture*`, and
closures got their own assembly directive - `.closure:`.
Functions declared as closures are not directly callable and
must be instantiated first using `closure` instruction.

CONCURRENCY

Concurrency-related aspects of the VM also got some love.
Blocking operations (`join` and `receive`) can now specify a timeout for
how long they should block a process.
The timeout may be specified in seconds, milliseconds, or
as the `infinity` token.

DEFAULT VALUES

Some instructions now can be written in a shortened form and
the assembler will inset default values for omitted tokens.
These include for example `join`, `receive`, `istore`, and `fstore`.

DEFAULT COMPILE-TIME KEYWORD

The new compile-time `default` keyword can be used wherever it is legal
to omit a token.
The assembler will change the `default` keyword into the default value
for that place.
It is useful when usage of the default value should be stated explicitly.

IOTA COMPILE-TIME KEYWORD

The new compile-time `iota` keyword generated an ever-increasing integer
starting from 1.
It can be used to automatically assign register indexes:

    .name: %iota answer
    istore %answer local 42
    print %answer local

It is especially useful when code is changed, as the assembler will
reindex the registers automatically and free the programmer from this
task.

WATCHDOG PROCESSES CANNIBALISE TIME OF CRASHED PROCESSES

Previously, when a process crashed, the VM spawned a watchdog and
run it to completion.
This is now fixed, and crashed processes become their own watchdogs on
failure - so watchdogs are now on the same level as all other processes,
and may be preempted to prevent starvation of "normal" processes.

Before the crashed process becomes a watchdog its stack is unwound.

VOID TARGET REGISTER

The new `void` keyword can be used as a target register.
Using void register as the target register will drop the value that
would by normally produced by the instruction.
Some examples:

    ; drop the result of function call (if any)
    call void foo/0

    ; delete the value
    move void %1 local

EXPLICIT REGISTER SET SPECIFIERS

The `tmpri` and `tmpro` instructions are no longer needed as the VM
supports explicit register set specifiers for register operands.
This means that to move a value from local register set to the static
register set the following instruction can be used:

    ; move <target> <source>
    move %1 static %1 local

...instead of this sequence:

    ress local
    tmpri %1
    ress static
    tmpro %1

Explicit register set specifiers make code shorted and more efficient.

ATOMS

Atoms are unique values whose only property is that they may be checked
for equality.
Useful as tags.
Supported in Viua by `atom` and `atomeq` instructions.

TEXT AND UTF-8

Starting with this release, Viua uses UTF-8 as its internal character
set for text values.
What is more, a special text type was added to the VM's list of
primitive types.
Text values must always be valid UTF-8.

A `text*` family of instructions was introduced to distinguish text
values from a "string of bytes" values produced by `strstore`
instruction.

As an additional feature, all values can be casted to text using the
`text` instruction:

    istore %1 local 42
    text %2 local %1 local      ; register 2 will contain text "42"

DEFERRED CALLS

A very useful feature.
Deferred calls may be registered to be called when the frame they were
registered in is popped off the stack (during unwinding, normal returns,
or tail calls).
They are useful as a debugging aid, and can be used to implement
resource management schemes.

More information about deferred calls can be found on the
weekly.viuavm.org blog.

ADDITIONAL NOTES

This release also introduces one very useful and important feature:
processes can now contain many stacks, and switch execution between
them.
Such an additionl will make implementation of several new
functionalities much easier and more intuitive.
For example:

- interrupts: they must be run in the context of a specified process,
  but are in no relation to what the process is currently executing so
  they are a prime candidate to be run on a different stack

- message filtering: a function could be provided to filter messages
  available in a process' mailbox, and as this function's execution is
  is only a tool to achieve some other goal, it would be a good idea to
  run the function on another stack

- stackful coroutines: coroutines that can yield from frames at
  arbitrary depth of the stack, not only from the top-most frame

and possibly more.

A style guide has been introduced with the help of the `clang-format`
tool.

VM got a basic static analyser that is able to catch errors at compile
time.
Aug 28, 2016
Release 0.8.4
NEWS

With 75 commits since 0.8.3 this relase brings few, but important improvements to
the core of the virtual machine.

SMP

Viua now support simultaneous multiprocessing for virtual processes.
The maximum number of virtual process that can be run in parallel depends on the settings the machine
is started with, and is limited by capabilities of the underlying hardware (for example - the VM will
not run 4 processes in parallel if only two CPU cores are available, assuming each core can run only
one thread).

INSTANT DETACHING

Processes can be spawned as immediately detached, by using 0 as the target register index.
The VM will interpret this as "spawn this process and execute it, and I am not interested in
communicating with it".
The VM will give parent process neither the possibility of joining spawned process, nor will it
return a PID for the process.

LAUNCH-TIME CONFIGURABLE SCHEDULER NUMBERS

Viua can be instructed to spawn a certain number of VP and FFI schedulers at launch-time, the limits
are no longer hard-set at compile time.
The number of spawned schedulers can not be changed at runtime.

Two environment variables control the number of schedulers spawned:

- VIUA_VP_SCHEDULERS: for number of VP schedulers
- VIUA_FFI_SCHEDULERS: for number of FFI schedulers

The CPU frontend provides the `--info` option; among other things, it provides information about
scheduler numbers the VM would spawn in current evironment.
Use `--json` option to get `--info` output in JSON format.
Aug 5, 2016
Release 0.8.3
NEWS

With 141 commits since 0.8.2 this release is mostly a gradual
Release 0.8.3 is mostly a gradual improvement over 0.8.2, the 141 commits pushed
to the repository introduce some new but not revolutionary features, and
a bunch of minor fixes.

The big things release brings are vector packing, multiple FFI schedulers, and
possibility of embedding metadata in Viua VM bytecode files.

NEW COMMENT SYNTAX

The assembler supports Lua-style comments beginning with `--` and
running till the end of line.
Semicolon comments are not deprecated and can still be used.

EXTERNAL FUNCTION AND BLOCK SIGNATURES INCLUDED IN BYTECODE DISASSEMBLY

Before 0.8.3 the disassembler did not emit signatures for functions and
blocks linked from external to the analysed translation units.
This proved to be extremely inconvenient and was fixed in this release.
Viua VM bytecode files now be freely disassembled and reassembled without
worrying about missing signatures during reassembling.

EMBEDDING METADATA IN BYTECODE FILES

Starting with this release Viua VM bytecode format supports embedding metadata in
bytecode files by employing `.info:` assembler directive.
Example line:

    .info: license "GNU GPL v3"

Metadata is stored in a simple map where keys are restricted ASCII identifiers, and
values are strings.
String is currently the only embeddable value type.

COMPILE-TIME JUMP TARGET VERIFICATION

Jump targets are now verified at compile-time and assembler refuses to generate bytecode when
it detects jump errors in input source code.
Some absolute jumps cannot currently be verified by assembler and their verification is
perfoemed at runtime (e.g. absolute forward jumps).

COMPILE-TIME BLOCK CHECKING

Assembler checks if signatures for blocks used in a translation unit are present, and
refuses to compile code when input source code contains references to undefined blocks.

VECTOR PACKING

`vec` instruction can now be used to pack objects during vector creation.
Until this release vectors had to be created by spawning a vector object, and
then pushing objects one by one to the newly created container.

    vec 1
    vpush 1 (istore 2 40)
    vpush 1 (istore 2 41)
    vpush 1 (istore 2 42)

This proved to be cumbersome, so `vec` instruction was augmented to support packing.
Now, vec instruction receives three operands instead of one (the second and
the third operands are optional, and default to zero).

    istore 2 40
    istore 3 41
    istore 4 42
    vec 1 2 3

Second operand to the vec instruction is the index of the first register to be packed,
the third is the number of registers to be packed.
Above code creates a vector in register 1, and packs three objects starting from register 2.
Packed objects are moved inside the vector, leaving input registers empty.

By placing objects into registers carefully vector packing can be used to speed up
multiple-value returns from functions.

REMOVE RACE CONDITION WHEN REGISTERING EXCEPTIONS THROWN BY FFI CALLS

This release removes a race condition from the code that could cause the machine to lose
exceptions that were thrown by FFI calls.

The race condition was triggering a bug when an exception was registered in a process by the
FFI scheduler while VP scheduler was inspecting the process's state after executing it.
If the exception was registered between checks for terminated-status, and stopped-status the
exception has been quietly dropped and the process removed from the pool;
the precise order of events that triggered the bug was:

- exception has been thrown in FFI call, but the FFI scheduler did not yet register it in the
  process that requested the call
- VP scheduler did not mark the process as terminated (and so the exception handling routines
  have not been invoked)
- FFI scheduler registered an exception (which pushed the process into a terminated-stopped
  state)
- VP scheduler checked if the process stopped and removed it from the list of running
  processes

MULTIPLE FFI SCHEDULERS

As of 0.8.3 Viua VM is now able to spawn and utilise multiple FFI schedulers (their number
configurable at compile time, by default the VM spawns two).
This means that several FFI calls can be serviced in parallel, so the machine blocks less and
is able to execute FFI-intensive programs faster.

By default Viua spawns three C++ threads now, which causes overscheduling on systems with single
or dual-core CPUs.
VM configuration can be adjusted if this is undesirable.
Jul 6, 2016
Release 0.8.2
NEWS

With 171 commits since 0.8.1, this release brings improvements
to machine's core code, and some long overdue fixes.
The changes in this utterly uninteresting release are mostly internal and
will not (*should* not) affect user code much.

EXTRACTED VIRTUAL PROCESS SCHEDULER OUT OF CPU

Virtual process scheduler was extracted out of the CPU code.
This means that the CPU is now only the source of static information about
running program - what modules are loaded, where a function begins, what
methods a class responds to, etc.
It also holds the queue of FFI call requests (it has not changed since the last
release).

The *dynamic*, process-related information that are prone to change during lifetime
of a program (e.g. frames on call stack of process, exceptions mid-flight, messeges
in transfer) are now handled by "virtual process scheduler", which is a special class
concerned only with managing VM processes.

For now, the VM spawns only one VPS (virtual process scheduler) but in future releases
it will be changed, and machine will spawn a number of VPSs - each in its own host
thread.
Introduction of multiple active VPSs will mark the move from just concurrency, to
true parallelism of virtual processes inside the machine (i.e. they will not just
run one-after-another round-robin style, but some of them might actually be running
at *the same* time, only under different scheduler).

FEATURE OF THE DAY: std::unique_ptr<>

Machine now employs `std::unique_ptr<>` in some places in code to manage
lifetime of dynamically allocated objects.
The `std::unique_ptr<>` is now heavily used only in the `VirtualProcessScheduler`
code to manage stack frames, exceptions, and global and static register sets, but
in later releases it will gradualy replace naked pointers in other parts of the VM.

LESS CRASHING, MORE STACK TRACES

In previous releaes, when one virtual process crashed, and the crash was not handled
by the watchdog process, it brought down the whole VM, and produced a stack trace.
This behaviour is now altered: when a process crashes and there is no watchdog to
catch the escaped exception the VM will print a stack trace for the crashed process
but will continue running.
This change is intended to enhance reliability, errors will always happend, they just
should be isolated from the "healthy" parts of the system and serviced, instead of
bringing the whole system down with them.

What if the `main/` function crashes?
If there is a watchdog process to handle the failure, the exception will be serviced and
the VM will continue execution, to finish with exit code 0.
If there is no watchdog process, machine will continue running and
when all other processes finally terminate it will close with exit code 1.

WATCHDOG PROCESS SPAWNED PER-VPS, NOT PER VM

Each VPS manages its own watchdog; thus, virtual-process-crashing errors are localised
to the next nearest point after the process, instead of being propagated further (to
the CPU) and clogging up the metaphorical pipes.
Currently, there is only one active VPS so this move does not change much, but this
will pay off when machine starts spawning multiple schedulers.

FUTURE

According to release schedule, next release should introduce multiple FFI schedulers.
This will provide speed improvements due to greater parallelism on machines with more
cores to utilise (as each scheduler will run in parallel on its own thread).
Machines with one and two cores will not notice any change; and there are no plans to
"overschedule", i.e. to spawn more threads than there are real cores available to
utilise (if the hardware provides only two cores, only one FFI scheduler will be spawned).
Jun 9, 2016
Release 0.8.1
NEWS

With 238 commits since 0.8.0, this release brings in another set of improvements
the code of the machine; stronger compile-time checks, improved entry function
generation in assembler, different flavours of main function and
fixed FFI scheduling.

STRONGER COMPILE-TIME CHECKS

Assembler is now able to better verify function calls; a whole new verification
stage was introduced into the assembler's code which is able to detect arity
mismatches between function calls and declarations, frames with gaps (i.e. when
not all parameter slots in the frame are filled), and double-passing parameters.

Also, all errors are now enabled by default - this change was suggested by Harald
Eilertsen (https://github.com/snake66), who also provided valuable insight on the
topic of multithreading in C++, which is highly relevant to another part of this
release.

FIXED FFI SCHEDULING

Viua VM spawns two threads when it launches: one runs virtual processes of the
machine, and the other runs functions executed directly on the host CPU (the
foreign functions).
Release 0.8.0 shipped with a race-condition which sometimes could make the
machine hang after the FFI call was scheduled.

IMPROVED EXPIRED POINTER HANDLING

When an expired pointer is encountered, its type may be safely inspected and
its string representation may be safely generated.
In previous releases this would throw an exception.
This is particularly important when an expired pointer is present in stack
trace after a virtual process crash as it may be printed without any
problems.

FATAL VM EXCEPTION WHEN WATCHDOG PROCESS EXITS

Watchdog process cannot finish execution normally.
If the machine detects it did, it throws a fatal exception and
shuts itself down.

IMPROVED SAMPLE CODE

Sample code must have been updated for current release due to stricter
compiler checks.
As a bonus, old non-testing code has been cleaned up and
rewritten in up-to-date VM assembly.

FUTURE

Next release will be mostly a refactoring one, so should arrive in less than
a month if everything goes smooth and as planned (fingers crossed).
It will bring a new virtual process scheduler, extracted from the main CPU
code; the CPU will become more static - only providing information about
entry points locations, type hierarchy status and routing calls between
virtual and FFI schedulers.
May 7, 2016
Release 0.8.0
NEWS

This is another release introducing substantial changes to machine's code and
capabilities.

CLOSURES

Viua 0.8.0 sports much better support for closures than previous releases by
giving programmers way to choose which parts of the environment (which values) are
captured and how they are captured (by copy, by move, or by reference).
The cost for this feature is lost backwards compatibility.

VIRTUAL PROCESSES

Threads are renamed to processes.
A unit of execution in Viua VM is a "virtual process".
Each virtual process is separated from every other one.
Processes communicate only by means of message packages.
That does not sound very much like a definition of a "thread".

INTER-FUNCTION TAILCALLS

Viua 0.8.0 supports inter-function tailcalls.
Looping can be now be elegantly implemented using recursion instead of
assembler-like `branch` and `jump` instructions.

IMMEDIATE COPY ON PASS-BY-COPY

Starting with this version, machine immediately copies parameters passed by copy.

FOREIGN CALL OFFLOADING

Before this version FFI calls were blocking whole machine, i.e. if one process called
a foreign function, no other process could run until foreign function returned.
This was caused by the fact that machine was single-threaded.
Viua 0.8.0 spawns two threads: one for running native Viua code, and a second one
for running foreign code.
The process that called a foreign function is suspended until the foreign function returns
but all other VM processes are not.

STANDARD LIBRARY IMPROVEMENTS

Some additions and improvements were made to standard library modules `std::misc`,
`std::vector` and `std::functional`.
Feb 16, 2016
Release 0.7.0
NEWS

This release introduces some substantial changes to the machine, and
that is the reason for jumping from 0.6.1 straight to 0.7.0.

Some backwards incompatible changes:

- `throw` no longer leaves thrown objects in their registers,
  thrown objects are moved out of their stack frame,
- `free` instruction was renamed to `delete`,

Some more backwards incompatible changes, this time due to introduction
of pointers.
Using pointers if cheaper than using references (by a large margin) so a few
standard library functions were rewritten to accept pointers where they used to
accept references.
Pointers are cheaper because they do not escape machine's basic memory management
model.

Then, a fix.
If a detached thread generates uncaught exception the machine will print out
trace of the correct stack.
Previously always the stack of `main()` function was printed.

One feature implemented in this release is a "pass-by-move", a very efficient method
of passing parameters to functions.

Last, but not least, come the enhancements in multithreading.
First, return values can be extracted from threads when they are joined.
Second, threads can be suspended and woken up; this is important because the CPU can
easily skip suspended threads without looking at their state (other than the `suspended`
status).

Last feature introduced by the 0.7.0 is also one of the more important ones.
Machine provides a way for the programmers to specify a function to be run as
a "watchdog thread".
If a thread dies when a watchdog is active the machine passes a message to the watchdog
telling it what thread died, and why.
This enables the watchdog to restart the deceased thread if necessary.
If the watchdog itself dies it is restarted automatically by the machine.
In my opinion, this is a great win for the reliability of software written to run on the
machine.
Instead of worrying about every edge case the programmer can let the software crash,
log the error, restart the part of the program that failed, and carry on like if nothing
at all has happened.

As always, there are also numerours small improvements and bug fixes.
Nov 14, 2015
Release 0.6.1
NEWS

This release introduces basic multithreading support to the machine.
The threads are green, i.e. they are implemented inside the machine and
are not system threads.

During the first (very simple and probably useless) benchmark, the machine
has been switching between 510,000+ threads.
Each thread had more than 1,000,000 opcodes to execute.

Next releases will focus on expanding theading functionality.
Features to implement include message passing, return value extraction, and
priority modifications.
Oct 24, 2015
Release 0.6.0
550 commits since last relase.

NEWS

This relase amounts to two months and a week of work.
The immediate jump from v0.5.0 to v0.6.0 is justified, because not only
the time period between releases is unusually long (in terms of Viua relases) but
also because the change scope is unsusually broad.

To sum up the relase in six words:

> Refactoring, bugfixes, features. In large quantities.

CHANGELOG

1) Added `Prototype` class for user-defined types

User-defined types can be created with `class`, `derive`, `attach` and `register` instructions.
Instances of these types are created with `new` instruction.
Read sample code and documentation to learn more.

2) Better error messages after crash

Long traces do not prevent users from discovering source of the crash - exception
details are printed *after* the stack trace.

3) Documented and tested function fall-through behaviour

Functions support falling-through to the next one.
This could lead to endless loops should the "next" function call the "falling" one.
Such bugs are prevented by limiting stack size to 8192 frames.
The limit may be changed at compile time.
An `Exception` is thrown when the limit is exceeded.

4) EXCALL opcode is no more

The `excall` instruction has been removed from the bytecode definition, and
`call` instructions should be used instead.
This provides a uniform call interface for both native and foreign functions.

5) EXIMPORT opcode is no more

The `eximport` instruction has been renamed to `import`.

6) `VIUAPATH` and `VIUAAFTERPATH` used by `link` instruction

It is now possible to put Viua libraries and tell the machine where to look for them.

7) Added `VIUAPRELINK` and `VIUAPREIMPORT` environment variables

These environment variables may be used to tell the machine to always import and link
specified modules.
Note that the machine must be able to find requested modules on its `PATH` variables.

8) Support for relative jumps in assembler

Users can now write `jump -9` or `branch 4 +8 +1` to tell the assembler to generate jumps
relative to the current instruction index.
Note: jumps are recalculated as absolute in final executables.

9) Merged pull-requests fixing build issues from @saper

A big "Thank you" for @saper for sending pull requests that fixed some build issues on FreeBSD.

10) More language constructs added to assembler

For example, instruction counting using `[ (instr0) (instr1) ... (instrN) ]` syntax or
instruction nesting using `instr2 (instr1 (instr0 op0))` syntax.
Again, read sample code and documentation to learn more.

11) Dynamic dispatch using `msg` instructions

Viua supports dynamic dispatch of methods and all types that have registered at least one method.

12) User-defined types written in C++

Viua provides support for nearly-direct usage of classes defined in C++ from machine's bytecode.
This is done using black magic, pointers to members and virtual functions.

The most important thing to know is that a C++ class must derive from `Type` and
provide methods with correct signatures to be loadable into machine's typesystem.
Read sample code and documentation to learn more.

13) Better implementation of references

References should not leak memory now.

14) Altered catch-throw mechanism instructions

Renamed `try` instruction to `enter`, and `tryframe` to `try` in order to have code read in a
more intuitive way.

15) Added `std::io` module

Very basic I/O capabilities.

16) Added `std::random` module

Very basic random-related functions using standard Linux interfaces.

17) Bytecode size and jumps are encoded using `uint64_t`

This makes the bytecode size bigger, but allows creating programs much bigger than 65 kilobytes.
Jul 15, 2015
Release 0.5.0
249 commits since last release.

NEWS

This release introduces several major features and changes.
It also continues the process of fixing machine's bytecode layout and assembly syntax.

Features and changes introduced in this version are **huge**.
They break compatibility with previous releases, but at the same time
greatly enhance capabilities of the machine.

Big newcomers are present both in standard runtime library, and
in core code.
Among them are:

* full support for dynamic linking,
* fixed exception handling,
* better compile-time checking,
* fixed external C++ libraries interface,
* enhanced standard runtime modules,

CHANGELOG

1) Changed interface for external C++ library modules

Interface that C++ libraries must provide for the machine has been changed.
Now, instead of two functions (`exports_names()` and `exports_pointers()`) they
must provide only one (`exports()`).
This single point of introduction makes it easier to spot interfacing bugs and
to create external libraries for Viua.

More information about external modules can be found in documentation.

2) Updated bytecode generation process

Bytecode generation process has been split into two stages.
Code responsible for each stage has also been separated.
High level *program generation* is driven by a bytecode generation API provided
by the `Program` class.
Low-level *bytecode generation* is carried out using functions
from `cg::bytecode` (i.e. *code generator - bytecode*) namespace.

3) Travis CI setup

Not a feature in the core of the machine, but a feature nevertheless, Travis CI environment has
been set up for the machine.

4) Opcode operands reordering

Many opcodes had their operands reordered.
The reason for this was to always have *target register* (i.e. the register whose value will
be changed by the instruction) as first operand.

5) Better compile-time code checking

Assembler is now able to catch more fatal errors during compilation.
New code privdes the assembler with ability to verify frame balance of the program, i.e.
detect frameless calls (situations where `CALL`, `FCALL` or `EXCALL` instructin is
executed without active frame), and excessive frame spawning (where two frames are spawned without
a calling instruction between).

This is a first implementation of this feature and as such should not be treated as entirely bugless.

Error reports are now more detailed and include line numbers to aid programmers in locating spotted bugs.

6) Dynamic linking

This release introduces full support for dynamic linking of modules written in native language of Viua.
Support is provided via `LINK` opcode, and `.signature:` and `.bsignature:` assembler directives.

Combined with better standard runtime library, this feature shines.
It is now possible to use the separate compilation capabilities machine provides to their fullest.
Programs that should be self-contained can be compiled statically and sent over the network.
Programs that should be lightweight and can rely on environment can use dynamic linking to
carry around less bytes and always use newest versions of the code.

7) `ARGC` opcode

This is a huge feature for frame introspection.
Until release 0.5.0, only `main/1()` function could easily detect number of its parameters.

Other functions had to wrap their parameters in a vector, and use `invoke/2()` from standard library
when they needed such functionality.
Now, with `ARGC` opcode available there is no need for such workaround.
Wvery function may easily inspect how many arguments it got, and if needed obtain vector
containing their copies with `misc::argsvector` block from standard runtime library.

8) Names of dynamically linked functions and blocks are not mangled

Current scheme for name mangling of dynamically linked code blocks has been dropped until
a better one emerges.
A no-mangling scheme may be adopted should it prove reliable enough.

9) Installation of Viua header files

Header files placement has been changed.
They can be now installed under `/usr/include/viua` for easy access during compilation of
external C++ libraries.