Skip to content

v0.12.0

Choose a tag to compare

@github-actions github-actions released this 30 Sep 09:15
· 164 commits to main since this release
Immutable release. Only release title and notes can be modified.
ca403e4
docker run --rm -v "$PWD":/work -u "$(id -u):$(id -g)" \
    ghcr.io/manticorephp/compiler:0.12.0 manticore compile app.php -o app

curl -fsSL https://github.com/manticorephp/compiler/releases/download/v0.12.0/install.sh | bash

The tarballs carry the compiler and its stdlib, not a toolchain: manticore
assembles its IR with clang and links with cc, against the host's
pcre2, openssl, sqlite3 and curl. install.sh checks for those and says
which are missing. The image carries the toolchain it was built with.
On Alpine (musl) use ghcr.io/manticorephp/compiler:0.12.0-alpine or the linux-musl tarball —
install.sh picks it by itself.

v0.12.0 adds Unicode: mbstring with every encoding Zend ships, and ext/intl over the
system ICU. It also adds a WebSocket server and client (ws:// and wss://, with
permessage-deflate) on top of Http\Server. Under async(), regular-file I/O and
getaddrinfo now run on a blocking-offload thread pool, and every async timer runs on the
monotonic clock. Alpine (musl) is now a supported target with its own images and tarballs.
php-cs-fixer now builds and runs natively. Most of the work behind that went into memory
ownership and into making the compiler faster.

Highlights

Unicode: ext/mbstring and ext/intl

  • mbstring, pure PHP, takes every encoding Zend knows (names, aliases and single-byte
    tables generated from Zend; UTF-16/32 and UCS-2/4 in PHP; CJK and stateful encodings via
    the host iconv). Case mapping in every MB_CASE_* mode, width, mb_detect_encoding
    scored as Zend scores it, MIME-header / numeric-entity / HTML-entity / UTF-7 / Base64 /
    QPrint / UUENCODE codecs. Fuzzed against Zend over 23 encodings. mb_ereg* not yet.
  • ext/intl is complete over ICU's C API: Normalizer, grapheme_*, Collator,
    NumberFormatter, Locale, Transliterator, IntlChar (62 methods, 666 constants),
    IntlTimeZone, IntlCalendar, IntlDateFormatter, IntlDatePatternGenerator,
    IntlBreakIterator, IDN, Spoofchecker, UConverter, ResourceBundle,
    MessageFormatter, IntlListFormatter, the procedural twins and the intl error state.
  • extension_loaded() answers true for mbstring and intl. ICU is linked dynamically,
    only into a program that uses intl; pkg-config finds it, including Homebrew's keg-only
    icu4c.

WebSocket (Http\WebSocket)

  • RFC 6455 server on Http\Server (via Http\Response::takeover and Server::onStop)
    and an outbound client, Http\WebSocket\connect; wss:// over the tls:// listener
    and client.
  • permessage-deflate (RFC 7692), all four parameters, context takeover; an inflation bound
    answers 1009 before a decompression bomb is allocated.
  • Callback layer: Handler + run()/handler() (onClose exactly once, 1011 on a
    throwing handler) and Hub broadcast. The client refuses CR/LF/NUL in headers and URLs.
  • Own test suite plus an Autobahn|Testsuite harness (tools/autobahn.sh). Guide:
    docs/websocket.md.

Async: blocking-offload pool and monotonic clock

  • Under async(), path-opened file I/O, file_get_contents/file_put_contents, the
    stat family, opendir/readdir/scandir, unlink/rename/mkdir/rmdir and
    getaddrinfo run on a small pool of native threads while the task parks: a slow file no
    longer stalls the loop. PHP never runs on a pool thread; output is byte-identical.
  • The pool is lazy and per process; MANTICORE_BLOCKING_THREADS sets its size (default 4,
    1..64); Async\stats() gains offloaded / pool_threads. A pooled call costs about
    10 µs; cancellation and starvation are documented in docs/async.md.
  • Deadlines, timers, stream timeouts and backoffs run on the monotonic clock: an NTP or
    date -s step no longer stalls or mass-fires them. awaitDeadline drops its waiter on
    expiry/cancel (an unbuffered send could report delivery and lose the value).

Alpine / musl

  • Alpine is a CI target on arm64 and amd64; releases ship :<v>-alpine / :alpine images
    and linux-musl tarballs, and install.sh picks the musl tarball on a musl host.
    musl surfaced three real memory bugs glibc had absorbed; all fixed.

Arithmetic errors, WeakMap

  • /, % and intdiv throw DivisionByZeroError; intdiv(PHP_INT_MIN, -1) and a
    negative shift throw ArithmeticError (they answered INF, 1, 0 or garbage). A shift of
    64+ is 0 / -1; % -1 no longer traps on x86_64.
  • WeakMap and WeakReference: entries die with their object.

php-cs-fixer runs natively

  • php-cs-fixer compiles and runs as a native binary; its 52 front-end type errors and five
    parse gaps were fixed at their roots. @PSR12 over Manticore's own src/: 242 of 254
    files byte-identical to Zend's output (was 203). @Symfony: 30.8 s to 26.1 s to 24.6 s.
  • On the way: #[Overload], core interfaces declared in the prelude, and the SPL iterator
    layer (IteratorIterator, RecursiveIteratorIterator, directory iterators,
    SplDoublyLinkedList, SplObjectStorage, …, each checked byte-for-byte against Zend).

Language & runtime

Objects and classes

  • Objects compare the way php compares them under ==, <=>, ordering, switch, sort,
    max and in_array. Private state counts, DateTime compares by instant, and enum cases
    and closures compare by identity. New attributes control this: #[CompareKey],
    #[CompareNone] and #[Uncomparable] (see docs/attributes.md). #[Struct] compares
    field by field.
  • (string)/echo/concat of an object without __toString throws php's Error instead of
    printing an address.
  • Property defaults are applied once, at allocation: parent::__construct() no longer
    resets a property the child already set.
  • A destructor runs exactly once (a resurrected object stays alive); $o = null runs it at
    that point; objects in a mixed/nullable property are owned, so their destructors run.
  • $o->$m() follows php's visibility, falling through to __call or php's exact Error.
  • new X of an unknown class throws "Class "X" not found"; an abstract class is never
    instantiated. instanceof self|static|parent and instanceof Closure work.
  • get_class() / get_debug_type() of a closure answer Closure.

Arrays and references

  • The int/string/closure element types of an array are kept on store.
    $f = [1.5]; $f[0] = 3 no longer gives float(3).
  • sort, rsort and usort reindex sparse and string-keyed arrays.
  • $r = &$a[...] on a cell element creates a counted reference box, so it survives
    buffer relocation. A nested write through a referenced element keeps the binding.
  • A by-ref foreach walks the live array, even when the body unsets elements.
  • A static property is by-ref addressable, so uksort(self::$defs, …) sorts the property
    itself.
  • Write targets evaluate the name, then the dimensions, then the value, each once.
    Previously $a[$i++] op= … and $o->{f()}[g()] = v ran side effects twice or out of
    order.

Control flow and generators

  • break/continue/goto edges, switch, match and try/catch join variable types
    like an if/else merge (a local re-typed on one path could print a pointer).
  • A loop condition that stores to its own variable (while (null !== ($i = next($i))))
    terminates.
  • yield from accepts any iterable; a finished generator answers current()/key() NULL;
    next()/send() prime an unstarted generator; an IteratorAggregate returning a
    generator iterates; an interface extending Iterator is Traversable.

Strings and scalars

  • ^ & | ~ are byte-wise on two strings; 10 < "9" compares numerically (false).
  • String offsets follow php's numeric-string rules (see Breaking for what now throws).
  • __NAMESPACE__ folds to the parse-time namespace; constant($name)/defined($name)
    resolve computed names at run time.
  • Parser: 0. floats, exponent underscores, the empty statement; !$o instanceof A and
    -2 ** 2 follow php's precedence.
  • Docblocks: int<a,b>, phpstan refined scalars, list<T>, non-empty-list<T>,
    function-level @template T and phpstan aliases lower to their base types.

Closures and callables

  • Every callable form pads omitted trailing parameters and packs variadics
    (array_map('trim') could trim with a random mask).
  • [$obj, 'm'], [C::class, 'm'] and invokable objects are invoked through an erased
    callee; bindTo on an unnamed closure reads its real layout.
  • Closure envs are reference-counted like objects; closures in arrays are owned by the array.

Standard library

  • mbstring, ext/intl, WeakMap and WeakReference: see Highlights.
  • zlib: incremental deflate_init/deflate_add (context takeover, sync flush) and
    inflate_init/inflate_add that decodes a stream fed in any chunking.
  • New: hash('xxh128') (pure PHP), get_defined_functions(), posix_mkfifo,
    isInternal()/isUserDefined() on ReflectionFunction and ReflectionClass;
    new ReflectionFunction($name) accepts a computed name.
  • array_any/array_all/array_find/array_find_key pass ($value, $key).
  • sprintf/snprintf results are no longer truncated at 255 bytes.
  • implode() of Stringable elements calls __toString; var_export writes a NUL byte
    outside the quotes; var_dump evaluates all its arguments before printing.
  • isset, ?? and empty() on an ArrayAccess value call offsetExists.
  • SplFixedArray reads in place, throws OutOfBoundsException, and setSize trims in place.
  • parse_url replaces control bytes with _; realpath() resolves relative paths against
    a cached working directory.
  • pcntl_fork() flushes stdout first; libc int returns are sign-extended, so
    pcntl_waitpid(-1, …) loops end on Linux.
  • strtr, preg_replace(_callback), str_(i)replace, filter_var and proc_open
    accept php's array forms; is_* predicates can be called by a runtime name.
  • FFI: peek_f64/poke_f64 read and write a C double; raw-memory builtins accept a
    Ffi\Ptr|false|null value.
  • A pure data file under a composer psr-4 root (return <literal>;) is kept, so
    symfony/polyfill-mbstring's tables load. PHP_VERSION reports 8.5.11.

Performance & memory

Emitted code

  • The string hash takes 8 bytes a step: a fresh 2 KB key hashed 2x slower than php, now
    2x faster.
  • Int cells add/subtract/multiply/compare/increment inline; SplFixedArray reads in place
    (a read loop runs at 4x Zend); constant int/string array literals are immortal globals
    (no malloc, no free); hot runtime helpers have an inlined fast path.
  • Darwin: a program exports only _main instead of binding every linkonce_odr body
    through a dyld stub (php-cs-fixer @PSR12 over src/: 23.7 s to 21.7 s).
  • Dynamic new $cls, instanceof $name, dynamic property access and dynamic method calls
    go through per-module tables instead of per-site strcmp chains.

Compiler

  • Self-build front-end CPU 22.1 s to 19.5 s (full inferences in the narrowing passes 9 to 5).
  • php-cs-fixer: front end 60 s to 48 s; EmitLlvm 33 s to 22 s and 20 s to 15 s (method
    indexes); 73 s of an earlier 107 s EmitLlvm was a system('rm') per staged body.
  • Compiler IR 86.1 to 82.6 MB and 81.9 to 80.7 MB; a shared return epilogue cut IR 3.7%
    and clang CPU 2.3%. Dead-stripping the manifest build: compiler 14.4 to 13.0 MB.
  • Applications with 128 MB+ of staged IR are split by default: php-cs-fixer's single
    clang -O2 never finished (>18 min, >5 GB); split, ~2 min at 2.2 GB. Parts compile at the
    host's concurrency (23 parts: 31 s to 6.4 s). The part object cache is on by default,
    bounded by MANTICORE_OBJ_CACHE_MAX_MB (default 2048), and parts stay byte-identical
    across unrelated edits.

Leaks fixed

  • Reference boxes are counted (16-160 B per call before, flat at 400k now); escaped
    closures are released (182 MB at 400k escapes before, flat now).
  • No longer leaking: out-parameters ($matches of every preg_match), omitted by-ref
    defaults, by-ref whole stores, static-property overwrites, literal arguments to
    methods/constructors, generator yielded values, getIterator() results on every loop
    exit, gzdeflate/gzinflate buffers and deflate contexts. php-cs-fixer no longer keeps a
    Tokens graph per file.
  • Array elements die once, with their buffer. The cycle collector drops a buffered root the
    moment its count reaches zero, and its threshold adapts as php's does.

Platforms & tooling

  • Memory ABI v14 ships (v9 counted reference boxes, v10 closure env shape and object
    compare view, v11 closure array elements, v12 method visibility, v14 counted BIGINT
    boxes). An ABI bump needs no cold seed: a plain bin/build crosses it.
  • Linux arm64 with a heap above bit 47: a pointer or id carried through the 48-bit inline
    int cell broke there; fixed.
  • glibc floor is 2.34, measured from a released binary's symbols (readelf -V).
  • libxml2 is bound strongly, so an --as-needed link (Alpine, Ubuntu) no longer drops it
    and sends simplexml/dom calls to address 0.
  • CI runs the suite per push on Linux arm64/amd64, Alpine and macOS; gate.yml adds
    difftest; doc-only pushes skip CI; the gate judges a self-built compiler. musl gets a
    <name>.musl.out expected-output tier; intl tests print only ICU-version-stable data.
  • Experimental native test runner (tests/aot/runner) with its own CI row.
  • Analyzer: async.blocking-io warns only on calls the pool does not cover (such as
    copy); glob() no longer warns.
  • Diagnostics: MANTICORE_FRAME_POINTERS=1 tags every body (full sample/lldb stacks);
    MANTICORE_IR_CENSUS=1; MANTICORE_BT_TOP_SKIP=N.

Fixes

  • (string) of a user object threw "Object of class could not be converted to string"
    once the stdlib emitted the helper; the helper is now module-local (745fb7f).
  • Destructors again run when the last owner goes, even when the stdlib's collector-less
    helper copy won the link.
  • An async/ws/http program no longer crashes the compiler on Linux arm64.
  • An x86_64 -O2 compiler crash on every program naming Http\ is fixed (variable slots
    start as null).
  • array_merge_recursive and similar nested-array walks no longer print garbage or SIGSEGV.
  • clone of an object with a string-keyed nested array property no longer loops forever.
  • A string offset write through a cell local no longer SIGBUSes; an ordering compare of a
    cell against a string no longer answers false for every byte.
  • ++/-- on a ?int/mixed cell local uses runtime inc/dec; null++ is 1.
  • A cell passed to an int parameter coerces like php instead of reading a string's address.
  • switch and if/else keep the arms' locals; an if-merge of null with a pointer keeps the
    pointer type instead of boxing int(0).
  • An omitted ?int $x = null default on a virtual call no longer reads as float(0).
  • By-ref params always take an address (Preg::match($re, $s) without $matches crashed);
    scalar by-ref params (?int &$count) are re-boxed by type after the call.
  • An array of Ffi\Ptr no longer free()s blocks the program still owns; a raw int[]
    word that looked like a reference tag is no longer dereferenced.
  • implode() over cell arrays no longer corrupts the heap.
  • Traits: an abstract trait method no longer shadows an inherited body; mixed-in trait
    properties carry the same per-slot facts as declared ones.
  • method_exists($this, 'm') in an abstract parent sees the runtime subclass; late static
    binding in a synthesised constructor names the subclass.
  • TLS reads and writes retry correctly, and TLS writes get a per-park timeout budget.
  • WebSocket: an empty message is never compressed; a non-UTF-8 close reason is refused with
    ValueError; only the owning task reads a Close answer.

Breaking / behaviour changes

  • Division/modulo by zero, intdiv(PHP_INT_MIN, -1) and a negative shift throw
    DivisionByZeroError / ArithmeticError instead of returning a value.
  • A null, bool, float or non-numeric string offset throws where Zend warns ("String offset
    cast occurred", "Illegal string offset"); an array or object offset throws TypeError.
  • (string) of an object without __toString throws; constant() of an unknown name
    throws "Undefined constant"; Transliterator with an id it cannot open throws
    IntlException where Zend warns.
  • $o->$m() enforces visibility (goes to __call or throws).
  • Property defaults are applied at allocation; a second __construct() call no longer
    resets properties.
  • file_get_contents() on a FIFO reads to EOF.
  • Under async(), covered file and DNS calls run on the offload pool; a task parked on a
    pooled call on a handle it already holds is not cancellable until that call returns.
    Async timers ignore wall-clock jumps.
  • async.blocking-io no longer warns on pooled file functions or glob().
  • The glibc floor rises from the documented 2.33 to 2.34.
  • Memory ABI v14: lib/*.o built by v0.11.0 must be rebuilt (bin/build does it).

Full log: v0.11.0...v0.12.0