Skip to content

Mirage 0.0.4

Latest
Latest

Choose a tag to compare

View all tags
@zechengz zechengz released this 23 Jul 07:05
v0.0.4
b6e4317
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

We are excited to announce the release of Mirage 0.0.4 🎉🎉🎉

Mirage 0.0.4 is an architecture and depth release. Path addressing becomes a real namespace layer with first-class symlinks and attribute overlays that survive restarts; the control plane moves onto pluggable state stores (RAM, disk, Redis, S3) with CAS-guarded writes; the shell grows GNU-grade bash semantics (arithmetic, control flow, brace expansion, printf, getopts, parameter expansion); commands can be run by pluggable runtimes that read and write workspace mounts; and eight new or rebuilt backends land. Python is mypy-clean and TypeScript is tsc-clean, both enforced as CI gates. Python and TypeScript are kept mirrored throughout.

pip install mirage-ai==0.0.4 · npm install @struktoai/mirage-core@0.0.4 (and -node, -browser, -server, -cli, -agents)

Highlights

  • The namespace is a real addressing layer. A Namespace facade consolidates mount resolution, symlink following, and attribute overlays behind one surface, and the Workspace routes dispatch, apply_io, and post-write invalidation through it instead of holding a dispatcher directly (#420, #424, #426). Symlinks are first-class namespace ops in both languages: ln -s creates a real link (not a copy), cat/ls/glob follow on read, rm/mv act on the link entry, cd honors -L/-P with ELOOP on cycles, and the link table is captured into snapshots (#421). FUSE and disk attributes route through the same layer, so a real inode base carries a residual overlay (#505, #509).
  • A persistent, CAS-guarded control plane. Namespace, observer, and session state unify under one WorkspaceStateStore (#523) with backends for disk (lockfile CAS), Redis, and S3 (#535, #544). Sessions survive restarts and are shared across processes (#521), with dirty-tracked flushes and generation CAS on both the session and workspace meta records (#530, #533). MIRAGE_HOME becomes the single root for the whole tree (#551), and state can be diffed and surgically restored per category (#552).
  • GNU-grade shell semantics. Bash arithmetic ((( )) and $(( ))), control flow, set options, and test/[[ semantics (#369, #477, #577); brace expansion (#563); full GNU printf with Python/TypeScript float parity (#580); getopts (#612); the command builtin (#595); sort -k KEYDEF grammar (#607); and three waves of agent-facing coreutils coverage — checksums, rmdir, unlink, :, type, chgrp, realpath -e, ln -r, rm safety flags, env, tr -C/-t, tee --output-error (#603, #613, #617, #618).
  • Pluggable runtimes that reach the filesystem. A WASI runtime runs CPython on wasmtime (#498), a node/js family runs on QuickJS in both languages (#504), and Python defaults to the monty sandbox with pyodide on TypeScript (#487). Sandboxed code reads and writes workspace mounts through interception (#508, #514), and a per-line routing ladder picks the runtime from the argument, route, entry script, or VFS (#547, #559).
  • Cross-mount commands get real strategies. STREAM, FANOUT, and RELAY run per-mount native commands and reassemble the result (#453), with a shared parse so cross-mount lines are parsed against the mount spec like single-mount ones (#474), GNU failure semantics and glob relay (#582), and du/md5/file fan-out where cp populates the read cache (#441).
  • Eight new and rebuilt backends. Google Drive read/write with a gws CLI and fake Workspace server (#549), Box (#562, #579), Dropbox subfolder mounts with a Python port (#558, #568), MongoDB GridFS with native revisions and server-side find (#566), Mem0 (#341), a Prisma-backed fake Slack Web API (#578), nested Trello and Linear CLI families with Linear documents (#581), and a Dify TypeScript backend (#601). OneDrive and SharePoint now share one Microsoft Graph drive core (#532, #534).
  • Resource change watching. A mount-scoped watch API lets an external agent service react to changes, with a Nextcloud source, attach/detach at runtime, per-root overflow collapse, and nested-mount coverage (#594, #598).
  • Provision you can trust. Field-wise combinators and default estimators by command family (#428), one NodeKind classification shared by the executor and planner so planner drift is structural rather than accidental (#431), exact glob and recursive-walk estimates with zero-cost stdin stages (#440), and cross-mount plans that sum per-mount estimates (#441).
  • One declarative integration harness. Python and TypeScript run the same shared harness against every backend (#520), with targets added for s3, onedrive, ssh, nextcloud, sharepoint, hf buckets, gapps, gmail, email, slack, gridfs, box, dropbox, databricks, and dify; the legacy per-backend scripts are retired (#539).
  • Types are a gate, not an aspiration. Python mypy went from a 441-module baseline to zero and is enforced (#500 through #522), TypeScript is tsc --noEmit clean with noImplicitOverride, noFallthroughCasesInSwitch, and knip dead-code checks in pre-commit (#463, #465).

Breaking Changes

  • PathSpec is pure-virtual. The prefix field and the strip_prefix/key properties are gone; a required resource_path is stamped by the mount at dispatch, with strip_mount/mount_key/rekey/mount_prefix_of in utils/key_prefix (#424). Core is PathSpec-only and the str coercion shims are removed (#496).
  • Per-backend op manifests are renamed to io.py/io.ts exporting IO, and the VFS/FUSE op layer is generated from each backend's CommandIO table rather than hand-written (#542, #545, #553).
  • The Workspace no longer holds a dispatcher directly — resolution, dispatch, and invalidation go through Namespace (#420).
  • Session mount grants become a modes vocabulary (read/write/exec), with tuple mounts in TS core (#436, #499).
  • index is non-optional; a NULL_INDEX no-op replaces None (#501).
  • Python defaults to the monty sandbox, TypeScript to pyodide, for python3 execution (#487).
  • FUSE sizes changed shape. The 100 MiB sentinel is gone in favor of direct_io (#491), and FileStat.size is render-derived or None — never a storage-side or source-side number (#488, #489).
  • Workspace and session ids are UUIDv7, and the reserved defaults are dropped (#526).
  • MIRAGE_HOME is the single root and the TypeScript daemon defaults to the disk store (#551).
  • Per-backend glob boilerplate is deleted in favor of the shared resolve_glob_with walk and a custom-backend SDK (#478, #540).

Features

  • box: Python + TypeScript Box backend with full read/write and an integ target (#562); special files served raw plus grep/rg search push-down (#579).
  • coreutils: env builtin, tr -C/-t, tee --output-error (#618); Tier 2 chgrp, realpath -e, ln -r, rm safety flags (#617); Tier 1 flag support (#613); Wave 1 checksums, rmdir, unlink, :, type (#603).
  • daemon: config.toml path settings and a mirage config command (#473).
  • dropbox: subfolder mounts via rootPath, a Python port, and battery targets (#558); grep/rg search push-down via files/search_v2 (#568).
  • gdrive: read/write Drive backend, gws CLI, fake Workspace server, integ targets (#549).
  • google: optional api_base override for GoogleConfig (#596).
  • gridfs: MongoDB GridFS backend with native revisions and server-side find (#566).
  • mem0: read-only Mem0 Memory resource, Python (#341).
  • namespace: symlinks as first-class namespace ops (#421); orphaned overlays reconciled on remote delete (#515).
  • nextcloud: server-side find via the Files Search API, with partial predicates and fallback (#475).
  • printf: full GNU printf builtin with Python/TypeScript float parity (#580).
  • provision: field-wise combinators and default estimators by command family (#428).
  • runtime: WASI runtime (CPython on wasmtime) with python3 safeguards following the script path in TS (#498); node/js command family on QuickJS (#504); per-line routing ladder (#547); run_line for runtimes that run whole lines (#559).
  • session: SessionStore seam so sessions survive restarts and are shared across processes (#521); dirty-tracked flush with generation CAS (#530).
  • shell: builtin specs — xargs batching, timeout enforcement, GNU echo/read/shift/return (#468); brace expansion in both hosts (#563); GNU control-flow, set-option, builtin, and test/[[ semantics with 147 integ cases (#577); command builtin (#595); getopts (#612).
  • slack: Prisma-backed fake Slack Web API and integ battery target (#578).
  • sort: full GNU -k KEYDEF grammar (#607).
  • store: generation CAS on the workspace meta record (#533); S3 backend for the sessions+meta group (#535); disk backend with lockfile CAS (#544).
  • trello, linear: nested CLI command families, Linear documents, integ coverage (#581).
  • ts: JS sandbox reads and writes workspace mounts (#514).
  • version: content-pure commit trees that strip cache and sessions (#536); whole-world commits via the .mirage/ control-plane subtree (#537).
  • watch: resource change watching with a Nextcloud source (#594); attach/detach runtime surface, per-root overflow collapse, nested-mount coverage (#598).
  • Dify TypeScript backend, Databricks JSON harness migration, and a systemic mkdir -p cache fix (#601).
  • Codex and Grok Build integrations (#575), a native OpenCode plugin with stale-write protection (#576), and scoped filesystem integration coverage (#587).

Bugfixes

  • cache: backend fingerprints thread into apply_io so ALWAYS mode stops evicting non-MD5 backends (#434); streamed reads background-drain into the Redis file cache (#439); the drain budget is clamped to the cache limit (#443).
  • crossmount: the GNU strerror suffix is appended on the not-found branch (#447); GNU failure semantics, relay glob, and dispatcher invalidation (#582).
  • daemon: the pid file path is configurable via MIRAGE_HOME / MIRAGE_PID_FILE (#466).
  • expand: per-position spec word kinds and a consumer-first word policy (#455).
  • find: walkFind reaches parity with Python walk_find and GNU — root emission, -empty, -mtime union (#565).
  • glob: mid-path segments, zero-match literals, and push-down spelling (#461).
  • onedrive: GNU overwrite semantics against real Graph conflict defaults (#531).
  • safeguard: mount-level timeouts bound the command body; python3 is guarded like any command (#495).
  • shell: relative words follow bash semantics (#456); ${var:?}/${var:=}, $$/$!, the exit builtin, and graceful unsupported constructs (#546); ${v:$o} errors loudly and quoted array-@ slices stay multi-word (#600); piped stdin forwards through bash -c on the TypeScript host (#599).
  • google: API endpoints stay fixed (#584); example command registrations corrected (#588); examples use real gws command names (#585).
  • Dify Knowledge follow-ups (#160), path.strip() calls and empty-input edge cases (#511), latent type-safety and lifecycle bugs (#528), docs logo alignment (#492).

Changes

  • Read family keeps partial output past missing operands (#464), matching GNU: partial stdout, one stderr line per failed operand, exit 1.
  • GNU alignment sweeps: EISDIR on directories and operand arity (#477); rg multi-pattern push-down on postgres/email with a GNU error shape for unsupported path-bound commands (#479); -size semantics with directories as 0, strict bounds, and round-up (#481); sed as a read-safe generic builder (#484); awk GNU semantics — default FS, multi-file NR, exit 2 (#462); text filters aligned with GNU (#556).
  • Command routing by consumer, with globs resolved exactly once (#446), an Argv argument vector unifying the expand pipeline (#444), and a factory walk fallback that drops 19 bespoke find wrappers (#482).
  • Metadata ops — chmod, chown, touch via setattr with a namespace overlay (#430) — and a namespace store so symlinks and attr overlays survive restarts (#503, #505, #509).
  • FUSE: size-unknown probes with an advisory WinFsp job (#493), an OS support matrix and Windows WinFsp setup page (#497).
  • Integ: grep/find/rg/zgrep flag coverage (#557); bash parameter-expansion, array, and quoting coverage (#560); redirect/heredoc/herestring/procsub cases (#574); unix/crossmount flag coverage with GNU shell fixes (#589); JSON cases nested as category/name/facet (#602); targets for gapps (#561), gmail and email (#564), hf buckets (#550), s3 (#524), onedrive (#529), ssh and nextcloud (#538).
  • Agent integrations upgraded (#569, #573), with native UI and stale-write protection for Pi (#571) and OpenAI file reading (#569).
  • Dead code removed across Python and TypeScript in several rounds (#423, #425, #485, #548, #554).
  • Security: high-severity Dependabot alerts and a @hono/node-server vulnerability cleared, plus code scanning alerts resolved (#614, #615, #616).
  • Dependencies: actions/cache, actions/setup-python, actions/setup-node bumped (#502, #591, #592, #593).

Contributors

Thanks to everyone who contributed to this release: @zechengz, @bytecii, @ki3nd, @rikaqu0223-arch, @sonhmai, @fix2015, @nnayz, and @dependabot.

New Contributors

  • @rikaqu0223-arch made their first contribution: cross-mount GNU strerror suffix on the not-found branch (#447)
  • @fix2015 made their first contribution: path.strip() calls and empty-input edge cases in commands (#511)
  • @bytecii made their first contribution: deleted per-backend glob boilerplate and opened a custom-backend SDK (#540)

Full Changelog: v0.0.3...v0.0.4

Contributors

fix2015, sonhmai, and 6 other contributors
Assets 2
Loading