Skip to content

MMD-DEV 1.0.0

Choose a tag to compare

@MMDBadCoder MMDBadCoder released this 24 Aug 14:58
· 86 commits to main since this release

Cloud development machines, billed by the hour in Toman.

Every customer gets what feels like their own Ubuntu server — root access, apt, Docker, AI coding tools preinstalled — reachable from a Persian web dashboard, and able to power fully off to near-zero cost without losing a byte. The whole product runs on one host.

This is 1.0: it is live and serving paying customers.


Why it is built the way it is

The host has no hardware virtualization — no /dev/kvm, no vmx/svm. That rules out every VM-based design on performance grounds: QEMU falls back to software emulation, and Firecracker, Kata, Cloud Hypervisor and incus launch --vm all require KVM.

Unprivileged Incus system containers on ZFS is what remains, and it turns out to satisfy the requirements well. Measured on this host, CPU and memory throughput inside a workspace are within noise of the host itself:

host workspace
CPU (sysbench, 1 thread) 4791 ev/s 5154 ev/s 107.6%
Memory (sysbench) 8211 MiB/s 8264 MiB/s 100.6%

The trade-off is stated rather than hidden: containers share the host kernel, so a kernel privilege-escalation bug crosses a boundary a hypervisor would resist. Admin approval of every signup is the compensating control, and it is load-bearing.

What a customer gets

  • A real machine. Root, apt install anything, docker run anything.
  • Power off to zero. No CPU, no RAM, no charge for either — while every file, package, config edit and Docker volume survives untouched.
  • Ways in: browser terminal, SSH with managed public keys, a full XFCE desktop over RDP, and a file manager with editing, upload, download and recursive zip.
  • Two permanent addresses. SSH and RDP ports reserved for the life of the account, so a saved config keeps working.
  • AI tools already signed in. One click installs Claude Code and carries the platform's sign-in across.
  • Publish a port so what they build stays reachable.
  • Factory reset behind a confirmation that requires three separate things, one of which is the account password.
  • Honest billing and support tickets built in.

Entirely Persian and right-to-left. Established technical terms stay Latin, because translating "Docker" helps nobody.

Billing

Disk bills every hour in every state, because a stopped workspace still holds its reservation. CPU and memory bill only while on, as a reservation component plus measured usage. Settlement is in arrears, and before each hour the balance must cover that hour at full capacity or the machine will not start. At zero credit the machine is archived, restorable for 30 days, then deleted.

Money is integer micro-Toman throughout — floats drift and the ledger stops reconciling. Charges are idempotent on (workspace_id, period_start, kind), so a worker restart cannot double-charge a real customer.

Security shape

The internet-facing service holds a restricted Incus certificate. Incus itself — not application logic — refuses it privileged containers, host-path disks and custom idmaps, so a compromise of the web app cannot reach the host. Everything genuinely privileged goes through a root daemon with no network listener, a SO_PEERCRED check and a fixed verb allowlist.

From inside a workspace, the host, the Incus API, cloud metadata, the provider LAN, RFC1918 and every other tenant are unreachable; internet and DNS work.

Known and accepted risks are written down in SECURITY.md rather than left to be discovered.

Verified, not asserted

  • 274 backend tests and 34 interface tests, needing no infrastructure
  • Verification suites that inspect the running host rather than the source
  • 4 busy threads on a 1-core tier deliver exactly 1.00 cores
  • A 2 GiB allocation against a 1 GiB tier stays capped at 809 MiB resident
  • Powered off: cgroup gone, no processes, Incus reports 0 bytes memory
  • Across a power cycle: packages, files, /etc edits, Docker images, volumes and containers all survive
  • Inside, nproc = 1 and free = 1024 MiB while the host is 4 cores / 7936 MiB

Bugs fixed on the way here

Each was found in production, several reported by customers:

  • An SSH lockout caused by the hardening script disabling password authentication when the only key present was the provider's
  • Two workspaces could not run at once — Incus registers DNS names per network, not per project
  • Docker's leftover nftables rules silently killed all workspace egress while DNS kept working
  • apt install firefox failed and wedged dpkg, because Ubuntu ships a stub that installs a snap and snaps cannot run in an unprivileged container
  • The reserved SSH and RDP ports were never actually allocated, so new customers saw blank addresses
  • A slow stop raised ReadTimeout and parked the workspace in error, which nothing reconciled
  • The browser terminal clipped its own last line, because the fit addon measured the border-box height
  • English text reached the Persian dashboard, saying "credits" where the product charges Toman
  • The file manager displayed //home/dev

Documentation

Known gaps

Stated so nobody mistakes them for oversights: single host with no HA and no off-host backup; Docker volumes are sparse zvols with no reservation; the AI feature shares one Claude subscription, which any customer with root can read; no SMTP, so notices are in-dashboard only; self-signed TLS until a domain exists.