Skip to content

Vyomi v2.0.8

Choose a tag to compare

@github-actions github-actions released this 20 Jun 20:51
· 11 commits to main since this release

Lightweight, progressive startup — the console is usable in ~30–60 s instead of 5–10 min. A fresh vyomi up previously built the simulator image and pulled the entire ~3.5 GB, 13-container backend stack before the user could reach anything (the two GCP-cloud-CLI emulators alone are ~1.4 GB). Now the launcher brings up only the console + HTTPS proxy first (Wave 1), then streams the heavy backends in the background (Wave 2) behind the existing "Appliance is getting ready" progress bar. License activation + read-only browsing work immediately; real service launches are gated until the backends are ready.

Added

  • Object-store upload / list / delete in the Azure & GCP consoles (parity with S3). The Azure Storage accounts → Containers view and the GCP Cloud Storage → bucket → Objects view are now real object browsers with an Upload button (multipart file → real bytes), a blob/object list, and per-item Delete (plus Delete-container for Azure). Azure: new /api/azure/storage/{account}/containers[/{container}/blobs[/{blob}]] routes in core/azure_dataplane.py + the blob browser in static/azure-console.html. GCP: new api_gcp_storage_upload_object (multipart → fake-gcs real bytes; the old JSON create_object path sent no uploadType and 400'd) wired in providers/gcp_routes.py, uploadObject/deleteObject in providers/gcp_catalog.py, the storage Objects sub-blade promoted from stub to a live renderer (core/gcp_subblades.py + static/gcp-console.html).

  • Per-cloud handler isolation for native object-store SDKs. Azure Blob requests carry an x-ms-version header that S3 never sends; a new azure_blob_dispatch_middleware (core/middleware.py, outermost) uses it to route native azure-storage-blob traffic onto the Azure Blob handler before the S3 catch-all can claim it — fixing the prior NoSuchBucket (S3-shaped) errors leaking onto Azure clients. Each cloud now owns its handler (and its errors) end-to-end.

  • Cross-cloud native-SDK integration sample (samples/java/cloud-probe). A Spring Boot service that drives the native AWS (SDK v2), GCP (google-cloud-*) and Azure (azure-storage-blob / azure-cosmos) SDKs against the appliance — GET /object/{cloud}?bucket&key[&account] reads an object a console uploaded, GET /probe/{cloud} runs a full object-store + NoSQL lifecycle. Used to confirm console writes are readable by unmodified cloud SDKs.

  • Progressive (lazy) two-wave startup. docker-compose.appliance.yml: the simulator now pulls the prebuilt published image (vyomi/appliance:<tag>, with build: . kept as a dev fallback) instead of building from source on every install, and no longer depends_on the 10 backend containers — it serves the console with just its SQLite state and connects to backends lazily. The launcher (scripts/cloud-learn appliance_exec_launcher) brings up Wave 1 = simulator + caddy (fast), then kicks off Wave 2 = the 10 backends + cloudsim detached via systemd-run; the launch health-check now fires READY on the runtime bridge + simulator only (CloudSim moved to Wave 2). Reuses the existing core/appliance_readiness probe + GET /api/runtime/readiness + the static/clouds.html "getting ready" progress banner.

  • Service-launch readiness gate. New _require_appliance_ready() (server.py, mirrors _disk_preflight) returns 503 appliance_not_ready with ready_pct + pending while backends are still downloading; bypass via CLOUDLEARN_READINESS_GATE=disabled, auto-off outside appliance distribution mode, fail-open on any probe error. Backed by core/appliance_readiness.probe_all_cached(). (In progress: wired into RDS create so far; remaining create endpoints + the frontend Create-button disable + making the readiness banner global across all console views are still to come.)

Changed

  • The "Appliance is getting ready" strip is now a persistent backend-status / diagnostics bar. Previously it polled /api/runtime/readiness, showed weighted pull progress, then hid itself once ready. It now stays visible at all times: while backends cold-start it shows the same blue progress bar, and once every service is up it flips to a green "Appliance is Ready — all N services running" bar. The Details panel (collapsed by default) lists every backend with a live status bullet (green = up) so users always have an at-a-glance view of the behind-the-scenes stack for diagnostics. It keeps polling after ready (slow 20s cadence) so the live status stays accurate. Extracted into a shared, self-mounting module (static/readiness-strip.js) and shown on both the launch page (/, static/pricing.html) and the workspaces page (/clouds, static/clouds.html) — previously it existed only on the workspaces page.
  • Azure Blob is now backed by Azurite — the official open-source emulator — instead of an in-process dict. Object bytes were the only one of the three stores held purely in memory (S3 already uses MinIO, GCS uses fake-gcs-server), so they vanished on every simulator restart. New core/azure_blob_store.py proxies the Azure Blob data plane to an azurite container (added to docker-compose.appliance.yml with a persistent volume; azure-storage-blob added to requirements.txt); core/azure_dataplane.py now routes put/get/list/delete + the native Blob REST surface through a backend-aware layer (Azurite when CLOUDLEARN_AZURITE_URL is set, in-process fallback otherwise). Verified end-to-end: a console upload lands in Azurite, the native azure-storage-blob SDK reads it back byte-for-byte, and it survives a simulator restart.

Fixed

  • Caddy crashed on first launch with "not a directory" — caught in pre-release testing. Wave 1 starts caddy immediately, but the launcher provisioned the Caddyfile + mkcert cert in a later phase, so docker auto-created /etc/vyomi/Caddyfile as a directory and the file bind-mount failed (the old flow only worked by accident because caddy's depends_on simulator → backends chain delayed it until after TLS provisioning). Fixed by moving ensure_vyomi_tls_cert + push_tls_into_vm before the stack in both the up and restart paths, and seeding /etc/vyomi/Caddyfile as a file (repairing a prior failed run that left it as a directory) before the Wave-1 docker compose up.
  • Native S3 PutObject from AWS SDK v2 no longer corrupts objects (aws-chunked decoding). Modern AWS SDKs stream PutObject/UploadPart with Content-Encoding: aws-chunked + x-amz-content-sha256: STREAMING-…, wrapping the body in chunk-size/signature framing with trailing checksums. The S3 handler stored that framed payload verbatim, so GET returned the framed bytes and the SDK's flexible-checksum validation failed ("Data read has a different checksum than expected"). routes/aws_s3.py now strips the chunk framing (_decode_aws_chunked) on PutObject + UploadPart. Full native S3 lifecycle (put/get/head/list/delete) now round-trips green.
  • Native GCS list no longer loops forever (nextPageToken omitted when complete). The GCS object/bucket list responses emitted "nextPageToken": ""; real GCS omits the field when the listing is done, and the google-cloud-storage SDKs treat a present (even empty) token as "another page exists" → they re-requested the same page until the sim rate-limited them (429), surfacing as IllegalArgumentException: key code and failed list/delete. providers/gcp_services.py now omits the empty token.
  • CLOUDLEARN_TIER_ENFORCE=0 now also disables per-resource quantity caps. The flag previously bypassed only the service-lock middleware, so quantity caps (e.g. free tier = 1 bucket) still fired with enforcement "disabled" — surprising for dev/test/probe runs. core/app_context.enforce_quantity_cap now honors the same escape hatch.
  • cloud-probe Firestore native SDK now reaches the emulator (verified end-to-end). Two stacked bugs kept the native google-cloud-firestore Java SDK off the emulator: (1) ProbeEnv derived the Firestore host from CLOUDPROBE_ENDPOINT (caddy/gateway, which don't expose :8080) → NoRouteToHostException; now defaults to the emulator service name cloudlearn-firestore:8080. (2) FirestoreOptions.setEmulatorHost() flips the channel to plaintext but leaves the endpoint at production firestore.googleapis.com:443 — the SDK fired a plaintext h2c preface at a TLS port (INTERNAL: http2 exception … 1503010002, confirmed by packet capture); GcpProbe.firestore() now pins the endpoint + usePlaintext() via InstantiatingGrpcChannelProvider. Full lifecycle (set/get/update/query/delete) verified green against the official cloud-firestore-emulator 1.21.0. Completes the trio: DynamoDB + Cosmos + Firestore are all natively SDK-compatible.

Build

  • scripts/build-release.sh aligned with the deb/rpm file lists. The .tar.gz release artifact previously omitted routes/providers/packs/setup_cython.py/docker-compose.yml/.env.example, while packaging/debian/build-deb.sh + packaging/rpm/cloud-learn.spec already bundle them (fixed in v2.0.7). Harmless when the simulator image is pulled, but the build-from-source fallback (Dockerfile COPY routes …) + the launcher source-sync need the complete set. Now every artifact (deb · rpm · snap · scoop · tarball) bundles a consistent file set.

Artifacts

  • SHA256SUMS
  • cloud-learn-0.1.0.tar.gz
  • cloud-learn-2.0.8-1.noarch.rpm
  • cloud-learn-2.0.8.tar.gz
  • cloud-learn_2.0.8_all.deb

Docker image: docker pull vyomi/appliance:2.0.8
Install: curl -fsSL https://raw.githubusercontent.com/vyomi-cloud/appliance/main/install.sh | bash
SHA256 checksums: see SHA256SUMS in attached artifacts.