Skip to content

v0.0.9 - Engine-agnostic file IO, import auto-load, and output capture

Latest
Latest

Choose a tag to compare

View all tags
@eddmpython eddmpython released this 13 Jul 14:25
v0.0.9
6e551e2

Three engine-agnostic Runtime capabilities so consumers (dartlab, codaro, xlpod) can drop rt.raw (Pyodide-specific access) and run purely on the Runtime plus its capabilities. Non-breaking: new methods and one new capability only, no existing surface changes.

What's new

  • Runtime.fs - engine-agnostic general file IO, always-on like Runtime.memory: writeFile / readFile (utf8 or binary) / mkdir / mkdirTree / readdir / stat / exists / unlink / rmdir. Mutations bump the execution-boundary counter (reactive guard); reads do not. Persistence (OPFS) stays mountHome - this is the file-op layer over the mounted FS, not a new VFS.
  • Runtime.loadPackagesFromImports(code) - scan a cell's import statements and auto-load the required C-extension wheels (numpy / pandas / ...), no explicit package list.
  • Runtime.setStdout(handler) / setStderr(handler) - capture execution output with a mutable sink for per-cell live streaming; pass null to restore the default.

All three delegate to the EngineContract (this._engine); unsupported engines degrade honestly (no-op or explicit error).

Verification

Real-browser probes (engineContract campaign): fsProbe 10/10 (including a cross-check that Python open() and rt.fs share the same filesystem), outputCaptureProbe 5/5 (mid-cell handler swap stays isolated), loadImportsProbe 3/3 (numpy auto-loaded from an import scan in 995ms). Structure gate 517/0, browser runtime gate 40/40.

Migration notes (swapping raw for rt.*)

These are behavioral differences, not bugs. Check them before swapping call sites:

  1. rt.fs.readFile(path) returns a Uint8Array (binary) by default. For text pass { encoding: "utf8" }. If your raw.FS.readFile calls already passed {encoding:"utf8"}, keep it.
  2. rt.fs.stat(path) returns a normalized minimal shape { size, isDir, isFile, mtimeMs }, not the full emscripten stat (no raw mode bits / uid / gid / atime / ino). Use .isDir in place of FS.isDir(FS.stat(p).mode). mtimeMs is best-effort (may be null).
  3. rt.fs mutations (writeFile/mkdir/mkdirTree/unlink/rmdir) bump the execution-boundary counter (execSeq); raw.FS.* did not. If you use ReactiveController, file writes now count as execution boundaries. This is intended (FS mutations are real state changes), but it changes checkpoint boundary counts versus raw.
  4. rt.setStdout(handler) is a mutable batched sink (fires on newline, Pyodide batched semantics). There is no unbatched / char-level or auto-scoped variant - you set it and reset it (null) per cell yourself.
  5. rt.fs.readdir(path) already filters . and .. - do not filter again.

Not included (out of scope this release)

  1. WASI parity is not verified. These three run on Pyodide today; on non-Pyodide engines they degrade honestly (no-op / error). "Runs unchanged on WASI" is a contract goal, not a measured fact yet. Do not assume an engine swap keeps fs / imports / stdout working until the WASI engine is wired to the contract and measured.
  2. The FS surface is the 8 ops only (writeFile / readFile / mkdir / mkdirTree / readdir / stat / exists / unlink / rmdir). No chmod / rename / symlink / truncate / copyFile / watch. Ask if you need more.
  3. loadPackagesFromImports uses Pyodide's package resolver (its package index), not a general PyPI resolver. Imports of packages outside that index will not auto-load (same as before).
  4. DeviceFs still uses raw.FS internally for device registration (registerDevice / makedev / mkdev) - a separate engine seam, intentionally not migrated. It does not affect your consumption (you use rt.fs); noted for completeness.
  5. No checkpoint adapter was built (per your request - you adapt to ReactiveController via an id / savedSP adapter; the pyproc side is unchanged).
  6. Your call sites (P4) are yours. Nothing in dartlab was touched; the raw-to-capability mapping is in the phasing doc. Success criterion ("zero raw references") completes on your side.

Consuming

Install from npm and pin the exact version:

"pyproc": "0.0.9"

Pyodide stays at v314.0.2 (CPython 3.14).

한국어

소비자(dartlab/codaro/xlpod)가 rt.raw(Pyodide 특정 접근)를 버리고 Runtime + 능력만으로 돌게 하는 엔진-무관 능력 3건. 비브레이킹(새 메서드/능력만, 기존 표면 무변경).

  • Runtime.fs - 엔진-무관 일반 파일 IO, Runtime.memory처럼 상시: writeFile/readFile(utf8·binary)/mkdir/mkdirTree/readdir/stat/exists/unlink/rmdir. 변이는 실행 경계 카운터를 올린다(리액티브 가드), 읽기는 불변. 영속(OPFS)은 mountHome, 이건 그 위 파일-op 레이어(새 VFS 아님).
  • Runtime.loadPackagesFromImports(code) - 셀 import 문을 스캔해 필요한 C확장 휠 자동 로드(명시 목록 없이).
  • Runtime.setStdout(handler) / setStderr(handler) - 가변 싱크로 실행 출력 캡처(셀별 라이브 스트리밍), null로 기본 복원.

전부 EngineContract에 위임, 미지원 엔진은 정직한 degradation(no-op/에러). 실 브라우저 검증: fsProbe 10/10(파이썬 open() <-> rt.fs 동일 FS 교차), outputCaptureProbe 5/5(셀 도중 핸들러 교체 격리), loadImportsProbe 3/3(numpy가 import 스캔만으로 995ms 자동 로드). 구조 게이트 517/0, 브라우저 런타임 게이트 40/40.

이관 주의 (raw -> rt.* 교체 시 동작 차이)

버그가 아니라 동작 차이다. 교체 전에 확인:

  1. rt.fs.readFile(path)는 기본이 Uint8Array(binary). 문자열은 { encoding: "utf8" }. raw에서 이미 {encoding:"utf8"}를 줬으면 유지.
  2. rt.fs.stat(path)는 정규화된 최소 형태 { size, isDir, isFile, mtimeMs }(전체 emscripten stat 아님 = mode 비트/uid/gid/atime/ino 없음). FS.isDir(FS.stat(p).mode) 대신 .isDir. mtimeMs는 best-effort(null 가능).
  3. rt.fs 변이(write/mkdir/mkdirTree/unlink/rmdir)는 execSeq를 올린다(raw.FS는 안 올렸다). ReactiveController를 쓰면 파일 쓰기가 실행 경계로 잡힌다(의도된 것: FS 변이는 진짜 상태 변화. 단 체크포인트 경계 카운트가 raw와 달라진다).
  4. rt.setStdout(handler)는 가변 batched 싱크(개행 경계 flush, Pyodide batched). unbatched/문자 단위나 자동 스코프 변형은 없다 = 셀마다 직접 set/reset(null).
  5. rt.fs.readdir(path)는 ./..를 이미 필터한다(재필터 금지).

이번 릴리즈 범위 밖 (안 한 것)

  1. WASI parity 미검증. 이 3건은 오늘 Pyodide에서 실동 + 비Pyodide는 정직한 degradation(no-op/에러). "WASI에서도 그대로"는 계약 목표지 실측 아님. WASI 엔진이 계약에 배선되고 실측될 때까지 엔진 교체로 fs/imports/stdout이 그대로 돈다고 가정하지 말 것.
  2. FS는 8op만(writeFile/readFile/mkdir/mkdirTree/readdir/stat/exists/unlink/rmdir). chmod/rename/symlink/truncate/copyFile/watch 없음. 필요하면 요청.
  3. loadPackagesFromImports는 Pyodide 리졸버(그 패키지 인덱스) = 일반 PyPI 리졸버 아님. 인덱스 밖 패키지 import는 자동 로드 안 됨(이전과 동일).
  4. DeviceFs는 내부적으로 raw.FS 유지(장치 등록 registerDevice/makedev/mkdev = 별개 엔진 seam, 의도적 비이관). 소비에는 무영향(당신은 rt.fs 사용), 완전성 위해 명시.
  5. 체크포인트 어댑터 미제작(요청대로 = ReactiveController에 id/savedSP 어댑터로 채택, pyproc 측 무변경).
  6. 당신 콜사이트(P4)는 당신 몫. dartlab은 손 안 댔다. raw -> 능력 매핑은 phasing 문서. 성공 기준("raw 0 참조")은 소비자 측에서 완성.

소비: npm에서 정확 버전으로 핀 "pyproc": "0.0.9". Pyodide는 v314.0.2(CPython 3.14) 유지.

Assets 2
Loading