Skip to content

Phase 3: Tier P parallel execution via girder actors #3

@zacharywhitley

Description

@zacharywhitley

Background

Design: docs/native-execution-and-parallelism.md §5, §6.
Depends on #1.

GIL-free parallelism by running many isolated CPython-WASM instances as girder
actors (each its own interpreter/GIL/memory), coordinated by message passing.

Tasks

  • CPython-WASM → girder adapter: export girder's turn-actor world
    (init/handle) and dispatch handle(msg) to a Python entrypoint
    (the offload shape adapted to the actor ABI).
  • Reproduce the parallel_mapreduce pattern (see
    ~/git/girder/crates/girder-runtime/examples/parallel_mapreduce.rs) with a
    CPython payload; spawn K actors, fan tasks, collect results.
  • Add the arrow codec for array/frame payloads.
  • Use girder SIR (Shared Immutable Regions) to broadcast large read-only
    inputs zero-copy.
  • Measure multicore scaling and the data-movement cost.

Acceptance

A CPU-bound Python map runs across N girder actors with measured speedup vs a
single instance, and a large read-only input shared via SIR.

Notes / deps

  • Sibling repo ~/git/girder (native host runtime; not browser — see design §7).
  • Requires the CPython→turn-actor adapter above as a prerequisite.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions