Skip to content

Add io/mmap state machine for tensor-backed model loading #61

@gabewillen

Description

@gabewillen

Summary

Add an io/mmap state machine under src/emel/io that provides a memory-mapped tensor-loading strategy beneath tensor-owned residency management.

This issue should land after the architectural seams in #59 and #60 are in place.

Motivation

Memory mapping is a distinct loading strategy with different behavior than staged reads or explicit copies:

  • low-copy / low-setup residency when the platform and file layout allow it
  • different lifetime and eviction semantics than copy-based loading
  • different failure modes and portability constraints than explicit read-based strategies

It should be modeled as its own strategy actor, not embedded inline into tensor or loader orchestration.

Scope

  • Add src/emel/io/... machine(s) for mmap-backed tensor loading.
  • Define the request/result events needed for tensor-owned loading to request a mapped tensor buffer.
  • Handle mmap-specific success/error behavior explicitly.
  • Integrate with the tensor-to-io boundary introduced by Add emel/io module and tensor-to-io orchestration boundary #60.

Hard Constraints

  • This issue implements the mmap strategy only.
  • Tensor lifecycle/residency semantics remain owned by model/tensor.
  • Platform-specific mapping details must stay behind the I/O abstraction boundary.
  • Do not fold read/copy or staged/chunked logic into this issue.

Acceptance Criteria

  • A dedicated mmap strategy state machine exists under src/emel/io.
  • Tensor-owned load flow can request mmap-backed residency through the io boundary.
  • mmap-specific errors are surfaced explicitly and deterministically.
  • Tests cover supported-path behavior and representative failure handling.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions