parser: adopt/enhance as canonical GGUF parser by extracting from corinth-canal

[rmems]

Summary

Extract and enhance reusable GGUF deserialization + MoE per-expert extraction logic from the rmems/corinth-canal experimental reference implementation into Limen-Neural/engram-parser (the canonical zero-dependency parser crate under the Limen-Neural org for modular reusable libraries).

This continues the modularization program (see LIM-9) and aligns with the existing positioning of engram-parser as the GGUF/parse + raw expert extractor base (see README Scope/Boundaries and open #5 for traits).

Sibling tracking issues (created together):

• moe: migrate GGUF parser layer to engram-parser dep (extract useful code from experimental ref impl) corinth-canal#115 (migration of parser layer in the experimental repo)
• moe/gguf: coordinate with engram-parser extraction from corinth-canal; avoid overlapping parser work cortex-tensor#8 (coordination + avoid overlap)
• Linear LIM-88 (cross-repo planning)
• Beads (corinth-canal): raulmc-v1a

Context

• engram-parser was initially refactored from logic in corinth-canal (see commit 8dd2af2), deliberately omitting routing/CUDA to stay zero-dep and focused.
• corinth-canal (rmems personal/experimental ref repo) still duplicates a full GGUF parser + mmap + dequants + adapters in src/moe/{checkpoint.rs,ggml.rs,...}.
• cortex-tensor (sibling Limen-Neural reusable lib) was also extracted from corinth-canal and has open issues Define MoE extraction traits inside this crate #5 (boundary) + parser: adopt/enhance as canonical GGUF parser by extracting from corinth-canal #7 (traits, notes reusability with engram-parser).
• Goal: extract useful code from the experimental corinth-canal into Limen-Neural org's focused library crates. corinth-canal remains the end-to-end research vehicle.
• No current GitHub/Linear/beads issue explicitly tracked this parser extraction + coordination (until these).

Related

• LIM-9 (Rust boundary matrix)
• engram-parser docs: produce engram-parser role in boundary matrix (LIM-9) + publish prep #4 (docs + LIM-9 link), Define MoE extraction traits inside this crate #5 (MoE traits)
• cortex-tensor Define MoE extraction traits inside this crate #5, parser: adopt/enhance as canonical GGUF parser by extracting from corinth-canal #7
• corinth-canal docs/MODULE_STATUS.md (parser still "reference"), docs/PROMOTION_RULES.md, AGENTS.md

Goal

Make engram-parser the single source of truth for GGUF v3 layout parsing + MoE expert weight ripping (raw bytes only). Enable future adoption by cortex-tensor (and potentially corinth-canal's higher layers).

Non-goals

• Do not move routing math, model adapters, full dequant math, Safetensors, CUDA/GPU registration, or SNN orchestration.
• Do not change corinth-canal's experimental runtime or validation behavior in this issue (that belongs to the companion migration issue).
• Keep engram-parser zero-dependency.

Acceptance criteria

• Additional dtypes from corinth-canal's ggml (e.g. IQ3_M=31 as Other or explicit opaque; verify byte_len handling).
• Port useful pure helpers where they fit zero-dep contract (e.g. ggml_type_label or equivalent; full value type consts for metadata if missing).
• MoE extraction traits defined (or advanced) inside the crate (ties directly to open Define MoE extraction traits inside this crate #5).
• Tests cover additional cases/synthetic patterns inspired by corinth-canal.
• README updated with explicit note on extraction source + cross-links to cortex-tensor coordination issue and (future) corinth migration issue; ecosystem table remains accurate.
• Crate remains zero-dep; cargo test --all-features + clippy/fmt clean.
• Issue cross-links to the new cortex-tensor coordination issue and corinth-canal migration issue.

Validation

# In engram-parser
cargo fmt -- --check
cargo clippy --all-targets --all-features -- -D warnings
cargo test --all-features

# Smoke against real GGUF (use paths from corinth-canal configs or $HOME models)
CHECKPOINT_PATH=... cargo run --example ... (or add simple bin if needed)

Suggested branch

feat/extract-gguf-parser-from-corinth

References

• Primary plan and research: this repo's session plan.md (extraction from corinth-canal)
• corinth-canal open relicense #110 (prereq for future dep from GPL crate)

Created as part of modularization follow-up to LIM-9 and engram #5.

---

Siblings (created 2026-07-01/02):

• corinth-canal#115
• cortex-tensor#8
• Linear LIM-88

[rmems]

Beads coordination issue in corinth-canal (via CLI): corinth-canal-930

[rmems]

Safetensors sibling extraction (Phase 2, parallel to this GGUF work): corinth-canal#116 (bootstrap/supporting issue), cortex-tensor#9 (coord), updated Linear LIM-88, beads raulmc-yqj.

Note (per review): no dep from corinth-canal; one-way copy of code as inspiration. New target crate will be safetensors-parser (GGUF-only scope of engram-parser preserved). See plan for details.

[rmems]

New CI/DX issues (Docker + Azure): #8 (Azure Pipelines cross-platform) and #9 (Docker workflow). These improve reliability and org consistency (mirroring corinth-canal patterns) alongside the extraction/modularization work. See the session plan.md.

[rmems]

New separate issue for Safetensors extraction (sibling/parallel to this GGUF #7, created in engram-parser per user request): #10

Tracks the Safetensors case (from corinth-canal#116 source). No dep on corinth-canal (copy from inspiration only). See plan and new #10 for details + cross ecosystem links (cortex#9 etc).