EMRY-034: Sidecar Python socket client - #72
Merged
Conversation
emry.wire: [u32 LE len][msgpack Event] encoder matching emry-ingest::wire (adjacently-tagged, structs-as-maps). emry.socket_backend.SocketBackend streams MetricsBatch/PhaseChange/RunFinished frames to a running 'emry engine' over its Unix socket. run()'s sidecar mode uses it via EMRY_SOCKET, falling back to the JSONL backend when no engine is listening. msgpack is an optional dep (emry[socket]), imported lazily. Verified cross-language: a Python sidecar run streamed 10 batches + a phase change + finish into the Rust engine's events.jsonl. The client does NOT send RunStarted — the engine owns run identity (and the Rust Uuid msgpack form is bytes, not a string).
- SocketBackend emits a PhaseChange on the first emit too (observers track phase from PhaseChange events, not the MetricsBatch field, so the initial phase must be sent or it reads as the default). - run() warns when sidecar mode is explicitly requested but the engine is unreachable or EMRY_SOCKET is unset, before falling back to JSONL — a silent file fallback could mask a misconfigured sidecar.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #25 (EMRY-034). Branched off
main. Pure Python.What
emry.wire—[u32 LE len][msgpack Event]encoder matchingemry-ingest::wire(adjacently-tagged, structs-as-maps,valuesas[[id, val]]).emry.socket_backend.SocketBackend— streamsPhaseChange/MetricsBatch/RunFinishedframes to a runningemry engineover its Unix socket.run()sidecarmode uses it (viaEMRY_SOCKET), falling back to JSONL — with a warning — when no engine is reachable.msgpackis an optional dep (emry[socket]), imported lazily;import emryand file/embedded runs don't need it.Verification
pytestgreen: 59 passed, 1 skipped, 95.38% coverage. socketpair-based test decodes the exact frames sent.run(mode="sidecar")streamed 10 metric batches + a phase change + finish into the realemry engine, producing the expectedevents.jsonl. (This caught a real bug — the RustUuidmsgpack form is a byte array, not a string — resolved by not sendingRunStartedfrom the client, which is correct anyway: the engine owns run identity.)Review-driven fixes
PhaseChange(observers read phase fromPhaseChange, not the batch field, so a run starting in EVAL would otherwise read as TRAIN).mode="sidecar"is explicit but the engine is unreachable /EMRY_SOCKETunset.Scope
Engine-side
metrics.jsonlfrom socket events needs a name-table protocol (ids only over the wire) — a documented follow-up.