Skip to content

Handle Dynamo request-priority headers in Pylon #620

Description

@along-2017

Description

Dynamo schedules on x-dynamo-request-priority (i32, higher value is more important) and silently ignores values that do not parse as i32. The platform resolves a caller priority and carries it as x-priority (u32, lower value is more urgent, absent means unconfigured).

Make Pylon the single component that handles the engine-facing Dynamo headers: it derives x-dynamo-request-priority from x-priority when forwarding to the upstream inference server, and sanitizes inbound values. The gateway and Stargate stay Dynamo-agnostic, so if the Dynamo header contract changes later (names, polarity, types), only Pylon changes.

Definition of Done

  • Pylon removes any inbound x-dynamo-request-* header before forwarding upstream, so client-supplied values never reach the engine.
  • Pylon emits x-dynamo-request-priority only when the inbound x-priority header is present. Absent stays absent: the parsed default of 0 is never inverted into a maximum engine priority.
  • The mapping inverts polarity and always produces an i32-valid value (for example i32::MAX - min(x, i32::MAX)), so no configured priority is silently dropped by Dynamo.
  • Pylon logs the emitted value with the request context for per-request triage.
  • No Dynamo-specific header handling is added to the gateway or Stargate.
  • Unit tests cover the mapping boundaries (0, i32::MAX, u32::MAX), the presence rule, and the inbound strip; an integration test pins end-to-end delivery using the mock-dynamo header recording.

Resources

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions