Skip to content

chore: merge main into release for v1.16.5 GA#7

Merged
catinspace-au merged 5 commits intoreleasefrom
fix/merge-to-release-v1.16.5
Mar 19, 2026
Merged

chore: merge main into release for v1.16.5 GA#7
catinspace-au merged 5 commits intoreleasefrom
fix/merge-to-release-v1.16.5

Conversation

@catinspace-au
Copy link
Copy Markdown
Collaborator

Summary

  • DfeSource — DFE platform topic naming convention (_land/_load), source-scoped consumer groups, ServiceRole enum (25 tests)
  • MemoryGuard from_envMemoryGuardConfig::from_env(prefix) and from_env_raw(prefix) for standard env var configuration
  • Tuned defaults — cgroup_headroom 0.90 → 0.85 (Rust has no GC spike), pressure_threshold stays 0.80
  • Standard env vars: {PREFIX}_MEMORY_LIMIT_BYTES, _MEMORY_PRESSURE_THRESHOLD, _MEMORY_CGROUP_HEADROOM

Test plan

  • 16 memory tests pass (including from_env defaults + parsing)
  • 25 kafka_config DfeSource tests pass
  • Merge conflicts resolved (VERSION, Cargo.toml, CHANGELOG)

semantic-release-bot and others added 5 commits March 19, 2026 06:04
## [1.16.2-dev.3](v1.16.2-dev.2...v1.16.2-dev.3) (2026-03-19)

### Bug Fixes

* add concurrent and edge-case tests for MemoryGuard ([b411d1c](b411d1c))
* add MemoryGuard — cgroup-aware memory backpressure for OOM prevention ([fba690f](fba690f))
DfeSource derives input/output topic names (_land/_load) and
consumer group IDs from a source name. ServiceRole distinguishes
Transform (source-scoped CG) from Universal (bare service CG).
Empty source on Transform role returns an error — bare CGs would
cause multiple pipelines to compete for messages. Override always
wins for operator-configured CGs.
## [1.16.2-dev.4](v1.16.2-dev.3...v1.16.2-dev.4) (2026-03-19)

### Bug Fixes

* add DfeSource convention for topic naming and consumer groups ([3b0c7da](3b0c7da))
- Add MemoryGuardConfig::from_env(prefix) for config-feature users
- Add MemoryGuardConfig::from_env_raw(prefix) for standalone use
- Standard env vars: {PREFIX}_MEMORY_LIMIT_BYTES, _PRESSURE_THRESHOLD, _CGROUP_HEADROOM
- Tune defaults: cgroup_headroom 0.90 -> 0.85 (Rust has no GC spike)
- Tune rationale: 85% cgroup + 80% pressure = 68% effective backpressure
- Aligns with OTel Collector limit_percentage:80 philosophy
- 16 memory tests pass
@catinspace-au catinspace-au merged commit 17997b2 into release Mar 19, 2026
2 checks passed
@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version 1.16.2-dev.5 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@github-actions
Copy link
Copy Markdown

🎉 This PR is included in version 1.16.5 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

catinspace-au added a commit that referenced this pull request Apr 11, 2026
Add the final review findings from the pre-merge audit:

* Python<->Rust classifier parity via shared JSON fixture
  (tests/fixtures/cel_classifier_parity.json). The matching
  dfe-engine test loads the same fixture and runs every entry
  through the Python classifier — both sides must agree on tier,
  op kind, op field, op value, and field references.

* Concurrency test: 32 tokio tasks share one Arc<TransportFilterEngine>
  and classify 1000 messages each. Also a compile-time
  assert_send_sync check for FilterRule/Action/Disposition.

* DLQ buffer drain tests covering take_filtered_dlq_entries():
  entries are exposed, draining empties the buffer, drop filters
  don't pollute the buffer, and no filters means zero overhead.

* Memmem nested-field limitation: documents the known false-positive
  case where has(field) matches a nested occurrence (needed for the
  50% performance gain on the common top-level case) and the sound
  case where JSON-escaped quotes prevent false positives inside
  string values.

* Per-transport smoke test for MemoryTransport filter wiring.

Also add docs/TRANSPORT-FILTER-FOLLOWUP.md tracking items #7-#11:
constant-time compare, log masking, pre-quoted equals fast path,
msgpack edges, and expression_text reuse on reload.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants