reliability audit remediation: erasure closure, durable state machines, provider budgets, and config hardening

[hwuiwon]

Checklist

• If you changed any .snap file, run cargo insta review and commit the result.

[coderabbitai]

Important

Review skipped

Too many files!

This PR contains 184 files, which is 34 over the limit of 150.

To get a review, narrow the scope:
• coderabbit review --type committed # exclude uncommitted changes
• coderabbit review --dir # limit to a subdirectory
• coderabbit review --base # compare against a closer base

Upgrade to a paid plan to raise the limit.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 99e1ea21-cbfd-4d4d-b194-4c95749e6070

📥 Commits

Reviewing files that changed from the base of the PR and between a794b5d and f509e60.

⛔ Files ignored due to path filters (1)

• Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (184)

• .agents/skills/moa-100-session-sweep/SKILL.md
• .agents/skills/moa-100-session-sweep/scripts/run_100_session_sweep.py
• SEQUENCE-DIAGRAMS.md
• crates/moa-analytics/Cargo.toml
• crates/moa-analytics/src/clickhouse_exec.rs
• crates/moa-analytics/src/compiler.rs
• crates/moa-analytics/src/error.rs
• crates/moa-analytics/src/executor.rs
• crates/moa-analytics/src/query.rs
• crates/moa-analytics/tests/clickhouse_executor_offline.rs
• crates/moa-analytics/tests/executor_statement_timeout_db.rs
• crates/moa-auth/auth0/src/auth0_provider.rs
• crates/moa-auth/auth0/src/jwks_cache.rs
• crates/moa-auth/auth0/src/oidc_provider.rs
• crates/moa-auth/authz-schema/src/lib.rs
• crates/moa-auth/authz-schema/src/schema_v1.json
• crates/moa-auth/authz-schema/src/tuple.rs
• crates/moa-auth/authz/Cargo.toml
• crates/moa-auth/authz/src/outbox.rs
• crates/moa-auth/authz/src/poller.rs
• crates/moa-auth/authz/tests/authz_db/authz_poller_db.rs
• crates/moa-auth/authz/tests/authz_db/outbox_basic_db.rs
• crates/moa-auth/providers/src/api_keys.rs
• crates/moa-auth/providers/src/contact_tokens.rs
• crates/moa-auth/providers/src/user_sessions.rs
• crates/moa-brain/assets/ner-gazetteer.txt
• crates/moa-brain/examples/replay_corpus.rs
• crates/moa-brain/src/pipeline/memory.rs
• crates/moa-brain/src/pipeline/memory/rendering.rs
• crates/moa-brain/src/retrieval/enrichment.rs
• crates/moa-brain/src/retrieval/hybrid.rs
• crates/moa-brain/src/retrieval/mod.rs
• crates/moa-brain/src/retrieval/ranking.rs
• crates/moa-brain/src/retrieval/types.rs
• crates/moa-brain/tests/brain_db_memory/hybrid_retrieval_db_memory.rs
• crates/moa-core/src/config/analytics.rs
• crates/moa-core/src/config/audit_security.rs
• crates/moa-core/src/config/auth.rs
• crates/moa-core/src/config/authz.rs
• crates/moa-core/src/config/context.rs
• crates/moa-core/src/config/database.rs
• crates/moa-core/src/config/env_overlay.rs
• crates/moa-core/src/config/learning.rs
• crates/moa-core/src/config/loader.rs
• crates/moa-core/src/config/mod.rs
• crates/moa-core/src/config/providers.rs
• crates/moa-core/src/events.rs
• crates/moa-core/src/lib.rs
• crates/moa-core/src/session_engine.rs
• crates/moa-core/src/types/tools.rs
• crates/moa-core/src/wire/privacy.rs
• crates/moa-edge/src/main.rs
• crates/moa-edge/src/routes/analytics.rs
• crates/moa-edge/src/routes/auth_accounts.rs
• crates/moa-edge/src/routes/tenant_accounts.rs
• crates/moa-edge/tests/direct_read_routes_db.rs
• crates/moa-eval/core/src/evaluators/output_match.rs
• crates/moa-eval/core/src/types.rs
• crates/moa-eval/examples/example-skill-suite.toml
• crates/moa-eval/scenarios/long_conversation/long_running_deploy_with_approval_pause_and_resume/transcript.jsonl
• crates/moa-eval/src/long_conversation/budgets.rs
• crates/moa-eval/src/long_conversation/score_card.rs
• crates/moa-eval/src/long_conversation/transcript_runner.rs
• crates/moa-eval/tests/fixtures/golden_100/01.json
• crates/moa-eval/tests/fixtures/golden_100/03.json
• crates/moa-eval/tests/long_conversation_foundation_eval.rs
• crates/moa-experiments/src/app.rs
• crates/moa-experiments/src/store.rs
• crates/moa-experiments/tests/experiment_store_db.rs
• crates/moa-hands/src/core/dispatch.rs
• crates/moa-hands/src/core/policy.rs
• crates/moa-hands/src/core/recovery.rs
• crates/moa-hands/src/core/registration.rs
• crates/moa-hands/tests/hands_offline/security_defaults.rs
• crates/moa-knowledge/Cargo.toml
• crates/moa-knowledge/src/error.rs
• crates/moa-knowledge/src/ingestion.rs
• crates/moa-knowledge/src/observability.rs
• crates/moa-knowledge/src/repository.rs
• crates/moa-knowledge/tests/knowledge_db_memory/ingestion_pipeline_db_memory.rs
• crates/moa-lineage/sink/src/fjall_journal.rs
• crates/moa-lineage/sink/src/writer.rs
• crates/moa-lineage/sink/tests/writer_db.rs
• crates/moa-loadtest/src/merge.rs
• crates/moa-memory/graph/src/store.rs
• crates/moa-memory/graph/src/write.rs
• crates/moa-memory/graph/tests/memory_graph_db_memory/write_protocol_db_memory.rs
• crates/moa-memory/ingest/src/contradiction.rs
• crates/moa-memory/ingest/src/error.rs
• crates/moa-memory/ingest/src/fast_path.rs
• crates/moa-memory/ingest/src/slow_path.rs
• crates/moa-memory/ingest/tests/contradiction_candidates_db_memory.rs
• crates/moa-memory/ingest/tests/fast_path_db_memory.rs
• crates/moa-memory/lifecycle/src/consolidate.rs
• crates/moa-memory/lifecycle/src/digest.rs
• crates/moa-memory/lifecycle/tests/memory_lifecycle_db_memory/digest_postgres_db_memory.rs
• crates/moa-memory/pii/src/erasure.rs
• crates/moa-memory/pii/tests/erasure_db_memory.rs
• crates/moa-memory/vector/src/backend.rs
• crates/moa-memory/vector/src/lib.rs
• crates/moa-memory/vector/src/sync.rs
• crates/moa-memory/vector/tests/memory_vector_db_memory/vector_sync_outbox_db_memory.rs
• crates/moa-messaging/src/slack.rs
• crates/moa-migrations/migrations/postgres/V000101__auth_baseline.sql
• crates/moa-migrations/migrations/postgres/V000322__workspace_authz_backfill.sql
• crates/moa-migrations/migrations/postgres/V000328__privacy_erasure_jobs.sql
• crates/moa-migrations/migrations/postgres/V000329__vector_sync_outbox_dead_letter.sql
• crates/moa-migrations/migrations/postgres/V000330__analytics_mv_refresh_state.sql
• crates/moa-migrations/src/lib.rs
• crates/moa-ocsf/Cargo.toml
• crates/moa-ocsf/src/audit_sink.rs
• crates/moa-orchestrator/src/action_reviews/app.rs
• crates/moa-orchestrator/src/runtime/deps.rs
• crates/moa-orchestrator/src/runtime/jobs.rs
• crates/moa-orchestrator/src/services/authz_admin.rs
• crates/moa-orchestrator/src/services/contacts.rs
• crates/moa-orchestrator/src/services/experiments.rs
• crates/moa-orchestrator/src/services/graph_memory_maint.rs
• crates/moa-orchestrator/src/services/knowledge/ingest.rs
• crates/moa-orchestrator/src/services/privacy/erase.rs
• crates/moa-orchestrator/src/services/privacy/repository.rs
• crates/moa-orchestrator/src/services/session_store/handlers.rs
• crates/moa-orchestrator/src/services/session_store/inner.rs
• crates/moa-orchestrator/src/services/session_store/mod.rs
• crates/moa-orchestrator/src/services/session_store/tests.rs
• crates/moa-orchestrator/src/services/tool_executor.rs
• crates/moa-orchestrator/src/tool_invocation/governed.rs
• crates/moa-orchestrator/src/workflows/experiment_cancel.rs
• crates/moa-orchestrator/src/workflows/experiment_run.rs
• crates/moa-orchestrator/src/workflows/experiment_trial_run.rs
• crates/moa-orchestrator/src/workflows/mod.rs
• crates/moa-orchestrator/src/workflows/procedure_node_actions.rs
• crates/moa-orchestrator/tests/integration/action_policy_flow_e2e.rs
• crates/moa-orchestrator/tests/integration/tool_executor_e2e.rs
• crates/moa-orchestrator/tests/knowledge_service.rs
• crates/moa-orchestrator/tests/orchestrator_db/authz_admin_db.rs
• crates/moa-orchestrator/tests/orchestrator_db/workspace_authz_db.rs
• crates/moa-orchestrator/tests/orchestrator_db_memory/privacy_service_db_memory.rs
• crates/moa-orchestrator/tests/orchestrator_offline/tool_executor.rs
• crates/moa-providers/Cargo.toml
• crates/moa-providers/src/adapters/anthropic/mod.rs
• crates/moa-providers/src/adapters/anthropic/tests.rs
• crates/moa-providers/src/adapters/gemini/mod.rs
• crates/moa-providers/src/adapters/openai_responses/provider.rs
• crates/moa-providers/src/core/concurrency.rs
• crates/moa-providers/src/core/concurrency_factory.rs
• crates/moa-providers/src/core/global_concurrency.rs
• crates/moa-providers/src/core/mod.rs
• crates/moa-providers/src/core/pacer.rs
• crates/moa-providers/src/core/retry.rs
• crates/moa-providers/src/embedding/cohere.rs
• crates/moa-providers/src/embedding/factory.rs
• crates/moa-providers/src/embedding/gemini.rs
• crates/moa-providers/src/embedding/openai.rs
• crates/moa-providers/src/embedding/zeroentropy.rs
• crates/moa-providers/src/lib.rs
• crates/moa-providers/src/rerank/cohere.rs
• crates/moa-providers/src/rerank/factory.rs
• crates/moa-providers/src/rerank/zeroentropy.rs
• crates/moa-providers/tests/cohere_reranker_live.rs
• crates/moa-providers/tests/providers_offline/cohere_reranker_offline.rs
• crates/moa-providers/tests/providers_offline/zeroentropy_reranker_offline.rs
• crates/moa-providers/tests/zeroentropy_reranker_live.rs
• crates/moa-security/Cargo.toml
• crates/moa-security/src/lib.rs
• crates/moa-security/src/mcp_proxy.rs
• crates/moa-session/src/lib.rs
• crates/moa-session/src/store/mod.rs
• crates/moa-session/src/store/session_store.rs
• crates/moa-session/tests/postgres_store_db.rs
• crates/moa-skills/src/format.rs
• crates/moa-skills/tests/distillation_db_memory.rs
• crates/moa-skills/tests/draft_proposals_db_memory.rs
• crates/moa-skills/tests/improver_db_memory.rs
• crates/moa-skills/tests/regression.rs
• crates/moa-skills/tests/support/common.rs
• crates/moa-skills/tests/support/fixtures/session_with_8_tool_calls.json
• crates/xtask/src/check_eval_budgets.rs
• crates/xtask/src/wixqa_rag_eval.rs
• docker-compose.yml
• docs/09-skills-and-learning.md
• docs/23-environment-variables.md
• fly.toml
• scripts/fly-live-smoke.sh

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch tr

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}