Skip to content

feat(cli): Add e2e tests and fix bugs.#3353

Merged
c-r33d merged 2 commits intomainfrom
e2e-rr-obligation-trigger
Apr 21, 2026
Merged

feat(cli): Add e2e tests and fix bugs.#3353
c-r33d merged 2 commits intomainfrom
e2e-rr-obligation-trigger

Conversation

@c-r33d
Copy link
Copy Markdown
Contributor

@c-r33d c-r33d commented Apr 21, 2026

Summary

Adds end-to-end coverage for namespaced-policy migration of registered resources and obligation triggers, plus a simple all-scopes migration test. This also fixes namespaced-policy planner/retriever bugs uncovered by the new tests.

What changed

  • Added migrate-namespaced-policy.bats coverage for:
    • registered resource migration
    • obligation trigger migration
    • a simple all-scopes migration flow with one object of each type
  • Updated the BATS helpers/assertions to:
    • handle registered resource values and obligation triggers
    • verify migrated obligation-trigger actions by fetching the action directly
    • seed legacy obligation triggers directly in Postgres when API validation blocks the legacy test shape
    • fix helper output-variable scoping so created IDs propagate correctly to callers
  • Fixed namespaced-policy retrieval for registered resources by hydrating values from ListRegisteredResourceValues instead of relying on partial ListRegisteredResources responses
  • Fixed obligation-trigger retrieval/planning to use action IDs, not action namespace fields from ListObligationTriggers, since that payload does not reliably include action namespace data
  • Reused the existing listActionsForNamespaces results when resolving existing obligation triggers and filtered existing triggers by allowed action IDs per namespace
  • Added an explicit invariant/error when obligation-trigger existing-target lookup is attempted for a namespace missing from the action lookup map
  • Added/updated unit tests for:
    • RR value hydration
    • source obligation-trigger filtering using ListActions
    • existing obligation-trigger filtering by namespaced action IDs
    • missing action-map namespace error handling

Why

The new e2e cases exposed two main issues:

  • registered resource value metadata was missing because RR values were not being fully hydrated
  • obligation-trigger planning could misclassify or duplicate matches because ListObligationTriggers does not provide enough action namespace information to use action.namespace safely

Summary by CodeRabbit

Release Notes

  • Tests
    • Enhanced migration test coverage for policy registered resources and obligation triggers, including action-attribute binding rewrites, metadata preservation, and idempotency validation.
    • Expanded test infrastructure with new fixture creation and tracking utilities for comprehensive migration scenario validation across all supported policy object types.

@c-r33d c-r33d requested a review from a team as a code owner April 21, 2026 16:28
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 21, 2026

Warning

Rate limit exceeded

@c-r33d has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 46 minutes and 36 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 46 minutes and 36 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 74debd1d-e54a-4441-bb37-9e89a56347f6

📥 Commits

Reviewing files that changed from the base of the PR and between 035213c and e9f93b6.

📒 Files selected for processing (2)
  • otdfctl/e2e/migrate-namespaced-policy.bats
  • otdfctl/migrations/namespacedpolicy/retrieve.go
📝 Walkthrough

Walkthrough

The changes extend the namespacedpolicy migration planner to retrieve and hydrate registered resource values, filter obligation triggers by namespaced action IDs, and introduce comprehensive end-to-end tests for registered resource and obligation trigger migration scenarios.

Changes

Cohort / File(s) Summary
E2E Test Additions
otdfctl/e2e/migrate-namespaced-policy.bats
Added CLI wrapper helpers, SQL/JSON utilities (sql_escape_literal, run_policy_db_sql, build_metadata_json_from_labels), fixture tracking/accounting for registered resources and obligation triggers, and three new migration test cases for registered resources, obligation triggers, and all scopes migrations.
PolicyClient Interface Extension
otdfctl/migrations/namespacedpolicy/planner.go
Added ListRegisteredResourceValues method to PolicyClient interface to support paginated listing of registered resource values by resource ID.
Planner Test Infrastructure
otdfctl/migrations/namespacedpolicy/planner_test.go
Extended test handler with stored state for registered resource values and call tracking; implemented ListRegisteredResourceValues method with fallback resolution logic.
Retriever Hydration & Filtering Logic
otdfctl/migrations/namespacedpolicy/retrieve.go
Added protobuf cloning support, registered resource value hydration via new hydrateRegisteredResource and listRegisteredResourceValues helpers, and updated obligation trigger filtering to use action ID membership checks instead of namespace-only validation.
Retriever Test Updates
otdfctl/migrations/namespacedpolicy/retrieve_test.go
Renamed and extended registered resource tests to verify value hydration with metadata; added obligation trigger tests for action-ID-based filtering and namespace validation; updated test handler with action and registered resource value pagination support.

Sequence Diagram

sequenceDiagram
    participant Retriever
    participant PolicyClient
    participant RegisteredResourceAPI
    participant ActionAPI
    participant ObligationTriggerAPI

    Retriever->>ActionAPI: ListActions (by namespace)
    ActionAPI-->>Retriever: action IDs per namespace
    
    Retriever->>RegisteredResourceAPI: ListRegisteredResources (by namespace)
    RegisteredResourceAPI-->>Retriever: registered resource list
    
    loop For each registered resource
        Retriever->>RegisteredResourceAPI: ListRegisteredResourceValues (resourceID)
        RegisteredResourceAPI-->>Retriever: hydrated values with metadata
    end
    
    Retriever->>ObligationTriggerAPI: ListObligationTriggers (by namespace)
    ObligationTriggerAPI-->>Retriever: obligation triggers
    
    Retriever->>Retriever: Filter triggers where action ID ∈ namespaced actions
    Retriever-->>Retriever: Return hydrated resources & filtered triggers
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested labels

comp:policy, comp:ci, size/m

Suggested reviewers

  • elizabethhealy
  • alkalescent

Poem

🐰 Hop, hop—the resources now hydrate with care,
With values retrieved and actions laid bare,
Obligation triggers filtered with grace,
Migrations dance through namespaced space! ✨

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title uses generic phrasing ('Add e2e tests and fix bugs') that does not clearly convey the specific nature of changes to namespaced-policy migration and related planner/retriever components. Consider using a more specific title like 'feat(cli): Add namespaced-policy migration e2e tests and fix RR value hydration' to better reflect the main changes.
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch e2e-rr-obligation-trigger

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces significant improvements to the namespaced-policy migration process by adding robust end-to-end test coverage and fixing critical bugs in the planner and retriever components. These changes ensure that registered resources and obligation triggers are correctly hydrated and planned during migration, improving the reliability and accuracy of the namespaced-policy migration flow.

Highlights

  • End-to-End Test Coverage: Added comprehensive BATS test coverage for namespaced-policy migration, specifically targeting registered resources, obligation triggers, and all-scopes migration flows.
  • Bug Fixes in Policy Planner/Retriever: Resolved issues with registered resource value hydration and obligation-trigger planning by ensuring reliable action ID usage and proper hydration from API responses.
  • Test Infrastructure Improvements: Enhanced BATS helpers to support registered resource values and obligation triggers, including direct Postgres seeding for legacy data and improved variable scoping.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.


The policy migrates with care, New tests are added everywhere. With triggers fixed and values found, The namespaced logic is now sound.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request extends the namespaced policy migration tool to support registered resources and obligation triggers. It introduces new retrieval logic in the planner to hydrate registered resources with their values and filters obligation triggers based on legacy action IDs. Additionally, the E2E test suite is significantly expanded with new helper functions and test cases covering various migration scenarios and idempotency. Review feedback highlights a potential N+1 performance issue during resource hydration, suggests minor optimizations for map initialization, and identifies opportunities for better consistency in the BATS test helpers.

Comment thread otdfctl/migrations/namespacedpolicy/retrieve.go
Comment thread otdfctl/migrations/namespacedpolicy/retrieve.go Outdated
Comment thread otdfctl/migrations/namespacedpolicy/retrieve.go Outdated
Comment thread otdfctl/migrations/namespacedpolicy/retrieve.go Outdated
Comment thread otdfctl/e2e/migrate-namespaced-policy.bats
@github-actions
Copy link
Copy Markdown
Contributor

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 170.893261ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 94.665202ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 408.501276ms
Throughput 244.80 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 42.462318756s
Average Latency 423.53435ms
Throughput 117.75 requests/second

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@otdfctl/e2e/migrate-namespaced-policy.bats`:
- Around line 56-69: Validate and/or quote OPENTDF_DB_SCHEMA before injecting it
into the SET search_path in run_policy_db_sql: ensure the env value matches a
safe SQL identifier regex (e.g. ^[A-Za-z_][A-Za-z0-9_]*$) and if not, fall back
to the default "opentdf_policy"; then interpolate the validated name as a quoted
identifier (wrap with double quotes) when building the SET search_path statement
so malformed values cannot break or inject SQL.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 8dde23c3-b912-4446-a67c-5418ffcc3784

📥 Commits

Reviewing files that changed from the base of the PR and between 3940bf8 and 035213c.

📒 Files selected for processing (5)
  • otdfctl/e2e/migrate-namespaced-policy.bats
  • otdfctl/migrations/namespacedpolicy/planner.go
  • otdfctl/migrations/namespacedpolicy/planner_test.go
  • otdfctl/migrations/namespacedpolicy/retrieve.go
  • otdfctl/migrations/namespacedpolicy/retrieve_test.go

Comment thread otdfctl/e2e/migrate-namespaced-policy.bats Outdated
@github-actions
Copy link
Copy Markdown
Contributor

Benchmark results, click to expand

Benchmark authorization.GetDecisions Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 191.227173ms

Benchmark authorization.v2.GetMultiResourceDecision Results:

Metric Value
Approved Decision Requests 1000
Denied Decision Requests 0
Total Time 99.84616ms

Benchmark Statistics

Name № Requests Avg Duration Min Duration Max Duration

Bulk Benchmark Results

Metric Value
Total Decrypts 100
Successful Decrypts 100
Failed Decrypts 0
Total Time 395.772332ms
Throughput 252.67 requests/second

TDF3 Benchmark Results:

Metric Value
Total Requests 5000
Successful Requests 5000
Failed Requests 0
Concurrent Requests 50
Total Time 42.083409757s
Average Latency 419.095553ms
Throughput 118.81 requests/second

@github-actions
Copy link
Copy Markdown
Contributor

⚠️ Govulncheck found vulnerabilities ⚠️

The following modules have known vulnerabilities:

  • examples
  • otdfctl
  • sdk
  • service
  • lib/fixtures
  • tests-bdd

See the workflow run for details.

@c-r33d c-r33d added this pull request to the merge queue Apr 21, 2026
Merged via the queue into main with commit 213d843 Apr 21, 2026
39 checks passed
@c-r33d c-r33d deleted the e2e-rr-obligation-trigger branch April 21, 2026 18:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants