Skip to content

Add characterization tests ahead of the merge into subscriptions - #70

Merged
johnworth merged 3 commits into
mainfrom
add-characterization-tests
Aug 6, 2026
Merged

Add characterization tests ahead of the merge into subscriptions#70
johnworth merged 3 commits into
mainfrom
add-characterization-tests

Conversation

@johnworth

Copy link
Copy Markdown
Collaborator

Summary

QMS had no tests at all. This adds a suite that runs the real router against a
real PostgreSQL instance (testcontainers, stock postgres:17, the repo's own
migrations), so the recorded responses describe what the service actually does
rather than what a mock says it does.

It exists to make the planned merge of QMS into cyverse-de/subscriptions
provably behavior-preserving: the same tests move with the code and must pass
unchanged against the merged binary. That is the merge's acceptance criterion.

Coverage is scoped to the eight /v1 path shapes terrain actually calls
(terrain/src/terrain/clients/qms.clj), asserting the full
{result, error, status} envelope terrain's route schemas require — not to
100% of the API, since the rest has no live caller.

  • Golden tests for the terrain-facing routes, with database-generated UUIDs
    and timestamps redacted. The migration-seeded UUIDs are asserted exactly, so
    a golden still pins which plan or resource type a response refers to.
  • Integration tests for the behaviors the subscriptions service implements
    differently — audit rows in updates, quota addressing, subscription
    deactivation — since those are where a merge silently changes semantics.
  • Unit tests for the plan-rate and quota-default selectors, which are
    duplicated in both codebases; only one implementation survives the merge.

Migrations are now embedded and applied through the iofs source driver
instead of file://$PWD/migrations, so neither the tests nor the service
depend on the working directory.

Bugs this surfaced

All three are recorded as-is with a KNOWN BUG comment rather than fixed,
so the merge cannot change them by accident. Each should be fixed as its own
change, updating the corresponding assertion in the same commit.

  1. POST /v1/subscriptions?force=false fails for every request and reports
    success.
    GetActiveSubscriptionForDate builds
    .Or("? > subscriptions.effective_start_date AND ...") with no argument for
    the placeholder, so PostgreSQL rejects the statement with
    syntax error at or near ">". The handler turns that into a per-item
    failure_reason and still answers 200, so terrain's
    POST /terrain/admin/qms/subscriptions silently creates nothing whenever
    force isn't set. This also means the upgrade-comparison rule the unforced
    path exists to implement has never run.
  2. Every updates row records ADD, even for a SET. The operation is
    looked up with tx.First(&model.UpdateOperation{Name: usage.UpdateType}),
    but GORM's First only filters on the primary key, so it returns whichever
    row sorts first. The running total in usages is correct; only the audit
    trail misreports. The same unreachable lookup is why an invalid update type
    returns 500 instead of 400.
  3. The two subscribe paths store different usernames for the same person.
    POST /v1/subscriptions passes req.Username through untouched while every
    other endpoint trims the configured suffix, so an admin bulk-subscribing
    user@domain creates a second user row that the user's own
    /terrain/qms/user/* routes never see. Existing databases likely hold
    duplicate rows that need reconciling.

Also worth noting, and captured in a golden: GET /v1/users/{username}/plan
creates the user and subscribes them to the default plan when QMS hasn't seen
them, so the read has a write side effect and returns 200 rather than 404.
That one appears intentional — terrain relies on it — so it is recorded as
expected behavior.

Notes

  • QMS had no test workflow; one is added. The lint pin moves from
    @v0.0.4/go1.21 to @v0.4.1/go1.25, and the newer analyzers flag four
    redundant type declarations in existing code, fixed here so the bump lands
    green.
  • testcontainers runs on GitHub-hosted runners with no workflow changes, so
    unlike the env-var-gated database tests elsewhere in the org, these actually
    execute in CI.
  • There is deliberately no golden regeneration flag. A missing golden is
    written and the test still fails, so a new baseline is committed
    deliberately; an existing one is never overwritten.

Testing

go test ./... and golangci-lint run ./... (v2.12.2, the CI pin) are clean;
the suite was run repeatedly to confirm the goldens are stable.

🤖 Generated with Claude Code

John Wregglesworth and others added 3 commits August 6, 2026 15:39
QMS had no tests at all. This adds a suite that runs the real router against
a real PostgreSQL instance started by testcontainers, so the recorded
responses describe what the service actually does rather than what a mock
says it does. It is the acceptance criterion for the pending merge into the
subscriptions service: the same tests must pass unchanged against the merged
binary.

The migrations are now embedded and applied through the iofs source driver
instead of file://$PWD/migrations, so neither the tests nor the service
depend on the working directory.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Covers all eight /v1 path shapes terrain's clients/qms.clj calls, asserting
the full {result, error, status} envelope its route schemas require.

Two behaviors are recorded as they are rather than as they should be, so the
merge into subscriptions cannot change them silently:

  - GET /v1/users/{username}/plan creates the user and subscribes them to the
    default plan when QMS has not seen them before, so the read has a write
    side effect and returns 200 rather than 404.
  - POST /v1/usages answers 500, not 400, for an invalid update type.
    httpStatusCode matches its sentinel errors with ==, but addUsage returns a
    formatted error wrapping the offending value, so no case ever matches.

Quota and usage collections come back without an ORDER BY, so the harness
sorts those fields before comparing. Arrays returned directly by the listing
endpoints keep their order and are still verified.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The behaviors covered here are the ones the subscriptions service implements
differently, so a merge has to choose an implementation deliberately: whether
a usage change writes an audit row to `updates`, how a quota is addressed,
and that overlapping subscriptions get deactivated.

Writing them surfaced three bugs, all recorded as-is with a KNOWN BUG comment
so the merge cannot change them by accident:

  - POST /v1/subscriptions?force=false fails for every request and reports
    success anyway. GetActiveSubscriptionForDate passes no argument for the
    placeholder in its Or() clause, so PostgreSQL rejects the statement, and
    the handler buries the failure in a per-item failure_reason behind a 200.
    Terrain's admin subscription route silently creates nothing.
  - Every `updates` row records the ADD operation, even for a SET. GORM's
    First only filters on the primary key, so the operation lookup ignores the
    name it was given and returns whichever row sorts first. The running total
    is unaffected; only the audit trail misreports.
  - The two ways of subscribing a user store two different usernames for the
    same person: the bulk endpoint keeps the configured suffix while every
    other endpoint trims it, so an admin bulk-subscribing a user creates a
    second user row their own routes never see.

QMS had no test workflow, so one is added, and the lint pin moves from
v0.0.4/go1.21 to v0.4.1/go1.25. The newer analyzers flag four redundant type
declarations in existing code; those are fixed here so the bump lands green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@johnworth
johnworth merged commit ffeacf8 into main Aug 6, 2026
2 checks passed
@johnworth
johnworth deleted the add-characterization-tests branch August 6, 2026 23:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant