Skip to content

ci(api-contract): track latest by default and surface Postman results - #578

Merged
roncodes merged 2 commits into
mainfrom
ci/postman-contract-latest-and-reporting
Aug 8, 2026
Merged

ci(api-contract): track latest by default and surface Postman results#578
roncodes merged 2 commits into
mainfrom
ci/postman-contract-latest-and-reporting

Conversation

@roncodes

@roncodes roncodes commented Aug 8, 2026

Copy link
Copy Markdown
Member

Answers two questions about the contract workflow: does the image have to be pinned, and why can't you see the Postman run?

1. It no longer has to be pinned

fleetbase-ref defaulted to dev-v0.7.53 — a value someone has to remember to bump on every release, and which silently tests an old platform when they forget.

  • fleetbase-ref now defaults to main.
  • New api-image input defaults to fleetbase/fleetbase-api:latest, which is what each release publishes.

Callers therefore need no version at all:

jobs:
  contract:
    uses: fleetbase/fleetbase/.github/workflows/api-contract.yml@main
    with:
      collections: "Fleetbase Storefront API"
      build-from-source: false
    secrets: inherit

Pinning still works for reproducing a specific failure (api-image: fleetbase/fleetbase-api:v0.7.53); a non-latest image is retagged so the compose file's fleetbase/fleetbase-api:latest reference resolves.

The tradeoff of tracking a moving tag is that a red run is hard to reproduce later, so the pull step now records the resolved tags, digest and build date into the step summary.

2. The run is now visible

Previously every collection ran inside ::group:: and reported only ✔ name passed — collapsed by default, with no assertion counts and no endpoints.

  • Output is printed uncollapsed and tee'd to postman-results/.

  • The step summary gets a per-collection table:

    Collection Result Requests Assertions Failed
    Fleetbase API ✅ passed 42/0 180/0 0
  • Failing assertions are expanded into a <details> block under the table.

  • Full logs upload as the postman-results artifact (14 day retention).

Bug fixed along the way

A missing collection directory was ::warning:: and did not fail the job — a typo in collections produced a green run that tested nothing. It is now ::error:: and fails.

Verification

Both files validate as YAML, and the run: scripts were extracted and checked with bash -n. The summary-table parsing was dry-run against a sample Postman reporter log — that caught a real bug, since the reporter's table is drawn with box-drawing characters rather than ASCII pipes, so the original requests regex matched nothing. The docker image inspect --format string was run against a local image, confirming join tolerates an image with no digest where index would have errored.

Not verified end-to-end against a live run — that needs POSTMAN_API_KEY, which is why this PR is worth watching on its first execution.

🤖 Generated with Claude Code

roncodes and others added 2 commits August 7, 2026 22:59
Two problems with the contract workflow: it pinned a release ref that had to
be remembered on every publish, and a run gave no visible evidence that
Postman had executed at all.

Track latest instead of pinning
- `fleetbase-ref` now defaults to `main` rather than `dev-v0.7.53`, so callers
  get the current platform without a per-release bump.
- New `api-image` input defaults to `fleetbase/fleetbase-api:latest`, which is
  what each release publishes. Callers can pin a tag to reproduce an old
  result; a non-latest image is retagged so the compose reference resolves.
- Because :latest is a moving target, the pull step records the resolved tags,
  digest and build date into the step summary — otherwise a red run months
  from now would be unreproducible.

Make the run visible
- Each collection's output is tee'd to postman-results/ and printed
  uncollapsed; ::group:: was hiding the one thing the job exists to show.
- A markdown table in the step summary gives per-collection result, requests,
  assertions and failure count, with failing assertions in a <details> block.
- Full logs upload as the `postman-results` artifact (14 day retention).
- A missing collection directory is now ::error:: and fails the job. It was a
  ::warning:: that passed silently, so a typo in `collections` looked green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@codecov

codecov Bot commented Aug 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 69.26%. Comparing base (122c933) to head (c337a11).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##               main     #578   +/-   ##
=========================================
  Coverage     69.26%   69.26%           
  Complexity       22       22           
=========================================
  Files           176      176           
  Lines          3153     3153           
=========================================
  Hits           2184     2184           
  Misses          969      969           
Flag Coverage Δ
api 41.46% <ø> (ø)
console 70.00% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@roncodes
roncodes merged commit 8f3c8ff into main Aug 8, 2026
10 checks passed
roncodes added a commit to fleetbase/fleetops that referenced this pull request Aug 8, 2026
fleetbase/fleetbase#578 changed the reusable workflow to default fleetbase-ref
to main and to test against fleetbase/fleetbase-api:latest, so there is no
longer a per-release ref to bump here. Drops the explicit fleetbase-ref and
moves the workflow reference from @v0.7.53 to @main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
roncodes added a commit to fleetbase/ledger that referenced this pull request Aug 8, 2026
fleetbase/fleetbase#578 changed the reusable workflow to default fleetbase-ref
to main and to test against fleetbase/fleetbase-api:latest, so there is no
longer a per-release ref to bump here. Drops the explicit fleetbase-ref and
moves the workflow reference from @v0.7.53 to @main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
roncodes added a commit to fleetbase/storefront that referenced this pull request Aug 8, 2026
fleetbase/fleetbase#578 changed the reusable workflow to default fleetbase-ref
to main and to test against fleetbase/fleetbase-api:latest, so there is no
longer a per-release ref to bump here. Drops the explicit fleetbase-ref and
moves the workflow reference from @v0.7.53 to @main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
roncodes added a commit to fleetbase/core-api that referenced this pull request Aug 8, 2026
fleetbase/fleetbase#578 changed the reusable workflow to default fleetbase-ref
to main and to test against fleetbase/fleetbase-api:latest, so there is no
longer a per-release ref to bump here. Drops the explicit fleetbase-ref and
moves the workflow reference from @v0.7.53 to @main.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant