Skip to content

Releases: bogdanov-igor/hephaestus-postman-framework

Hephaestus v4.0.1

Choose a tag to compare

@github-actions github-actions released this 22 Jul 13:03

Documentation and release-plumbing only — no engine or CLI behaviour changed.
Published so the npm page carries the corrected docs (npm cannot update the
README of an already-published version).

Fixed

  • The docs had fallen behind the code. An audit against the shipped source
    found: the banner and the social image still read v3.9; both READMEs claimed
    the package was "not yet published to npm"; the badges and prose advertised
    113/46 tests and 200 golden assertions across 17 requests when the real
    figures are 143 and 464 across 48.
  • 9 of the engine's 43 config keys were undocumentedstrictMode,
    extraKeys, maxBytes and the six shipped-plugin keys. All 43 are now in
    docs/config-reference.html, with the plugin keys grouped under a heading
    that states they are read by docs/plugins/* rather than by the engine core.
  • 6 of the 18 CLI commands were missing from both README tablesflaky,
    coverage, trends, mock, doctor, bench — along with the --history
    and --demo flags. All 18 are listed now, in both languages.
  • docs/features.html gained sections for init --demo, the plugin gallery,
    assertHeaders and extraKeys, plus a CSS rule for the .note class it had
    been using without ever defining.
  • README config tables gained strictMode, extraKeys, graphql and
    retryOnStatus / respectRetryAfter; the snapshot paragraph now covers the
    structural mode.

Added

  • Architecture diagrams — there were none. Three per language: the
    two-runtime layout (engine as collection data inside Postman, CLI outside
    reading Newman's output), a request-lifecycle sequence with the real module
    order including the retryOnStatus short-circuit, and a CI-gate flowchart
    showing which command fails a build on what. All six were verified to render.

Changed

  • Publishing moved to npm Trusted Publishing (OIDC). No npm token is stored
    anywhere any more — GitHub Actions proves the workflow's identity to npm
    directly, which also means releases now carry provenance attestations. The
    publish workflow pins Node 22 and upgrades npm, since OIDC requires npm CLI

    = 11.5.1 and Node >= 22.14.0.


Hephaestus v4.0.0

Choose a tag to compare

@github-actions github-actions released this 21 Jul 06:42

A major release: the engine gained five new capabilities, the CLI grew from a
reporting helper into a set of CI gates plus local tooling, and the project now
runs and is verified on Windows as well as Linux.

Upgrading is a drop-in for existing collections. Every new engine feature is
opt-in and off by default, and the ru output stays byte-identical — with one
deliberate exception, noted under Changed: the retry-in-progress test name was
previously hardcoded English even in ru.

Added

  • Engine i18n — every user-facing string (test names, log lines, errors) is
    routed through a locale catalog (engine/src/shared/i18n.js). Set
    "locale": "en" in config for English output; default "ru" reproduces the
    historical strings byte-for-byte. Covers both the pre-request and post-request
    planes across all modules.
  • English test-engine fixtures — the golden baseline now locks both ru and
    en output, including the failure messages (464 assertions / 48 requests). The
    earlier pass only exercised the success half of each catalog entry; a broken
    English error string is invisible until something fails, so the negative
    fixtures lock those too.
  • Override typo-guard — an unrecognised top-level override key now warns with
    a "did you mean" suggestion (suppressed at logLevel: "silent"); set
    "strictMode": true to fail the run so CI blocks on a typo (e.g. snapshsot was
    silently ignored before). First-party + shipped-plugin keys are recognised; custom
    keys are allowlisted via "extraKeys": [...]. The golden harness gained a neg-*
    "expected-fail" convention so negative fixtures don't muddy the pass count.
  • maxBytes — a response-size budget in override (bytes); the run fails if the
    response body is larger. The maxResponseTime analogue for payload size.
  • securityAudit v2 — three opt-in response checks: cookieFlags (Set-Cookie must
    carry Secure / HttpOnly / SameSite), checkJwt (reject alg:none and expired
    exp on JWTs in the body/cookies), and requireNoStore (auth responses must send
    Cache-Control: no-store). All off by default — existing configs are unchanged.
  • GraphQL asserts — a new opt-in graphql override for the trap where GraphQL
    answers HTTP 200 even with an errors[] array: noErrors (errors empty/absent),
    errorCount / errorContains (negative testing), and dataShape (type checks
    under data.*). graphql: true is shorthand for { noErrors: true }.
  • retryOnStatus honors Retry-After — set respectRetryAfter: true and the
    engine reads the server's Retry-After header on a retried response (delta-seconds
    or HTTP-date). It waits that long (up to retryAfterCapMs, default 10 s) before
    re-running; if the server asks for longer than the cap, it stops retrying instead of
    hammering. Opt-in — default retryOnStatus behavior is unchanged. (The wait is a
    bounded blocking busy-wait — the Postman sandbox has no async sleep that survives
    setNextRequest.)
  • Snapshot mode: "structural" — a third snapshot mode alongside strict /
    non-strict that compares the response shape (every leaf path → its type, array
    indices collapsed to [*]) and ignores leaf values. Catches contract changes (a
    field added/removed, a type flipped) without the false diffs that volatile values
    (timestamps, ids, counts) or array length cause in a strict diff.
  • hephaestus generate — an interactive, zero-dependency wizard (node:readline,
    no LLM/network) that asks plane / auth / fields / shape / snapshot and prints a
    ready-to-paste override block plus the engine eval(...) line, so you scaffold a
    request's config without memorising the schema.
  • hephaestus panel — a local dev panel: a zero-dependency node:http server on
    127.0.0.1 serving one self-contained page. It shows run history with pass-rate /
    p95 trend sparklines, a snapshot diff viewer (structural / non-strict /
    strict), an editable defaults.json with live JSON-Schema validation (Save is
    blocked while the schema is violated), an override builder that produces a
    paste-able block, and the local docs. Nothing is hosted and nothing leaves the
    machine. Because it can write, it is locked down deliberately: loopback bind, a
    loopback-only Host check (anti DNS-rebinding) against the actually-bound port, no
    request input ever joined into a path (so no directory traversal), X-Frame-Options
    • CSP frame-ancestors 'none', and writes that require Content-Type: application/json
      with a same-origin Origin and no CORS headers — a foreign page cannot POST to it.
  • hephaestus init --demo — scaffolds a runnable offline demo in one command: a
    collection that carries its own snapshots, so hephaestus mock serves it and Newman
    runs green with no account, no API key and no network (five requests, one headline
    feature each).
  • hephaestus openapi --negative — alongside the happy-path import, generates
    error-path tests for the cases the spec actually lets you trigger: withheld auth, a
    substituted id, an emptied required body, a dropped required query parameter. It does
    not invent a test for a declared status it cannot provoke.
  • Run-history trends in the HTML reporthephaestus report results.json out.html --history overlays pass-rate and p95 sparklines with the change since the previous
    run. The sparkline rendering is shared with trends and the panel.
  • Plugin gallerygallery/plugins/: four ready-to-use, zero-dependency plugins
    (response budget, timing histogram, PII redactor, CSV metrics) plus a commented
    template. Also corrects the three shipped example plugins (slack-notifier,
    teams-notifier, custom-assertions), which read ctx.api members the extractor
    does not expose and registered with a code: descriptor the engine ignores — so they
    never ran.
  • Copy-paste CI templatesdocs/ci/ with ready-to-use GitHub Actions, GitLab CI
    and Jenkins pipelines that run the collection and gate on the Hephaestus CLI's exit
    codes.
  • hephaestus bench — measures what the engine actually costs per request by
    running an identical request set through Newman twice (full engine vs a no-op) so
    the delta isolates engine work: ~1.6 ms/request here. --json for CI and
    --max-ms as a regression gate.
  • Windows support, verified in CI — the test matrix now runs windows-latest
    alongside ubuntu-latest. A .gitattributes pins LF in the working tree, without
    which a Windows checkout produced CRLF and broke the build check, the golden
    byte-compare, and the doctor SHA-256 integrity check — that last one also gave
    every Windows user a false engine-integrity failure.
  • npm run check:locales — validates the locale catalog so a translation can be
    reviewed: every message carries every locale, parameter counts and argument types
    match across locales, nothing renders undefined, and the status map covers each
    locale (which is what makes it selectable). See CONTRIBUTING for the locale guide.

Changed

  • The npm package no longer ships the test suite or engine sources. The published
    tarball dropped the golden fixtures, the test/build scripts and engine/src/ (the
    bundles it produces are what runs), narrowing it from 52 files to 38. The typed config
    contract docs/override.schema.json is now shipped so the panel can validate against
    it, and it was brought back in step with the engine's known-key list (32 → 43 keys, so
    a valid config is no longer reported as unknown).
  • Any locale the catalog carries is now selectable. locOf() was hardcoded to
    'en' : 'ru', so a fully translated third locale could never be used. It now picks
    any locale present in the catalog and still falls back to ru; ru / en / absent
    / unknown behave exactly as before.
  • The retry-in-progress test name is localised. It was hardcoded English even in
    ru; with locale: "ru" it now reads ⚡ Повтор 1/3 (статус 503) instead of
    ⚡ Retry 1/3 (status 503). This is the one user-visible ru string that changed in
    4.0 — retryOnStatus had no golden coverage, which is how it stayed English so long.
  • The stated engine size is measured, not remembered — the READMEs claimed 172 KB;
    hephaestus bench reports the real figure (207 KB for both planes) and the badges
    now track it.
  • iterationData is now a shared module (engine/src/shared/iteration-data.js),
    single-sourced into both engine planes via esbuild instead of two hand-synced copies.
  • Secret-redaction check single-sourced into engine/src/shared/mask.js and
    unit-tested. Matching stays substring on key names — the fail-safe default for a
    redaction feature. (A boundary-precise variant was evaluated and rejected: adversarial
    review showed it under-masked concatenated secret names like passwd/apikey/privatekey.)

Fixed

  • docs no longer destroys the collection it reads. Without -o, the output
    path resolved to args[args.indexOf('-o') + 1] — which is args[0], the input
    file — so hephaestus docs collection.json (documented as printing to stdout)
    overwrote the user's collection with the generated Markdown. The path is now
    resolved explicitly, and writing docs over the input is refused outright. Every
    previous docs test passed -o, which is exactly why this shipped since v3.7.
  • watch without -c resolved the collection path to the first argument, so
    watch --delay 500 tried to watch a file named --delay. It now reports usage.
  • snapshot.storage: "postman-api" no longer silently no-ops (which left a run
    with zero snapshot protection while looking configured). It now warns once and
    falls back to collection-vars, so snapshots actually save/compare. (Real
    postman-api storage is intentionally not implemented — it would need a network +
    API-key dependency, against the offline...
Read more

v3.0.0 — Initial Release

Choose a tag to compare

@bogdanov-igor bogdanov-igor released this 20 Feb 18:10

⚒️ Hephaestus v3.0.0

First public release of the modular API testing automation framework for Postman.

What's included

  • Pipeline engine: pre-request.js + post-request.js
  • Auth plugin: none, basic, bearer, headers, variables
  • Snapshot regression testing (strict / non-strict)
  • JSON Schema validation via tv4
  • Secret masking for logs and URL query params
  • Auto-update engine from Git (public + private repos via PAT)
  • Postman collection template with system methods
  • Full Apidog compatibility

Quick start

  1. Import collection/hephaestus-template.postman_collection.json into Postman or Apidog
  2. Run ⚙️ defaults → configure your API
  3. Run 🔧 engine-update → load the engine
  4. Write requests using the override pattern

Documentation