Releases: bogdanov-igor/hephaestus-postman-framework
Release list
Hephaestus v4.0.1
Documentation and release-plumbing only — no engine or CLI behaviour changed.
Published so the npm page carries the corrected docs (npm cannot update the
README of an already-published version).
Fixed
- The docs had fallen behind the code. An audit against the shipped source
found: the banner and the social image still readv3.9; both READMEs claimed
the package was "not yet published to npm"; the badges and prose advertised
113/46 tests and 200 golden assertions across 17 requests when the real
figures are 143 and 464 across 48. - 9 of the engine's 43 config keys were undocumented —
strictMode,
extraKeys,maxBytesand the six shipped-plugin keys. All 43 are now in
docs/config-reference.html, with the plugin keys grouped under a heading
that states they are read bydocs/plugins/*rather than by the engine core. - 6 of the 18 CLI commands were missing from both README tables —
flaky,
coverage,trends,mock,doctor,bench— along with the--history
and--demoflags. All 18 are listed now, in both languages. docs/features.htmlgained sections forinit --demo, the plugin gallery,
assertHeadersandextraKeys, plus a CSS rule for the.noteclass it had
been using without ever defining.- README config tables gained
strictMode,extraKeys,graphqland
retryOnStatus/respectRetryAfter; the snapshot paragraph now covers the
structuralmode.
Added
- Architecture diagrams — there were none. Three per language: the
two-runtime layout (engine as collection data inside Postman, CLI outside
reading Newman's output), a request-lifecycle sequence with the real module
order including theretryOnStatusshort-circuit, and a CI-gate flowchart
showing which command fails a build on what. All six were verified to render.
Changed
- Publishing moved to npm Trusted Publishing (OIDC). No npm token is stored
anywhere any more — GitHub Actions proves the workflow's identity to npm
directly, which also means releases now carry provenance attestations. The
publish workflow pins Node 22 and upgrades npm, since OIDC requires npm CLI= 11.5.1 and Node >= 22.14.0.
Hephaestus v4.0.0
A major release: the engine gained five new capabilities, the CLI grew from a
reporting helper into a set of CI gates plus local tooling, and the project now
runs and is verified on Windows as well as Linux.
Upgrading is a drop-in for existing collections. Every new engine feature is
opt-in and off by default, and the ru output stays byte-identical — with one
deliberate exception, noted under Changed: the retry-in-progress test name was
previously hardcoded English even in ru.
Added
- Engine i18n — every user-facing string (test names, log lines, errors) is
routed through a locale catalog (engine/src/shared/i18n.js). Set
"locale": "en"in config for English output; default"ru"reproduces the
historical strings byte-for-byte. Covers both the pre-request and post-request
planes across all modules. - English test-engine fixtures — the golden baseline now locks both
ruand
enoutput, including the failure messages (464 assertions / 48 requests). The
earlier pass only exercised the success half of each catalog entry; a broken
English error string is invisible until something fails, so the negative
fixtures lock those too. - Override typo-guard — an unrecognised top-level
overridekey now warns with
a "did you mean" suggestion (suppressed atlogLevel: "silent"); set
"strictMode": trueto fail the run so CI blocks on a typo (e.g.snapshsotwas
silently ignored before). First-party + shipped-plugin keys are recognised; custom
keys are allowlisted via"extraKeys": [...]. The golden harness gained aneg-*
"expected-fail" convention so negative fixtures don't muddy the pass count. maxBytes— a response-size budget inoverride(bytes); the run fails if the
response body is larger. ThemaxResponseTimeanalogue for payload size.- securityAudit v2 — three opt-in response checks:
cookieFlags(Set-Cookie must
carry Secure / HttpOnly / SameSite),checkJwt(rejectalg:noneand expired
expon JWTs in the body/cookies), andrequireNoStore(auth responses must send
Cache-Control: no-store). All off by default — existing configs are unchanged. - GraphQL asserts — a new opt-in
graphqloverride for the trap where GraphQL
answers HTTP 200 even with anerrors[]array:noErrors(errors empty/absent),
errorCount/errorContains(negative testing), anddataShape(type checks
underdata.*).graphql: trueis shorthand for{ noErrors: true }. retryOnStatushonorsRetry-After— setrespectRetryAfter: trueand the
engine reads the server'sRetry-Afterheader on a retried response (delta-seconds
or HTTP-date). It waits that long (up toretryAfterCapMs, default 10 s) before
re-running; if the server asks for longer than the cap, it stops retrying instead of
hammering. Opt-in — defaultretryOnStatusbehavior is unchanged. (The wait is a
bounded blocking busy-wait — the Postman sandbox has no async sleep that survives
setNextRequest.)- Snapshot
mode: "structural"— a third snapshot mode alongsidestrict/
non-strictthat compares the response shape (every leaf path → its type, array
indices collapsed to[*]) and ignores leaf values. Catches contract changes (a
field added/removed, a type flipped) without the false diffs that volatile values
(timestamps, ids, counts) or array length cause in a strict diff. hephaestus generate— an interactive, zero-dependency wizard (node:readline,
no LLM/network) that asks plane / auth / fields / shape / snapshot and prints a
ready-to-pasteoverrideblock plus the engineeval(...)line, so you scaffold a
request's config without memorising the schema.hephaestus panel— a local dev panel: a zero-dependencynode:httpserver on
127.0.0.1serving one self-contained page. It shows run history with pass-rate /
p95 trend sparklines, a snapshot diff viewer (structural / non-strict /
strict), an editabledefaults.jsonwith live JSON-Schema validation (Save is
blocked while the schema is violated), an override builder that produces a
paste-able block, and the local docs. Nothing is hosted and nothing leaves the
machine. Because it can write, it is locked down deliberately: loopback bind, a
loopback-onlyHostcheck (anti DNS-rebinding) against the actually-bound port, no
request input ever joined into a path (so no directory traversal),X-Frame-Options- CSP
frame-ancestors 'none', and writes that requireContent-Type: application/json
with a same-originOriginand no CORS headers — a foreign page cannot POST to it.
- CSP
hephaestus init --demo— scaffolds a runnable offline demo in one command: a
collection that carries its own snapshots, sohephaestus mockserves it and Newman
runs green with no account, no API key and no network (five requests, one headline
feature each).hephaestus openapi --negative— alongside the happy-path import, generates
error-path tests for the cases the spec actually lets you trigger: withheld auth, a
substituted id, an emptied required body, a dropped required query parameter. It does
not invent a test for a declared status it cannot provoke.- Run-history trends in the HTML report —
hephaestus report results.json out.html --historyoverlays pass-rate and p95 sparklines with the change since the previous
run. The sparkline rendering is shared withtrendsand the panel. - Plugin gallery —
gallery/plugins/: four ready-to-use, zero-dependency plugins
(response budget, timing histogram, PII redactor, CSV metrics) plus a commented
template. Also corrects the three shipped example plugins (slack-notifier,
teams-notifier,custom-assertions), which readctx.apimembers the extractor
does not expose and registered with acode:descriptor the engine ignores — so they
never ran. - Copy-paste CI templates —
docs/ci/with ready-to-use GitHub Actions, GitLab CI
and Jenkins pipelines that run the collection and gate on the Hephaestus CLI's exit
codes. hephaestus bench— measures what the engine actually costs per request by
running an identical request set through Newman twice (full engine vs a no-op) so
the delta isolates engine work: ~1.6 ms/request here.--jsonfor CI and
--max-msas a regression gate.- Windows support, verified in CI — the test matrix now runs
windows-latest
alongsideubuntu-latest. A.gitattributespins LF in the working tree, without
which a Windows checkout produced CRLF and broke the build check, the golden
byte-compare, and thedoctorSHA-256 integrity check — that last one also gave
every Windows user a false engine-integrity failure. npm run check:locales— validates the locale catalog so a translation can be
reviewed: every message carries every locale, parameter counts and argument types
match across locales, nothing rendersundefined, and the status map covers each
locale (which is what makes it selectable). See CONTRIBUTING for the locale guide.
Changed
- The npm package no longer ships the test suite or engine sources. The published
tarball dropped the golden fixtures, the test/build scripts andengine/src/(the
bundles it produces are what runs), narrowing it from 52 files to 38. The typed config
contractdocs/override.schema.jsonis now shipped so the panel can validate against
it, and it was brought back in step with the engine's known-key list (32 → 43 keys, so
a valid config is no longer reported as unknown). - Any locale the catalog carries is now selectable.
locOf()was hardcoded to
'en' : 'ru', so a fully translated third locale could never be used. It now picks
any locale present in the catalog and still falls back toru;ru/en/ absent
/ unknown behave exactly as before. - The retry-in-progress test name is localised. It was hardcoded English even in
ru; withlocale: "ru"it now reads⚡ Повтор 1/3 (статус 503)instead of
⚡ Retry 1/3 (status 503). This is the one user-visiblerustring that changed in
4.0 —retryOnStatushad no golden coverage, which is how it stayed English so long. - The stated engine size is measured, not remembered — the READMEs claimed 172 KB;
hephaestus benchreports the real figure (207 KB for both planes) and the badges
now track it. iterationDatais now a shared module (engine/src/shared/iteration-data.js),
single-sourced into both engine planes via esbuild instead of two hand-synced copies.- Secret-redaction check single-sourced into
engine/src/shared/mask.jsand
unit-tested. Matching stays substring on key names — the fail-safe default for a
redaction feature. (A boundary-precise variant was evaluated and rejected: adversarial
review showed it under-masked concatenated secret names likepasswd/apikey/privatekey.)
Fixed
docsno longer destroys the collection it reads. Without-o, the output
path resolved toargs[args.indexOf('-o') + 1]— which isargs[0], the input
file — sohephaestus docs collection.json(documented as printing to stdout)
overwrote the user's collection with the generated Markdown. The path is now
resolved explicitly, and writing docs over the input is refused outright. Every
previous docs test passed-o, which is exactly why this shipped since v3.7.watchwithout-cresolved the collection path to the first argument, so
watch --delay 500tried to watch a file named--delay. It now reports usage.snapshot.storage: "postman-api"no longer silently no-ops (which left a run
with zero snapshot protection while looking configured). It now warns once and
falls back tocollection-vars, so snapshots actually save/compare. (Real
postman-apistorage is intentionally not implemented — it would need a network +
API-key dependency, against the offline...
v3.0.0 — Initial Release
⚒️ Hephaestus v3.0.0
First public release of the modular API testing automation framework for Postman.
What's included
- Pipeline engine:
pre-request.js+post-request.js - Auth plugin:
none,basic,bearer,headers,variables - Snapshot regression testing (strict / non-strict)
- JSON Schema validation via
tv4 - Secret masking for logs and URL query params
- Auto-update engine from Git (public + private repos via PAT)
- Postman collection template with system methods
- Full Apidog compatibility
Quick start
- Import
collection/hephaestus-template.postman_collection.jsoninto Postman or Apidog - Run
⚙️ defaults→ configure your API - Run
🔧 engine-update→ load the engine - Write requests using the
overridepattern