fix(cli): SSL/gRPC/traceroute — valid assertion builders, honest reporter, rich checks get [SIM-286][SIM-287][SIM-291]#1391
Open
danielpaulus wants to merge 5 commits into
Conversation
… gRPC/SSL detail [SIM-286] Suppress the Assertions block when a result has a requestError (nothing was evaluated); humanize gRPC assertion source labels; render gRPC response time; add an SSL response-time reason line and a security-baseline verdict line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
…valid payloads [SIM-287] responseTime() defaults property to 'avg' + avg/min/max/stdDev selectors; SslBaselineSeverity 'warn' -> 'degrade'; add greaterThanOrEqual and matches builders; add SSL sources OCSP_STAPLED/HANDSHAKE_TIME_MS/SAN_CONTAINS; add degraded<=max validation. Codegen updated to round-trip all of the above. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
…et [SIM-291] checkly checks get --result rendered the per-type RESULT block but omitted the assertions (so a failing SSL/gRPC/traceroute check showed no reason) and a lot of available detail. Bring it on par with checkly test and surface the rich result body agents can use: - Extract the assertion-line renderer into a shared formatters/assertion-line.ts used by BOTH the checkly test reporter and checks get, so they render identically and cannot drift. - Add an ASSERTIONS section to the SSL/gRPC/traceroute detail formatters (terminal + markdown), including a Received value for falsy actuals (0/false/''). - gRPC: add a TIMING breakdown (dns/connect/total). - SSL: add a CERTIFICATE section (subject/issuer CN, validity, key, signature, SHA-256 fingerprint, SANs, serial, self-signed/CA, OCSP stapled) and a per-rule SECURITY BASELINE breakdown, keeping the one-line baseline summary. - Traceroute: add probe protocol + DNS timing. - Harden the shared truncate() to cap on the stringified length (objects/numbers no longer bypass it) and slice by code point (no split surrogate pairs); give the markdown request-error its own heading. The --output json path already emits the full result body and is unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
…M-291] Adds the two branches the tri-model review flagged as untested: a scalar securityBaseline rule (rendered as key: value) and selfSigned/isCA=true cert flags. Test-only; no behavior change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
…SIM-287] GrpcMonitor capped degradedResponseTime/maxResponseTime at 30000 (JSDoc @maximum and the validate() bounds), but gRPC calls run to the 180s timeout and the backend allows up to 180000 (grpcResponseTimeLimitFields). Users couldn't set a gRPC threshold above 30s. Raise the cap to 180000 to match the backend (and the terraform provider fix). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NC9pESYE5wGqy5TKRzufyc
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes SSL/gRPC/traceroute CLI support on top of #1362: assertion constructs emit backend-valid payloads,
checkly testoutput is honest, andcheckly checks getshows rich per-type detail. Stacked onfeat/cli-pertype-result-rendering(#1362).What changed
Assertion constructs (SIM-287)
responseTime()defaultspropertytoavg+ adds.avg()/.min()/.max()/.stdDev()(previously emitted an empty property the backend rejects → 400).'warn'→'degrade'(matches the backend enum).greaterThanOrEqual+matchesbuilders (for KEY_SIZE_BITS / TLS_VERSION / CIPHER_SUITE / ISSUER_CN) and SSL sourcesOCSP_STAPLED/HANDSHAKE_TIME_MS/SAN_CONTAINS, all round-tripped through codegen.degradedResponseTime ≤ maxResponseTimecross-field validation.checkly testreporter (SIM-286)checkly checks get --result(SIM-291)checkly testandchecks get(can't drift).--output jsonunchanged (full result body).Testing
tscclean. Reporter refactor is behavior-neutral (guarded by the unchanged reporter snapshot).deploy+checkly testrender typed blocks and ✔/✖ assertions withReceived:(no green-on-error);checks get --resultshows per-type CERTIFICATE/BASELINE/TIMING/hop-table + the new ASSERTIONS section;checks list/stats/delete/triggerwork for the new types;--output jsonemits the full body. Prod run = 3 pass / 3 fail as configured, real 11-hop trace.🧪 Manually tested (live CLI runs)
Exercised end-to-end against a local devenv and a prod account, for SSL / gRPC / traceroute, pass + fail:
checkly test— typed result blocks + ✔/✖ assertions withReceived:, no green-on-error (prod: 3 pass / 3 fail as configured; gRPCHEALTHcarried; traceroute real 11-hop trace).checkly checks get --result— SSLCERTIFICATE+ per-ruleSECURITY BASELINE, gRPC status/health/TIMING, traceroute hop table, all with the newASSERTIONSsection; also--stats-range/--group-by location.checkly checks list(table,--output md, filters),checks stats,checks delete,trigger— all work for the three types.checkly deploy— a 35-check exhaustive set (14 SSL / 12 gRPC / 9 traceroute) covering every property, enum, and assertion source × comparator deploys clean.--output json— full result body for agents.checkly import(reverse codegen) — generates validSslMonitor/GrpcMonitor/TracerouteMonitorconstructs from existing checks. This surfaced a backend gap where the import plan dropped the SSL/gRPC per-type config (both threwreading 'mode'/'hostname'before) — fixed in monorepo #2864 — then verified end-to-end on both a devenv and a prod account: the SSL/gRPC/traceroute constructs regenerate cleanly with fullrequest.sslConfig/grpcConfigand assertions.