feat(perf-test): self-contained mTLS Locust harness + real per-device seed certs#211
Merged
Merged
Conversation
This was referenced Jul 16, 2026
Closed
Closed
doanac
reviewed
Jul 16, 2026
doanac
left a comment
Member
There was a problem hiding this comment.
None of your commits have signed-off-by or co-authored-by.
…o cert generator Existing openssl-fork based cert generation takes minutes for thousands of devices; a single in-process Go generator does 5000 ECDSA certs in ~250ms. Verified end-to-end (10 and 5000 devices): golang:1.24 builder bumped to 1.25 to match go.mod, setup.sh runs auth-init before tuf-init (TUF needs the HMAC secret auth-init creates), and the locust service's /data mount is read-write so headless runs can write their CSV/HTML report. Names the built binary and hostname "fioserver" throughout, matching cmd/server's actual binary name (see contrib/server/Dockerfile, contrib/run-local.sh) rather than an earlier prototype's "dg-sat". Signed-off-by: Eric Bode <ebode@qti.qualcomm.com> Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The admin REST API (GET /v1/devices) is a structurally different actor from mTLS devices: separate host (:8080 vs :8443), Bearer-token auth instead of client certs, and its own low, fixed call cadence — so it gets its own module and User class rather than being bolted onto DeviceUser. Sized independently via --num-admins (fixed_count), so admin traffic never displaces device registrations by stealing virtual users from -u/ --num-devices. Locust's Runner.spawn_users() overwrites every registered User class's `host` attribute with the mTLS device host right before each spawn, so AdminUser can't rely on self.host/HttpSession.base_url — list_devices always builds absolute URLs instead, including when following the pagination Link header (which the server returns as a relative path). Signed-off-by: Eric Bode <ebode@qti.qualcomm.com> Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
StealthyCoder
force-pushed
the
perf-test-pr1
branch
from
July 16, 2026 11:04
4c302ed to
714efa3
Compare
doanac
approved these changes
Jul 16, 2026
StealthyCoder
added a commit
to StealthyCoder/update-server
that referenced
this pull request
Jul 21, 2026
setup.sh calls `fioserver user-add --tokenfile ...` to seed an admin API token before the server starts, but UserAddCmd never grew that flag — it only existed on an unrelated, unmerged perf-test-idea branch. Add it back: after creating the user, generate a 30-day token scoped to the requested roles and write its value to the given file. Reported against PR foundriesio#212's stacked setup.sh, but the call was actually introduced in PR foundriesio#211 (merged into main), so this targets that gap directly rather than rebasing merged history. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
StealthyCoder
added a commit
to StealthyCoder/update-server
that referenced
this pull request
Jul 21, 2026
setup.sh calls `fioserver user-add --tokenfile ...` to seed an admin API token before the server starts, but UserAddCmd never grew that flag — it only existed on an unrelated, unmerged perf-test-idea branch. Add it back: after creating the user, generate a 30-day token scoped to the requested roles and write its value to the given file. Reported against PR foundriesio#212's stacked setup.sh, but the call was actually introduced in PR foundriesio#211 (merged into main), so this targets that gap directly rather than rebasing merged history. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
contrib/perf-test/, a self-contained Locust-based mTLS performancetest for
dg-sat: a fast in-process Go cert generator (replacingcontrib/scale-test's openssl-fork-per-device approach), a device actor(mTLS
:8443,GET /device), and an admin actor (Bearer token:8080,paginated
GET /v1/devices). This is deliberately scoped down —/config,/events, the update-check/download flow, and task tagging/named Makefiletargets are follow-up work in a stacked PR (see below), so this stays
reviewable as a first increment.
Also:
contrib/scale-test(the superseded Python/openssl harness) isremoved, and
cmd/seedis updated to generate real per-device certs insteadof a shared dummy pubkey, per review feedback that the two shouldn't
duplicate cert-generation logic.
Why a new folder instead of extending
scale-testcontrib/scale-test's cert generation forksopensslper device — at scalethat's thousands of process spawns, minutes just to get to a runnable state.
contrib/perf-test/gen-certsis a single Go binary doing all keygen+signingin-process: 5000 ECDSA P-256 device certs in ~250ms.
What it covers
:8443)GET /device:8080)GET /v1/devices(paginated)cmd/seed now shares cert generation with gen-certs
cmd/seed'sseedDevices()used to register every seeded device with onehardcoded dummy RSA pubkey — fine for populating the UI, but nothing that
could ever mTLS-authenticate. This PR extracts
gen-certs's CA/leafgen-certsandcmd/seedcall it:cmd/seednow issues each device a real ECDSA leaf cert from a CAgenerated lazily on first use, appended to
cas.pem(not overwritten, socontrib/run-local.sh's own dev TLS leaf-as-CA keeps working alongsideit). Cert+key land under
datadir/certs/seed-devices/<uuid>/client.pem.seed-device-%05dscheme— only the pubkey/cert material changed, not device identity.
seedDevicesagainst an existing datadir skips cert(re)generation for devices that already have one on disk, so it doesn't
silently invalidate a cert still valid in the DB. Covered by
TestSeedDevicesIdempotent.Design notes
self.host. Locust's runneroverwrites every registered
Userclass'shostwith the mTLS devicehost immediately before each spawn —
AdminUserbuilds full URLsinstead, including when following the paginated
Linkheader (theserver returns it as a relative path).
storage/devcertsis crypto-only, no file I/O — callers keep fullcontrol over on-disk layout and concurrency (
gen-certs's worker poolover
runtime.NumCPU()is unchanged).Testing
go build ./...,go test ./...,go vet ./...,gofmt -l .all clean.storage/devcertshas its own tests: CA generation, leaf issuance,signature verification against the issuing CA.
cmd/seed'sTestSeedextended to assert seeded devices get distinctreal pubkeys (not a shared dummy), a real cert file is written, and the
cert chains to the CA appended in
cas.pem. NewTestSeedDevicesIdempotentcovers re-running against an existing datadir.
./contrib/run-local.shend-to-end: confirmedcas.pemcontains both the original dev TLS leaf and the new seed CA, and a seeded
device's cert verifies against it via
openssl verify.contrib/perf-test: builtgen-certs, ran it standalone (device certsverify against
cas.pem), and confirmeddocker compose configparses.Follow-up
A stacked PR (targeting this branch) adds
/config,/events, thecheck-for-update/download flow, per-task tagging, and Makefile/profile/scene
ergonomics. Kept separate per review feedback that the original combined PR
was too large to review as one unit.