test(appsec): make server startup failures diagnosable and port reuse safe

[christophe-papazian]

APPSEC-69623

All 13 quarantined tests here fail in appsec_application_server, not in their own bodies. The recorded error is the "Server failed to start" assertion, and the runs with a duration cluster at 22.0–22.6s against a ~13s startup budget.

The failures are undebuggable. The assertion interpolated getattr(server_process, "stdout", None) — None for a multiprocessing.Process, a stream rather than text for a Popen — so every one reports a literal None. It now reports what is not already in the captured output: exit code (None = too slow, non-zero = died), whether the port is still taken, and the command.

Port 8050 is shared by 31 tests, these suites run serially, and teardown is best effort (SIGTERM to the process group, join(timeout=5), each step in except/pass). A gunicorn worker outliving that leaves the port taken and the next test can't bind. Now waits for the port both before starting and after tearing down.

The suites never ran. The first push was green with zero test events for any of the 13 tests: tests/appsec/appsec_utils.py matched 15 suites but none of the appsec_integrations_* ones that import it, because @appsec covers ddtrace/appsec/* source only and each suite lists just its own test directory. The second commit adds the file to the six suites whose tests import it (grep-verified: flask_tests 7 files, fastapi_tests 2, django_tests 1, plus iast_packages and iast_tdd_propagation). It now matches 21 suites; suitespec-check passes both gates.

Notes:

• The port wait tests whether the port can be bound, not whether it accepts connections. connect() reports a port free once a bound server's listen backlog fills — exactly the wedged state worth catching — and opens real connections to a live server. My first version did use connect() and wrongly reported "released" after 0.3s against a still-bound socket.
• Warns rather than raises when the port never frees; raising in teardown would turn one leak into a failure across every appsec server suite.
• No per-worker port offsets: these suites have no -n in the riotfile, so the collision is sequential, not concurrent.
• The newly triggered suites also hold quarantined tests this branch does not fix — the SCA reachability pair (fixed in test(iast): assert SCA reachability on the event that recorded the call #19671) and test_django_insecure_cookie_secure (an index_aspect bug). Deliberately left unkeyed rather than un-quarantined on a commit that would not fix them.

Blast radius: appsec_application_server backs all five server context managers, and the suitespec change widens what CI runs on such edits — both intended.

🤖 Generated with Claude Code

[cit-pr-commenter-54b7da]

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

No remaining files require a CODEOWNERS review.

[cit-pr-commenter-54b7da]

Circular import analysis

⚠️ Existing circular imports

There are 5 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.contrib.internal.pytorch._distributed -> ddtrace.contrib.internal.pytorch._rank_root -> ddtrace.contrib.internal.pytorch._distributed
ddtrace.contrib.internal.django.patch -> ddtrace.contrib.internal.django.response -> ddtrace.contrib.internal.django.patch
ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

[cit-pr-commenter-54b7da]

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 255 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 255 highest severity)

ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=134)
ddtrace.profiling.scheduler -×-> ddtrace.trace  (product:profiling -> product:tracing, score=132)
ddtrace.llmobs._integrations.anthropic -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)
ddtrace.llmobs._integrations.crewai -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=132)
ddtrace.profiling.collector.pytorch -×-> ddtrace.trace  (product:profiling -> product:tracing, score=132)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

[datadog-official]

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a487a79 | Docs | Datadog PR Page | Give us feedback!}