Skip to content

Remove requests.Session test patches from conftest.py files#23314

Merged
mwdd146980 merged 16 commits intomwdd146980/httpx-migration-basefrom
mwdd146980/step4-conftest-session-patches
May 1, 2026
Merged

Remove requests.Session test patches from conftest.py files#23314
mwdd146980 merged 16 commits intomwdd146980/httpx-migration-basefrom
mwdd146980/step4-conftest-session-patches

Conversation

@mwdd146980
Copy link
Copy Markdown
Contributor

@mwdd146980 mwdd146980 commented Apr 14, 2026

Motivation

Test fixtures that patch requests.Session directly will silently break when integrations switch from requests to httpx. The mock_http fixture intercepts HTTP at the AgentCheck.http level, making tests transport-agnostic.

Approach

  • OM v2 / AgentCheck integrations: Replaced mocker.patch('requests.Session.get', ...) with mock_http fixture + MockHTTPResponse.
  • OM v1 / Prometheus integrations: The v1 mixin's get_http_handler() constructs its own RequestsWrapper, bypassing self.http. Fixtures patch get_http_handler via mocker.patch to return mock_http. No production code changes. Shared fixtures mock_openmetrics_http / mock_prometheus_http extracted to the ddev pytest plugin.

Notable changes

  • nutanix: Larger refactor than the other integrations. In conftest.py, mock_http_get was rewritten to dispatch URLs via MockHTTPResponse instead of building mocker.Mock(spec=Response) per branch.
  • yarn test_ssl_verification: Old test asserted requests-library SSL behavior (raises SSLError on bad cert), which is already covered in datadog_checks_base/tests/. The mocked_bad_cert_request fixture was removed.
  • haproxy fixture scope: Narrowed haproxy_mock, haproxy_mock_evil, and haproxy_mock_enterprise_version_info from scope="module" to function scope, since mock_http is function-scoped and pytest forbids broader-scoped fixtures from depending on narrower ones. These are mock-config fixtures only so runtime impact is negligible.

Verification

# All target integrations
for intg in temporal fluxcd dcgm impala nutanix yarn external_dns \
  datadog_cluster_agent scylla prometheus cilium gitlab_runner gitlab haproxy; do
  ddev test -fs $intg && ddev --no-interactive test $intg
done
# Regression (get_http_handler consumers)
for intg in kube_scheduler kube_controller_manager; do
  ddev --no-interactive test $intg
done

All target integrations pass. Cilium has 2 pre-existing legacy test failures (confirmed on clean branch).

Copy link
Copy Markdown
Contributor Author

mwdd146980 commented Apr 14, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 14, 2026

Codecov Report

❌ Patch coverage is 96.32353% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.90%. Comparing base (7e8968a) to head (8a139cc).
⚠️ Report is 1 commits behind head on mwdd146980/httpx-migration-base.

❗ There is a different number of reports uploaded between BASE (7e8968a) and HEAD (8a139cc). Click for more details.

HEAD has 1 upload less than BASE
Flag BASE (7e8968a) HEAD (8a139cc)
2 1
Additional details and impacted files
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-prod-us1-5
Copy link
Copy Markdown

datadog-prod-us1-5 Bot commented Apr 14, 2026

Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
Patch Coverage: 96.32%
Overall Coverage: 75.77% (-11.49%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 8a139cc | Docs | Datadog PR Page | Give us feedback!

@mwdd146980 mwdd146980 force-pushed the mwdd146980/step3e-conftest-migration branch from 14387fd to 33ed764 Compare April 14, 2026 20:29
@mwdd146980 mwdd146980 force-pushed the mwdd146980/step4-conftest-session-patches branch from 4cbf0d3 to 39292db Compare April 14, 2026 20:29
@mwdd146980 mwdd146980 changed the title Migrate temporal, fluxcd, dcgm, impala conftest.py to mock_http Remove requests.Session test patches from conftest.py files Apr 14, 2026
@mwdd146980 mwdd146980 force-pushed the mwdd146980/step3e-conftest-migration branch from 33ed764 to e310970 Compare April 15, 2026 19:44
@mwdd146980 mwdd146980 force-pushed the mwdd146980/step4-conftest-session-patches branch from 14a4779 to 3bc04d2 Compare April 15, 2026 19:44
@mwdd146980 mwdd146980 force-pushed the mwdd146980/step3e-conftest-migration branch from e310970 to c22010a Compare April 20, 2026 13:09
dd-agent-integrations-bot[bot] and others added 13 commits April 29, 2026 18:03
Replace requests.Session.get patches with mock_http fixture for 4
OpenMetricsBaseCheckV2 integrations. MockHTTPResponse natively supports
iter_lines() and content, removing the need for MagicMock wrappers.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace mocker.patch('requests.Session.get') with mock_http fixture.
Replace all mocker.Mock() responses with MockHTTPResponse(json_data=...).
107 unit tests pass; 10+ URL handlers with pagination and time-based
filtering preserved unchanged.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove the requests.Session.get patch from yarn conftest.py.
Convert test_ssl_verification from an end-to-end SSL behavior test
to a config assertion test that verifies ssl_verify propagates to
http.options['verify']. Remove unused mock/patch/SSLError imports.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Change get_http_handler() in both openmetrics/mixins.py and
prometheus/mixins.py to return self.http instead of constructing
a new RequestsWrapper, allowing mock_http to intercept v1 mixin
HTTP calls. Migrate 8 conftest.py files from requests.Session
patches to mock_http + MockHTTPResponse.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The production mixin change (returning self.http from get_http_handler)
broke kubelet's per-endpoint TLS isolation and prometheus E2E tests.
Revert to RequestsWrapper construction in production. Instead, patch
get_http_handler directly in each conftest fixture via mocker.patch
to return mock_http — same decoupling result, zero production risk.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add mock_openmetrics_http and mock_prometheus_http to pytest plugin,
  centralizing the get_http_handler patch in one place
- Update 7 conftest files to use mock_openmetrics_http, 1 to use
  mock_prometheus_http (eliminates repeated magic string)
- Remove unnecessary get_http_handler patch from 3 haproxy fixtures
  (legacy check uses self.http.get directly, not the OM mixin)
- Fix gitlab version endpoint: json_data=json.loads(text_data) so
  .json() returns a dict instead of a string (pre-existing bug)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rewrite 8 nutanix test_retry.py tests to use mock_http_get instead of
  dead requests.Session.get patches (MockHTTPResponse + HTTPStatusError)
- Update changelog to include mock_openmetrics_http/mock_prometheus_http
- Remove unreachable /-/health duplicate in gitlab conftest

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@mwdd146980 mwdd146980 changed the base branch from mwdd146980/httpx-migration-base to graphite-base/23314 April 29, 2026 22:03
@mwdd146980 mwdd146980 force-pushed the graphite-base/23314 branch from 9709aed to 7e8968a Compare April 29, 2026 22:10
@mwdd146980 mwdd146980 force-pushed the mwdd146980/step4-conftest-session-patches branch from 362e3dd to 573ae6c Compare April 29, 2026 22:10
@mwdd146980 mwdd146980 changed the base branch from graphite-base/23314 to mwdd146980/httpx-migration-base April 29, 2026 22:10
Comment thread yarn/tests/test_yarn.py
mwdd146980 and others added 3 commits May 1, 2026 11:14
Replaces test_tls_verify_config_propagates with test_ssl_verification.
Two-phase test mirrors original shape: SSLError -> CRITICAL service
check, then verify=False -> 4 OK service checks. Uses mock_http instead
of patching requests.Session.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restore original try/except/else body. Only diff vs pre-migration:
fixture arg (mocked_bad_cert_request -> mock_http) and side_effect
setup lines. No pytest.raises rewrite.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment removal was out of scope for this PR.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts Bot commented May 1, 2026

Validation Report

All 20 validations passed.

Show details
Validation Description Status
agent-reqs Verify check versions match the Agent requirements file
ci Validate CI configuration and Codecov settings
codeowners Validate every integration has a CODEOWNERS entry
config Validate default configuration files against spec.yaml
dep Verify dependency pins are consistent and Agent-compatible
http Validate integrations use the HTTP wrapper correctly
imports Validate check imports do not use deprecated modules
integration-style Validate check code style conventions
jmx-metrics Validate JMX metrics definition files and config
labeler Validate PR labeler config matches integration directories
legacy-signature Validate no integration uses the legacy Agent check signature
license-headers Validate Python files have proper license headers
licenses Validate third-party license attribution list
metadata Validate metadata.csv metric definitions
models Validate configuration data models match spec.yaml
openmetrics Validate OpenMetrics integrations disable the metric limit
package Validate Python package metadata and naming
readmes Validate README files have required sections
saved-views Validate saved view JSON file structure and fields
version Validate version consistency between package and changelog

View full run

@mwdd146980 mwdd146980 requested a review from steveny91 May 1, 2026 15:47
@mwdd146980 mwdd146980 merged commit 242c4ad into mwdd146980/httpx-migration-base May 1, 2026
50 of 79 checks passed
@mwdd146980 mwdd146980 deleted the mwdd146980/step4-conftest-session-patches branch May 1, 2026 15:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment