Skip to content

Add AuTest for connect_attempts rr_retries and max_retries#12932

Draft
masaori335 wants to merge 2 commits intoapache:masterfrom
masaori335:asf-master-hostdb-autest
Draft

Add AuTest for connect_attempts rr_retries and max_retries#12932
masaori335 wants to merge 2 commits intoapache:masterfrom
masaori335:asf-master-hostdb-autest

Conversation

@masaori335
Copy link
Contributor

Covers combination of proxy.config.http.connect_attempts_rr_retries and proxy.config.http.connect_attempts_max_retries.

The connect_attempts_max_retries_down_server will be covered by #12922

@masaori335 masaori335 added this to the 11.0.0 milestone Mar 1, 2026
@masaori335 masaori335 self-assigned this Mar 1, 2026
@masaori335 masaori335 force-pushed the asf-master-hostdb-autest branch from ab39cfa to 3b069bc Compare March 2, 2026 01:41
@masaori335 masaori335 requested a review from Copilot March 2, 2026 01:41
@masaori335 masaori335 changed the title Add AuTest for connect_attempts Add AuTest for connect_attempts rr_retries and max_retries Mar 2, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds new AuTest coverage for origin connect-attempt retry behaviors (round-robin retries vs max retries) and extends the ATS replay test harness to support DNS record injection and error.log validation via replay YAML.

Changes:

  • Added three new Proxy Verifier replay scenarios covering proxy.config.http.connect_attempts_rr_retries and proxy.config.http.connect_attempts_max_retries.
  • Added new dns/connect_attempts.test.py to run the new replay tests and gold files for error.log expectations.
  • Extended tests/gold_tests/autest-site/ats_replay.test.ext to support DNS records injection and error_log validation (including gold-file based validation).

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
tests/gold_tests/dns/replay/connect_attempts_rr_retries.replay.yaml New replay scenario for round-robin retry behavior and down-server cache behavior.
tests/gold_tests/dns/replay/connect_attempts_rr_no_retry.replay.yaml New replay scenario covering baseline behavior with no retries.
tests/gold_tests/dns/replay/connect_attempts_rr_max_retries.replay.yaml New replay scenario covering connect_attempts_max_retries behavior.
tests/gold_tests/dns/gold/connect_attempts_rr_retries_error_log.gold Expected error.log output for rr-retries scenario.
tests/gold_tests/dns/gold/connect_attempts_rr_no_error_log.gold Expected error.log output for no-retry scenario.
tests/gold_tests/dns/gold/connect_attempts_rr_max_retries_error_log.gold Expected error.log output for max-retries scenario.
tests/gold_tests/dns/connect_attempts.test.py Test driver that runs the three new replay tests.
tests/gold_tests/autest-site/ats_replay.test.ext Adds error_log validation support and DNS records wiring for replay-based tests.

Comment on lines +293 to +298
# wait for error log
if log_validation != None and log_validation['error_log'] != None:
wait_for_log = obj.AddTestRun('Wait for logs of ' + ats_config.get('name', 'ts'))
wait_for_log.Processes.Default.Command = (
os.path.join(obj.Variables.AtsTestToolsDir, 'condwait') + ' 60 1 -f ' + os.path.join(ts.Variables.LOGDIR, 'error.log'))
wait_for_log.Processes.Default.ReturnCode = 0
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In ATSReplayTest, the guard if log_validation != None and log_validation['error_log'] != None: can raise a KeyError when autest.ats.log_validation exists but does not define error_log (e.g. many existing replay tests only use traffic_out / diags_log). Please switch to a safe lookup (e.g. error_log_cfg = (log_validation or {}).get('error_log')) and check that value instead.

Copilot uses AI. Check for mistakes.
# limitations under the License.

#
# This replay file assumes that caching is enabled.
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header comment says caching is enabled, but this replay config explicitly sets process_config.enable_cache: false. Please update or remove the comment so it matches the test configuration.

Suggested change
# This replay file assumes that caching is enabled.
# This replay file runs with caching disabled.

Copilot uses AI. Check for mistakes.
proxy-response:
status: 502

# request expected hit down_server cache and immidiately get 500
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling: "immidiately" should be "immediately" in this comment (and consider rewording to a full sentence for clarity).

Suggested change
# request expected hit down_server cache and immidiately get 500
# This request is expected to hit the down_server cache and immediately receive a 500 response.

Copilot uses AI. Check for mistakes.
# limitations under the License.

#
# This replay file assumes that caching is enabled.
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header comment says caching is enabled, but this replay config explicitly sets process_config.enable_cache: false. Please update or remove the comment so it matches the test configuration.

Suggested change
# This replay file assumes that caching is enabled.
# This replay file runs with ATS caching disabled.

Copilot uses AI. Check for mistakes.
proxy-response:
status: 502

# request expected hit down_server cache and immidiately get 500
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling: "immidiately" should be "immediately" in this comment (and consider rewording to a full sentence for clarity).

Suggested change
# request expected hit down_server cache and immidiately get 500
# The request is expected to hit the down_server cache and immediately receive a 500 response.

Copilot uses AI. Check for mistakes.
# limitations under the License.

#
# This replay file assumes that caching is enabled.
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The header comment says caching is enabled, but this replay config explicitly sets process_config.enable_cache: false. Please update or remove the comment so it matches the test configuration.

Suggested change
# This replay file assumes that caching is enabled.
# This replay file assumes that caching is disabled.

Copilot uses AI. Check for mistakes.
proxy-response:
status: 502

# request expected hit down_server cache and immidiately get 500
Copy link

Copilot AI Mar 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling: "immidiately" should be "immediately" in this comment (and consider rewording to a full sentence for clarity).

Suggested change
# request expected hit down_server cache and immidiately get 500
# The request is expected to hit the down_server cache and immediately receive a 500 response.

Copilot uses AI. Check for mistakes.
@bryancall bryancall added the DNS label Mar 2, 2026
@bryancall bryancall self-requested a review March 2, 2026 22:39
@masaori335
Copy link
Contributor Author

Somehow, the connect_attempts_rr_max_retries test is keep trying 0.0.0.1 even after it's marked "down" instead of round robin to 0.0.0.2 in this env.

20260302.01h50m26s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=0 url='http://backend.example.com:62299/path/'
20260302.01h50m28s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=1 url='http://backend.example.com:62299/path/'
20260302.01h50m30s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=2 url='http://backend.example.com:62299/path/'
20260302.01h50m30s CONNECT : ETIMEDOUT [110] connecting to 0.0.0.1:62299 for host='example.com' url='http://backend.example.com:62299/path/' fail_count='1' marking down
20260302.01h50m31s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=0 url='http://backend.example.com:62299/path/'
20260302.01h50m33s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=1 url='http://backend.example.com:62299/path/'
20260302.01h50m34s CONNECT: attempt fail [CONNECTION_ERROR] to 0.0.0.1:62299 for host='example.com' connection_result=ETIMEDOUT [110] error=ETIMEDOUT [110] retry_attempts=2 url='http://backend.example.com:62299/path/'

@masaori335 masaori335 marked this pull request as draft March 3, 2026 00:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

3 participants