Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle async requests using Guzzle #2460

Merged
merged 15 commits into from
Jan 16, 2024
Merged

Handle async requests using Guzzle #2460

merged 15 commits into from
Jan 16, 2024

Conversation

PROFeNoM
Copy link
Contributor

@PROFeNoM PROFeNoM commented Jan 11, 2024

Description

Fixes #2444

TL;DR: Set the curl information as soon as possible, and don't overwrite it.

Shortened Explanation

The current implementation fundamentally assumes that curl_multi_info_read is called after curl_multi_exec's completion, which is not the case in Guzzle, resulting in either:

  1. Overwriting the information that was already set on a curl_exec span by dummy information;
  2. Setting dummy information on the curl_exec span.
Explanation

The existing implementation fundamentally assumes a specific execution flow, particularly that curl_multi_info_read is invoked after the completion of curl_multi_exec. However, in the context of Guzzle Promises and its tick method (which repeatedly checks only once for completed requests), this assumption is not valid, leading to unintended consequences.

The sequence of operations during a typical execution involves the following flow:

Completion and curl_getinfo

  • As the completion of curl_multi_exec occurs in CurlMultiHandler::tick under the CurlMultiHandler::execute loop, a final call to CurlMultiHandler::tick is triggered, as the queue might not be empty (array_shift left to do).
    • Therefore, an additional call to the already completed and processed handle may be done again. This handle may or may not lead to a furnished curl_getinfo, depending on whether it was previously read.
  • Alternatively, the curl_multi_exec call may complete, but Guzzle can retry handles, hence inducing the creation of an additional curl_multi_exec span. This retry scenario is the root cause of the flakiness observed in the added test.

To address these issues, the proposed changes in this pull request aim to set the curl information as soon as possible, avoiding overwriting or putting dummy information during the execution flow outlined above, i.e.:

  • Set the information either from the curl_multi_info_read or curl_multi_exec hook, whichever comes first with the curl information
  • Only overwrite the information if no network_destination is set (or the currently set information resulted in unparsable-host), which is theoretically always set when setting the curl attributes.

Notes:

  • This PR also adds snapshot testing for non-web requests :)
    • A Memcache test was changed as a result, but I guess it shouldn't matter, considering the test is still valid; it's just another environment?

Reviewer checklist

  • Test coverage seems ok.
  • Appropriate labels assigned.

@PROFeNoM PROFeNoM changed the title Alex/tmp/debug2444 Fix curl_multi_exec cURL information retrieval Jan 11, 2024
@PROFeNoM PROFeNoM changed the title Fix curl_multi_exec cURL information retrieval Handle async requests using Guzzle Jan 11, 2024
@PROFeNoM PROFeNoM self-assigned this Jan 11, 2024
@PROFeNoM PROFeNoM added tracing 🐛 bug Something isn't working labels Jan 11, 2024
@PROFeNoM PROFeNoM marked this pull request as ready for review January 11, 2024 15:40
@PROFeNoM PROFeNoM requested a review from a team as a code owner January 11, 2024 15:40
@pr-commenter
Copy link

pr-commenter bot commented Jan 11, 2024

Benchmarks

Benchmark execution time: 2024-01-12 09:15:02

Comparing candidate commit b8db1b0 in PR branch alex/tmp/debug2444 with baseline commit b032b1e in branch master.

Found 1 performance improvements and 0 performance regressions! Performance is the same for 40 metrics, 49 unstable metrics.

scenario:PDOBench/benchPDOOverheadWithDBM

  • 🟩 execution_time [-15.860µs; -8.615µs] or [-5.088%; -2.764%]

Copy link
Collaborator

@bwoebi bwoebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, hopefully we now cover everything :-)

@PROFeNoM PROFeNoM merged commit ba0cf68 into master Jan 16, 2024
569 of 572 checks passed
@PROFeNoM PROFeNoM deleted the alex/tmp/debug2444 branch January 16, 2024 11:14
@github-actions github-actions bot added this to the 0.97.0 milestone Jan 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working tracing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: seeing host-unparsable-host and "CURL request failure" in apm in curl_multi_exec context
2 participants