Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection reuse in sparse index is not working #46

Closed
Shnatsel opened this issue Feb 4, 2024 · 1 comment · Fixed by #47
Closed

Connection reuse in sparse index is not working #46

Shnatsel opened this issue Feb 4, 2024 · 1 comment · Fixed by #47
Labels
bug Something isn't working

Comments

@Shnatsel
Copy link
Contributor

Shnatsel commented Feb 4, 2024

Describe the bug
When using tame_index::index::AsyncRemoteSparseIndex::krates or ...::krates_blocking, there is no connection reuse. A separate connection is created for fetching each crate. There is also a separate DNS lookup for every crate.

This is causing issues for cargo audit because some DNS servers are unhappy about getting 300 requests for the same domain at once and reject some of them, causing crate information download to fail.

To Reproduce
Steps to reproduce the behavior:

  1. Run cargo audit on a project
  2. Observe the threads being spawned for DNS lookups either with strace or by profiling it with samply

Additional context
It should be possible to mitigate this by making just one request, waiting for it to complete, and only then issuing all the other requests. They should reuse the existing connection in this case.

@Shnatsel Shnatsel added the bug Something isn't working label Feb 4, 2024
Jake-Shadle added a commit that referenced this issue Feb 7, 2024
Force connection reuse (and DNS request limitation) by performing one
full request before going wide.

Resolves: #46
@Shnatsel
Copy link
Contributor Author

Shnatsel commented Feb 7, 2024

Thanks a lot for the swift fix!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant