Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

[networking] Fix various bugs in socks proxy implementation #8065

Merged
merged 12 commits into from Sep 18, 2023

Conversation

coletdjnz
Copy link
Member

@coletdjnz coletdjnz commented Sep 9, 2023

Fixes #7959
Fixes ytdl-org/youtube-dl#15368

  • Fixed support for IPv6 socks proxies
  • Fixed support for IPv6 over socks5
  • Fixed --source-address is not obeyed for either socks4 and socks5
  • Fixed Using an IP instead of a domain name for a destination does not work with socks4a
Template

Before submitting a pull request make sure you have:

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

  • I am the original author of this code and I am willing to release it under Unlicense
  • I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

Copilot Summary

馃 Generated by Copilot at 6801a5f

Summary

馃悰馃寪馃И

This pull request enhances the socks proxy support in yt-dlp by adding IPv6 and source_address capabilities, fixing bugs, and improving the testing and implementation of the proxy handlers. It modifies the files test/test_socks.py, yt_dlp/networking/_helper.py, yt_dlp/networking/_urllib.py, and yt_dlp/socks.py.

We are the proxy handlers, we fix the bugs and flaws
We create connections, we set the source and protocol
We support IPv6, we handle the address families
We are the proxy handlers, we rock the socks off the world

Walkthrough

  • Add support for source_address and IPv6 destination addresses in socks proxies (link, link, link, link, link, link, link, link)
  • Define a new create_connection function in yt_dlp/networking/_helper.py that allows specifying the source_address for the socket connection (link, link)
  • Use the create_connection function in yt_dlp/networking/_urllib.py to simplify the _create_http_connection function and to add the _create_connection attribute to the SocksConnection class (link, link, link)
  • Modify the _resolve_address, _setup_socks4, and _setup_socks5 methods in yt_dlp/socks.py to handle both IPv4 and IPv6 addresses and to pass the address family information to the proxy (link, link, link, link)
  • Fix a bug in the socks4a implementation and remove the xfail marks from the corresponding tests (link, link, link)
    • Add a condition to check if the ipaddr is equal to the SOCKS4_DEFAULT_DSTIP in the _setup_socks4 method in yt_dlp/socks.py, and only append the domain name to the packet in that case (link)
    • Remove the xfail mark from the test_socks4a_ipv4_target test case in test/test_socks.py, and modify the assertion to accept both the ipv4_address and the domain_address as valid (link, link)
  • Remove the xfail marks from the tests that were failing due to the lack of support for source_address and IPv6 in the socks proxies, and modify some assertions to make them more robust (link, link, link, link, link, link)
    • Remove the xfail mark from the test_ipv4_client_source_address test case in test/test_socks.py, which was failing for both socks4 and socks5 proxies (link, link)
    • Remove the xfail mark from the test_ipv6_socks4_proxy test case in test/test_socks.py, which was failing for the socks4 proxy (link)
    • Remove the xfail mark from the test_socks5_ipv6_destination test case in test/test_socks.py, which was failing for the socks5 proxy, and remove the port check from the assertion (link, link)
    • Modify the assertion in the test_socks5_domain_target test case in test/test_socks.py, to accept both the ipv4_address and the ipv6_address as valid (link)
  • Move the sleep argument from the class level to the handle function level in the Socks4ProxyHandler and Socks5ProxyHandler classes in test/test_socks.py, and modify the timeout value in the test_timeout test case (link, link, link, link, link)
    • Move the sleep argument from the init method to the handle method in the Socks4ProxyHandler and Socks5ProxyHandler classes, which are used to simulate a proxy server for the tests (link, link, link, link)
    • Modify the timeout value in the test_timeout test case in test/test_socks.py, to make it more likely to trigger the timeout exception (link)

@coletdjnz coletdjnz added the bug Bug that is not site-specific label Sep 9, 2023
@coletdjnz coletdjnz marked this pull request as draft September 9, 2023 00:08
@coletdjnz coletdjnz marked this pull request as ready for review September 9, 2023 00:30
yt_dlp/networking/_helper.py Outdated Show resolved Hide resolved
yt_dlp/networking/_helper.py Show resolved Hide resolved
yt_dlp/networking/_helper.py Outdated Show resolved Hide resolved
yt_dlp/networking/_urllib.py Show resolved Hide resolved
yt_dlp/socks.py Show resolved Hide resolved
yt_dlp/networking/_helper.py Outdated Show resolved Hide resolved
yt_dlp/networking/_helper.py Outdated Show resolved Hide resolved
yt_dlp/networking/_urllib.py Show resolved Hide resolved
coletdjnz and others added 4 commits September 15, 2023 21:14
@coletdjnz coletdjnz requested review from Grub4K and bashonly and removed request for pukkandan September 16, 2023 21:22
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
@coletdjnz coletdjnz merged commit 20fbbd9 into yt-dlp:master Sep 18, 2023
16 checks passed
@coletdjnz coletdjnz deleted the fix/socks-proxies branch September 18, 2023 07:33
aalsuwaidi pushed a commit to aalsuwaidi/yt-dlp that referenced this pull request Apr 21, 2024
- Fixed support for IPv6 socks proxies
- Fixed support for IPv6 over socks5
- Fixed --source-address not being obeyed for socks4 and socks5
- Fixed socks4a when the destination address is an IPv4 address

Closes yt-dlp#7959
Fixes ytdl-org/youtube-dl#15368

Authored by: coletdjnz
Co-authored-by: Simon Sawicki <accounts@grub4k.xyz>
Co-authored-by: bashonly <bashonly@bashonly.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Bug that is not site-specific
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SOCKS proxy mechanism does not support IPv6 [networking] Bugs in socks proxy implementation
3 participants