[rh:websockets] Migrate websockets to networking framework

[coletdjnz]

#8361 blocks this. This requires websockets>=12 which is Py3.8+ only.

Moves our Websocket support (in the form of WebSocketsWrapper) into its own RequestHandler. This way extractors do not need to directly interact with the library.

As part of this I have also created a standard websocket framework - where the standard websocket response has send and recv functions.

Improvements

• Deprecates and removes the Websocket wrapper from use (fixes WebsocketWrapper deprecation warning: remove loop argument #8439)
• Unified websocket request interface not tied to any websocket library
• Integration with yt-dlp networking options: cookies, headers, socket timeout, source address, print traffic, mtls, legacy server connect, no check cert, certifi, and proxies (socks proxy only) etc.

Known issues

• No http proxy support. We'll need to add support for websocket-client to have proxies (among other things)
• Older versions of websockets will not be supported as we move to the synchronous client.
• Closing can hang due to the closing handshake. For now I've set close_timeout to 0, which isn't ideal.
• Test suite improvements. For now I've just put websocket tests in another file.

Out of scope

• Detecting if we can support websockets ahead of time and other helpful information for users.
• Rewriting websocket fragment downloader to use the websocket handlers
• Rewriting test suite

Template

Before submitting a pull request make sure you have:

• At least skimmed through contributing guidelines including yt-dlp coding conventions
• Searched the bugtracker for similar pull requests
• Checked the code with flake8 and ran relevant tests

In order to be accepted and merged into yt-dlp each piece of code must be in public domain or released under Unlicense. Check all of the following options that apply:

• I am the original author of this code and I am willing to release it under Unlicense
• I am not the original author of this code but it is in public domain or released under Unlicense (provide reliable evidence)

What is the purpose of your pull request?

• Fix or improvement to an extractor (Make sure to add/update tests)
• New extractor (Piracy websites will not be accepted)
• Core bug fix/improvement
• New feature (It is strongly recommended to open an issue first)

Copilot Summary

🤖 Generated by Copilot at 04f8749

Summary

🌐🛠️🧪

This pull request adds support for websockets request handler and response in the networking module, and simplifies the websockets handling in some extractors. It adds the files _websockets.py and websocket.py that implement the websockets logic, and modifies the files test_networking.py, fc2.py, niconico.py, _legacy.py, _utils.py, and YoutubeDL.py to use the new websockets classes and methods. It also fixes a bug in the debug output of the downloader.

We're sailing on the web with websockets in our hand
We've moved the wrapper class to a new file and command
We've fixed the bugs and tests and made the code more neat
So heave away, me hearties, on the count of three

Walkthrough

• Implement a websockets request handler and response for the downloader (link, link)
• Use the websockets request handler and response in the FC2LiveIE and NiconicoLiveIE extractors, and remove the dependency on the websockets library (link, link, link, link, link, link)
• Move the WebSocketsWrapper class from the _utils.py file to the _websockets.py file, and import it in the _legacy.py file for backward compatibility (link, link, link)
• Add tests for the websockets request handler and response in the test_networking.py file, and test the validation of different URL schemes and options (link, link, link, link, link, link)
• Print the request handlers used by the downloader in the debug header, and fix a bug in accessing the handlers attribute (link)

[pukkandan]

• older versions of websockets will not be supported as we move to the synchronous client.

Make sure to update requirements.txt. We could also try to detect in dependencies.py and issue warning?

[pukkandan]

We want to error early

[coletdjnz]

I'll add an equivalent error in ydl.urlopen() when the unsupported request errors come back. Cause extractors shouldn't know about the underlying libs, and we may support many websocket libs in the future.

[pukkandan]

It will then still error only after the initial webpage request. Checking dependency and erroring before any request is better. We do the same for cryptodome deps

[coletdjnz]

I'll add this to the "nice to have" list that we can work on later. It's not required for this to be merged imo. It requires a bit of work which is being started in #7595