feat!(pypi): enable bazel downloader by default

[aignas]

Summary:

• change: Set https://pypi.org/simple as the default index.
• refactor: Leave the code for the legacy behaviour intact in case we
  need to do more work on the configuration to exclude the cases where
  the downloader should be used.
• add: add index_url configuration option for setting the defaults.
• add: add a small utility for parsing the arguments.
• fix: downloader will not be used if there is no url associated with
  the source irrespective of what the index_url setting is.
• fix: ensure all of the URLs are normalized when used.

Fixes #260
Fixes #1357
Fixes #2241
Fixes #2951

[aignas]

@dougthor42, FYI, this may be interesting to test, but it can wait until rc0 is released.

[gemini-code-assist]

Code Review

This pull request transitions the pip extension from experimental index URL attributes to standard pip arguments like --index-url and --extra-index-url, while introducing a new index_url attribute in pip.default. It includes a new argparse utility for extracting these values from requirement files and updates the hub builder logic to support these changes. Review feedback identified a critical bug in parse_requirements.bzl where the Bazel downloader could be unintentionally disabled due to incorrect argument passing. Additionally, suggestions were made to label experimental attributes as deprecated rather than no-ops to reflect their current fallback behavior and to ensure consistent URL formatting for the default index.

[rickeylev]

is this a breaking change? I see some of the experiemtnal_xxx args are doc'd as no-ops, but it looked like the code used them as a fallback.

If so, that's fine. Lets just document it and add ! to the summary line.