Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bugfix] Fix ipv6 address parsing bug #3641

Merged
merged 1 commit into from
Mar 26, 2024
Merged

Conversation

liiliiliil
Copy link
Contributor

In a pure IPv6 environment, the get_distributed_init_method function in vllm/utils.py returns a URL such as "tcp://2001:4860:4860::8888:8086", where "2001:4860:4860::8888" is the IP address. This format leads to an error when urllib.parse.urlparse attempts to parse the URL and extract the port number:

>>> urllib.parse.urlparse("tcp://2001:4860:4860::8888:8086").port
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/root/mambaforge/envs/vllm/lib/python3.10/urllib/parse.py", line 185, in port
    raise ValueError(f"Port could not be cast to integer value as {port!r}")
ValueError: Port could not be cast to integer value as '4860:4860::8888:8086'

We can resolve this issue by constructing the URL with square brackets around the IPv6 address, like so: "tcp://[2001:4860:4860::8888]:8086". With this format, the port is correctly parsed:

>>> urllib.parse.urlparse("tcp://[2001:4860:4860::8888]:8086").port
8086

This fix addresses a new bug encountered following the fix for issue #2685

@simon-mo simon-mo merged commit a979d97 into vllm-project:main Mar 26, 2024
33 checks passed
njhill added a commit to njhill/vllm that referenced this pull request Mar 26, 2024
vllm-project#3641 was a fix for ipv6 but broke ipv4 case.
@njhill
Copy link
Collaborator

njhill commented Mar 26, 2024

@liiliiliil @simon-mo this seems to have broken the non-ipv6 case, please check #3645 🙏

@liiliiliil
Copy link
Contributor Author

@njhill The ipv4 environment works for me with python 3.10. Didn't know it is a bug report here python/cpython#103848. Thank you for the additional fix. :)

>>> urllib.parse.urlparse("tcp://[127.0.0.1]:8086").port
8086

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants