Passes --public-ip-address to hailctl dataproc start#14653
Passes --public-ip-address to hailctl dataproc start#14653hail-ci-robot merged 4 commits intohail-is:mainfrom
--public-ip-address to hailctl dataproc start#14653Conversation
* Add support for public-ip-address in dataproc * Use the same code style as previous code --------- Co-authored-by: Michael Franklin <illusional@users.noreply.github.com>
ehigham
left a comment
ehigham
left a comment
There was a problem hiding this comment.
Thanks for picking this change. Looks good.
cjllanwarne
left a comment
cjllanwarne
left a comment
There was a problem hiding this comment.
Small question about whether this needs to be an option, since it seems like a strict necessity. But happy to move over to 👍 if the answer is "yes"
| Opt( | ||
| help='Allow nodes to have a public IP address, and hence make requests on the public internet (default is internal-only from dataproc 2.2).' |
There was a problem hiding this comment.
From your description:
since we need internet access to install some of our dependencies
Is there any use case where someone would want to disable public IP addresses and not have the dependencies set up? If not, should this just be forced to be true (instead of being an option)?
There was a problem hiding this comment.
Good point, looks like we get the list of dependencies from hail/python/hailtop/hailctl/deploy.yaml, which is generated when we build the wheel, so there is not a way for the user to avoid using the packages we specify. Revised to just make this part of the command string instead of an option!
|
FWIW, this change worked fixed our dataproc initialisation :) |
iris-garden
changed the title
--public-ip-address arg to hailctl dataproc start and enables it by default--public-ip-address to hailctl dataproc start
5 participants
Closes #14652.
See populationgenomics#346. Thanks for the contribution @illusional!
Gives Dataproc clusters started via
hailctl dataproc startinternet access by default, since we need it to install some of our dependencies, per the error message in the linked issue.