[hailtop.batch] add default_regions to hb.Batch, improve docs #14224
Conversation
`hb.Batch` now supports `default_regions` which completes the natural hierarchy of: config, envvar, backend, batch, job. The ServiceBackend doc page also had several basic formatting issues.
| @@ -413,42 +413,96 @@ async def _async_close(self): | |||
|
|
|||
|
|
|||
| class ServiceBackend(Backend[bc.Batch]): | |||
| ANY_REGION: ClassVar[List[str]] = ['any_region'] | |||
|
|
|||
| """Backend that executes batches on Hail's Batch Service on Google Cloud. | |||
There was a problem hiding this comment.
This was actually the doc-string for the ANY_REGION class variable. By moving the class variable down, everything started rendering properly again.
| Cloud regions in which jobs may run. :attr:`.ServiceBackend.ANY_REGION` indicates jobs may | ||
| run in any region. If unspecified or ``None``, the ``batch/regions`` Hail configuration | ||
| variable is consulted. See examples above. If none of these variables are set, then jobs may | ||
| run in any region. :meth:`.ServiceBackend.supported_regions` lists the available regions. |
There was a problem hiding this comment.
The py:attribute: stuff didn't seem to have any effect, it just rendered as text. These generate correct links.
|
Please clone and build the docs ( |
|
@danking Do you want me to adopt this PR? It looks like there's a bunch of annoying errors to fix with the doctests as well as maybe a missing field in the spec? |
| >>> b = hb.Batch( | ||
| ... backend=hb.ServiceBackend( | ||
| ... gcs_bucket_allow_list=['cold-bucket', 'cold-bucket2'], | ||
| ... ), |
There was a problem hiding this comment.
| ... ), | |
| ... ) |
There was a problem hiding this comment.
This is valid Python and the preferred style of the ruff formatter we're now using.
|
I think it's passing fine now. Just a transient failure in the latest one: https://ci.hail.is/batches/8118694 |
hb.Batchnow supportsdefault_regionswhich completes the natural hierarchy of: config, envvar, backend, batch, job. I went a little hog wild with examples. I think we should have more examples everywhere!The ServiceBackend doc page also had several basic formatting issues which I addressed.