[DM-35482] Add support for generating the HiPS list #42
Conversation
The HiPS list accumulates the properties files for all of the HiPS data sets for a given deployment of the Rubin Science Platform, with the addition of the URL (which varies by environment and thus cannot be encoded in the properties file of the individual HiPS data set). Serve that list by getting the individual properties files of each data set, assembling, and then caching the list. Including this in datalinker may look like an odd choice, since it's unrelated to the DataLink standard. But, viewed more broadly, the function of datalinker is to provide registries, catalogs, and redirects to other services in the deployment, and HiPS is another example of that type of service. The HiPS data sets themselves will be served directly from GCS in the future, and GCS won't be able to serve the dynamic HiPS list directly, so we need some small web application to do so. Including this in datalinker seems better than writing a separate application solely for that purpose.
This is now on PyPI, so we can avoid Git dependencies and re-enable dependency hash pinning.
|
When this is ready for review (I see it's still a draft), could you attach a sample of the HiPS list file output to the PR? Thanks! |
|
My recollection from previous discussion was that it was okay to expose the HiPS list without requiring authentication, so I have done that and you can see a preview of the output at https://data-int.lsst.cloud/api/hips/list. |
|
Adding @gpdf to the review list to check that the output is as expected, and Christine as the code reviewer. |
There was a problem hiding this comment.
Examined the output. Verified that the hips_service_url values are appropriate to the specific instance. Verified that it's not behind authorization.
Not tested with Firefly's HiPS-list access yet - that will require a deployment action. Do you want that test done before closing the ticket? We could do it during co-work today?
|
A Firefly test would be great but I don't think it's a blocker for merging this. If it uncovers problems, we can always fix it later. But if you feel like testing that today and have the free time I wouldn't say no! |
| safir | ||
|
|
||
| # lsst.daf.butler is not yet on PyPI | ||
| git+https://github.com/lsst/daf_butler.git@main#lsst-daf-butler |
The HiPS list accumulates the properties files for all of the HiPS
data sets for a given deployment of the Rubin Science Platform,
with the addition of the URL (which varies by environment and thus
cannot be encoded in the properties file of the individual HiPS
data set). Serve that list by getting the individual properties
files of each data set, assembling, and then caching the list.
Including this in datalinker may look like an odd choice, since it's
unrelated to the DataLink standard. But, viewed more broadly, the
function of datalinker is to provide registries, catalogs, and
redirects to other services in the deployment, and HiPS is another
example of that type of service. The HiPS data sets themselves will
be served directly from GCS in the future, and GCS won't be able to
serve the dynamic HiPS list directly, so we need some small web
application to do so. Including this in datalinker seems better
than writing a separate application solely for that purpose.