Validating multi-burst groups shouldn't require locally downloaded SLCs

[Alex-Lewandowski]

I am working on adding multi-burst support to the new asf_search.SBASNetwork class. I would like to verify multi-burst groups during SBASNetwork initialization so that HyP3 users don't order huge stacks of invalid multi-burst jobs that fail and burn through their monthly quotas. I'd like to avoid writing my own validation (like this), which would duplicate code elsewhere and could diverge over time. It would be great to stick to a single source of truth.

S1 multi-burst validation occurs in burst2safe, specifically with the Safe.check_group_validity method, which calls the Swath.check_burst_group_validity method. To create a Safe object, you must include a list of BurstInfo objects. To create a BurstInfo object, you must have local burst data on hand. If I understand correctly, you can only check multi-burst group validity if you have downloaded SLCs.

This doesn't work for a tool that simply performs a search and doesn't download data. Given this current flow, hyp3-isce2 needs to download SLCs before it is able to recognize that a multi-burst job is invalid, which seems like a waste of resources. If we separate multi-burst validation from burst2safe or handle it in functions that don't require Safe objects, we could open up multi-burst validation to more tools. It would work for SBASNetwork and when Vertex adds multi-burst ordering support, it could flag or disallow invalid multi-burst groups.

[jhkennedy]

I raised similar concerns about needing to download data before deciding things in #250, so I def. agree it's worth re-thinking how this package is structured.

In this case, I believe you can check burst validity without downloading; you just need asf_search.Products.S1BurstProduct.S1BurstProduct instance objects.

In the burst2safe workflow, this line does a search and returns a list of S1BurstProduct instance objects:
https://github.com/ASFHyP3/burst2safe/blob/develop/src/burst2safe/burst2safe.py#L55

Then, this line gets all the burst_infos:
https://github.com/ASFHyP3/burst2safe/blob/develop/src/burst2safe/burst2safe.py#L56

Notably, get_burst_infos is effectively just a pass-through function to create_burst_info, which is just parsing info out of S1BurstProduct instances and setting up Path objects -- no data or files are actually written yet, it's just where they will be written:
https://github.com/ASFHyP3/burst2safe/blob/develop/src/burst2safe/utils.py#L72-L90

It isn't until the download_bursts call later that data is actually localized:
https://github.com/ASFHyP3/burst2safe/blob/develop/src/burst2safe/burst2safe.py#L63

Nicely, the static Safe.check_group_validity(burst_infos) class method is called before the burst files are downloaded:
https://github.com/ASFHyP3/burst2safe/blob/develop/src/burst2safe/burst2safe.py#L60

And for some reason, it is also checked again when the Safe class is actually initialized, after download:
https://github.com/ASFHyP3/burst2safe/blob/develop/src/burst2safe/safe.py#L35

So it should be totally possible to do your search and get your list of S1BurstProduct instances outside of burst2safe in your network class, and then make two calls to util.create_burst_infos and Safe.check_group_validity without actually downloading anything.

[Alex-Lewandowski]

Thanks, @jhkennedy! I will give that a shot and report back.

[Alex-Lewandowski]

My goal was to check the group validity before even searching for a single S1BurstProduct and that was possible! So, the premise of this discussion was based in my lack of understanding. I agree that it still might be worth restructuring some of this. I had to use some do-nothing placeholder args in BurstInfo, which is probably a sign that the interface abstraction is slightly off, but I can proceed with multi-burst updates for asf_search.SBASNetwork using burst2safe in its current form. Thanks for your help, @jhkennedy!

from pathlib import Path

from burst2safe.safe import Safe
from burst2safe.utils import BurstInfo

multiburst_dict = {
    "173_370305": ("IW1", "IW2", "IW3"),
    "173_370306": ("IW1", "IW2", "IW3"),
    "173_370307": ("IW1", "IW2", "IW3")
}

burst_infos = [
    BurstInfo(
        granule=f"{k}_{swath}",
        slc_granule=None,
        swath=swath,
        polarization="",
        burst_id=int(k.split('_')[1]),
        burst_index=0,
        direction="",
        absolute_orbit=0,
        relative_orbit=int(k.split('_')[0]),
        date=None,
        data_url="",
        data_path=None,
        metadata_url=None,
        metadata_path=Path.cwd(), # a do nothing path if simply checking group validity
    )
    for k, swaths in multiburst_dict.items()
    for swath in swaths
]

Safe.check_group_validity(burst_infos)

[jhkennedy]

Ooh, I agree being able to check best validity from just the "relative" burst IDs and polarizations would be really handy. Glad this works, but yes we can expose that a bit nicer