POD down and ASF requires credentials

[SteffanDavies]

SciHub POD is down and ASF alternative is not working due to authentication.

Quick workaround to sentineleof until the situation is resolved, add credentials and change url:

def _download_and_write(url, save_dir="."):
    """Wrapper function to run the link downloading in parallel

    Args:
        url (str): url of orbit file to download
        save_dir (str): directory to save the EOF files into

    Returns:
        list[str]: Filenames to which the orbit files have been saved
    """
    fname = os.path.join(save_dir, url.split("/")[-1])
    if os.path.isfile(fname):
        logger.info("%s already exists, skipping download.", url)
        return [fname]

    logger.info("Downloading %s", url)
    # Fix URL
    if 's1qc.asf.alaska.edu' in url:
        url='https://urs.earthdata.nasa.gov/oauth/authorize?response_type=code&client_id=BO_n7nTIlMljdvU6kRRB3g&redirect_uri=https://auth.asf.alaska.edu/login&state='+url+'&app_type=401'
    # Add credentials
    response = requests.get(url, auth=(user, pass))
    response.raise_for_status()
    logger.info("Saving to %s", fname)
    with open(fname, "wb") as f:
        f.write(response.content)
    if fname.endswith(".zip"):
        _extract_zip(fname, save_dir=save_dir)
        # Pass the unzipped file ending in ".EOF", not the ".zip"
        fname = fname.replace(".zip", "")
    return fname

Replace user, pass with credentials.

[scottstanie]

Thanks for noticing the issue. It looks like the ASF problem may be due to high demand (based on the message that pops up https://search.asf.alaska.edu/ and that the 403 error mentions 2022-10-24 08:03:45 ERROR 403: EGRESS CUTOFF LIMIT.)

Just checking, is this redirecting the download to use earthdata.nasa.gov? Or is it still using ASF?
I'm not sure how I would incorporate this as a permanent fix, but i'll leave this up for now for anyone who needs to download orbits today

[scottstanie]

although I'll note that my attempt just now to use eof worked without changes

[SteffanDavies]

although I'll note that my attempt just now to use eof worked without changes

It worked because SciHub POD is back up again, thankfully. However, try using ASF to download and you will find a 402 error. The link is to authorize on earthdata servers before download. It's actually the link generated by the response.raise_for_status() output.

Best regards.

[SteffanDavies]

This is not a permanent fix. Just in case anybody was / will be stuck with POD servers and has to use ASF.

A permanent fix should support receiving username/password for ASF, considering POD already should support "gnssguest" as default credentials.

[scottstanie]

I'll assume that your PR has fixed this issue, thanks again @SteffanDavies

[SteffanDavies]

I'll assume that your PR has fixed this issue, thanks again @SteffanDavies

Yes the custom link was included. 👍