Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Determine how many granules exist over this time and area of interest #10

Closed
Fumeng-zh opened this issue Jun 28, 2023 · 6 comments
Closed

Comments

@Fumeng-zh
Copy link

I run the Customize and Access NSIDC Data to download the data, but an error occured in the part of Determine how many granules exist over this time and area of interest.
微信图片_20230628152039
微信图片_20230628152118
Is there something wrong? all the parameters are default.

@MattF-NSIDC
Copy link
Member

Can you share the value of response.ok and results?

Possibly we need to add a check that the requests.get() was successful before proceeding.

@Fumeng-zh
Copy link
Author

The results is empty, is there somthing wrong with the granule_search_url = 'https://cmr.earthdata.nasa.gov/search/granules', because this link returns an error.

@MattF-NSIDC
Copy link
Member

MattF-NSIDC commented Jun 29, 2023

That value for granule_search_url looks right. When you click that link, you're getting an error because you haven't passed any search parameters. The code above is doing some extra work to pass those parameters to the server with requests.get(..., params=search_params, ...) and that is causing the request URL to be modified to add the key-value pairs in search_params as URL query parameters. Example:

>>> response = requests.get('https://example.com', params={'foo': 'bar'})
>>> response.request.url
'https://example.com/?foo=bar'
>>> response.ok
True
>>> response.status_code
200
>>> response.content
b'<!doctype html>\n<html>\n<head>\n    <title>Example Domain</title>\n\n    <meta charset="utf-8" />\n    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />\n    <meta name="viewport" content="width=device-width, initial-scale=1" />\n    <style type="text/css">\n    body {\n        background-color: #f0f0f2;\n        margin: 0;\n        padding: 0;\n        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;\n        \n    }\n    div {\n        width: 600px;\n        margin: 5em auto;\n        padding: 2em;\n        background-color: #fdfdff;\n        border-radius: 0.5em;\n        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n    }\n    a:link, a:visited {\n        color: #38488f;\n        text-decoration: none;\n    }\n    @media (max-width: 700px) {\n        div {\n            margin: 0 auto;\n            width: auto;\n        }\n    }\n    </style>    \n</head>\n\n<body>\n<div>\n    <h1>Example Domain</h1>\n    <p>This domain is for use in illustrative examples in documents. You may use this\n    domain in literature without prior coordination or asking for permission.</p>\n    <p><a href="https://www.iana.org/domains/example">More information...</a></p>\n</div>\n</body>\n</html>\n'

Can you please provide:

  • response.request.url: The actual URL the code is querying
  • response.ok: Whether the query was successful
  • response.content: The raw content received from the server. If the content is not JSON (as in the example above), the json.loads step would fail. Since that didn't happen, I think you got valid JSON, but the JSON contains an errors key instead of a feed key.

I think the issue may be with the data you entered; I was able to reproduce this issue by specifying an invalid bounding box.

@asteiker I think we need a validity check before trying to access results['feed']:

    # or `if 'errors' in results:`
    if 'feed' not in results:
        # An error occurred, handle!
        # should we check before json.loads() and print `response.content` instead of `results`?
        # That way if we receive invalid JSON, we can see what the response looks like instead
        # of failing with:
        #     json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
        # Maybe this isn't a realistic case to be concerned with.
        print(f'An error occurred: {results["errors"]}')
        print(f'Request URL: {response.request.url}')
        break # or raise? ¯\_(ツ)_/¯

With this new check, the code outputs a less cryptic message for my invalid input data:

An error occurred: ['[-180, -90, 180, 90] is not a valid URL encoded bounding_box']
Request URL: https://cmr.earthdata.nasa.gov/search/granules?short_name=ATL03&version=006&temporal=2021-01-01T00%3A00%3A00Z%2C2022-01-01T00%3A00%3A00Z&page_size=100&page_num=1&bounding_box=-180%2C+-90%2C+180%2C+90

@Fumeng-zh
Copy link
Author

Fumeng-zh commented Jun 29, 2023 via email

@MattF-NSIDC
Copy link
Member

Great! Thanks for surfacing this issue :) I'm going to close this issue and open a new one for the error handling.

@andypbarrett
Copy link

This search should be replaced with earthaccess. I have a candidate workflow here

I've added a class to handle Service capabilities as well. Maybe a plugin?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants