Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve polygon coordinate handling in search_data #504

Open
jhkennedy opened this issue Mar 29, 2024 · 5 comments
Open

Improve polygon coordinate handling in search_data #504

jhkennedy opened this issue Mar 29, 2024 · 5 comments
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@jhkennedy
Copy link
Collaborator

Note: Reported by @iamdonovan in chat

search_data allows you to specify a polygon= keyword argument, who's value is handled by:
https://github.com/nsidc/earthaccess/blob/v0.9.0/earthaccess/search.py#L734-L742

and passed to python-cmr's polygon method:
https://github.com/nasa/python_cmr/blob/develop/cmr/queries.py#L458

python-cmr expects a list of (lat, lon) tuples, in clockwise order, as documented in the CMR API docs:
https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#g-polygon

However, there is very little handling along the way to ensure we meet that specification.

It would be very natural to do something like:

search_area = shapely.Polygon(...)   # some shapely Geometry

earthdata.search(polygon=search_area, ...)

But right now, you'd need to do:

earthdata.search(polygon=search_area.coords, ...)

which may end up resulting in:

RuntimeError: {"errors":["The polygon boundary points are listed in the wrong order. Points must be provided in counter-clockwise order."]}

We currently have to do:

exterior = search_area.exterior
if not shapely.is_ccw(exterior)
    exterior = exterior.reverse()

earthdata.search(polygon=exterior.coords, ...)

or

polygon = shapely.geometry.polygon.orient(search_area, sign=1).exterior.coords
earthdata.search(polygon=polygon, ...)

Ideally, we'd accept any of the following:

  • valid WKT string
  • valid WKB string
  • GDAL OGR Geometry
  • Shapely Geometry
  • List of coordinate tuples

and ensure we provide the correctly oriented coordinate list to python-cmr.


Additionally, our search_data documentation could be improved. Currently, we imply, by linking out to the CMR API docs, that polygon (and point, line, and circle) is a search keyword argument, but all our examples tend to use a bounding box, and the only geometric parameter we do list is bounding_box:
image

https://earthaccess.readthedocs.io/en/latest/user-reference/api/api/#earthaccess.api.search_data

@jhkennedy jhkennedy added enhancement New feature or request good first issue Good for newcomers labels Mar 29, 2024
@iamdonovan
Copy link

Now that I'm up and have had some coffee, it looks like some of the documentation is already available in the DataGranules class:

https://earthaccess.readthedocs.io/en/latest/user-reference/granules/granules-query/

But, it's not immediately clear that the names of these methods can be passed directly to earthaccess.search_data() - so, either a pointer to the DataGranules documentation for valid CMR kwargs, or a full list of the CMR kwargs in the documentation for earthaccess.search_data() (and probably correspondingly for DataCollections and search_datasets()) would be very helpful.

@mfisher87
Copy link
Member

mfisher87 commented Mar 30, 2024

But, it's not immediately clear that the names of these methods can be passed directly to earthaccess.search_data() - so, either a pointer to the DataGranules documentation for valid CMR kwargs, or a full list of the CMR kwargs in the documentation for earthaccess.search_data() (and probably correspondingly for DataCollections and search_datasets()) would be very helpful.

💯 Manually maintaining that list of kwargs in the docs has been a huge pain point, and I think we need an architecture change to automatically generate the list of kwargs. I apologize for the friction that's causing our user base!

We're tracking this in #345. I think this would be an excellent ticket to get started on in an upcoming Hack Day. Would you be interested in joining us this Tuesday, @iamdonovan? More details following the link. I'll send you an invite if you confirm you're interested!

@iamdonovan
Copy link

Alas, I will be camping on a beach in Donegal this Tuesday for the Easter holiday, but if they are still happening bi-weekly I can try to join on 16 April. I may not be able to make the start exactly, but should be able to join for most of the meeting.

I have started putting together a proposed fix to how earthaccess handles polygon coordinates, and will open a PR once I manage to get the testing environment set up, either later this week or early next week. I would be happy to try to contribute to the documentation side as well.

@mfisher87 mfisher87 changed the title Improve polygon coordinate hadnling in search_data Improve polygon coordinate handling in search_data Apr 1, 2024
@mfisher87
Copy link
Member

mfisher87 commented Apr 1, 2024

We don't yet have a stop-date in mind for these hack days 🥳

I'll learn how to invite you tomorrow and take care of it :)

I may not be able to make the start exactly, but should be able to join for most of the meeting

These are very informal, that's totally cool. You'll miss introductions and perhaps a demo if the group wants to have one. We record demos, so you'll be able to catch it on YouTube later. We also take notes in a collaborative document, so you'll be able to catch up on anything you like when you arrive!

I have started putting together a proposed fix to how earthaccess handles polygon coordinates, and will open a PR once I manage to get the testing environment set up, either later this week or early next week. I would be happy to try to contribute to the documentation side as well.

Amazing :) We need that documentation help for sure :)

@mfisher87
Copy link
Member

@iamdonovan calendar invite sent to your gmail address! Let me know if you don't receive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
Status: 🆕 New
Development

No branches or pull requests

3 participants