Skip to content

Paginated query results after first page do not gracefully fallback to GET on HTTP-405 response #846

@m9brady

Description

@m9brady

I've come across an issue with trying to retrieve multi-page results (i.e. when using pystac-client.Client.search with limit=5 and the number of matched items is >5) and not sure if it's been covered before in past issues. Since the retry-on-405 strategy does seem to be covered in the pystac-client test suite I'm confused why I'm hitting it here.

Minimum working example:

from pystac_client import Client, __version__ as libver
print(f"Using pystac-client v{libver}")
stac_url = 'https://www.eodms-sgdot.nrcan-rncan.gc.ca/stac'
catalog = Client.open(stac_url)
print(f"{stac_url} contains 'rel' link with type 'search': {'search' in [link.rel for link in catalog.links]}")
print(f"{stac_url} conforms to ITEM_SEARCH: {catalog.conforms_to('ITEM_SEARCH')}")

returns

Using pystac-client v0.9.0
https://www.eodms-sgdot.nrcan-rncan.gc.ca/stac contains 'rel' link with type 'search': True
https://www.eodms-sgdot.nrcan-rncan.gc.ca/stac conforms to ITEM_SEARCH: True

which to me means that the pystac_client.Client.search method will used as per docs.

Moving on to the actual searching, I expect the following query parameters to return 8 results.

bbox = [-105.40747286434971, 68.70958793236613, -97.90802872229544, 71.11275348080056]
time_window = "2025-10-27/2025-10-28"
limit = 5

and since I do not specify the method= argument, the default request type is POST. The result of the following block fails with the subsequent APIError

query = catalog.search(
    collections=['rcm'],
    bbox=bbox,
    datetime=time_window,
    limit=limit
)
print(f"Query is using {query.method} method")
results = []
# I would normally use enumerate(query.pages()) here but I need to show in this issue that the first page is returned/parsed properly
page_num = 1
for page in query.pages():
    print(f"Reading result page #{page_num}", end= " ", flush=True)
    results.extend(page.items)
    print("Done!")
    page_num += 1
print(f"Query returned {len(results)} matches")
Query is using POST method
Reading result page #1 Done!

<traceback snip>
APIError: {"detail":"Method Not Allowed"}
</traceback snip>

but what's interesting is that it DOES parse the first result page properly (i.e. properly retrying with GET) before failing on the second page. I would expect the query.pages() iterator to properly fallback to GET for all pages but it seems to me that it isn't?

The error can be completely avoided by using method="GET" in the pystac_client.Client.search call:

query = catalog.search(
    collections=['rcm'],
    bbox=bbox,
    datetime=time_window,
    limit=limit,
    method="GET"                 # <---- explicit setting to GET
)
print(f"Query is using {query.method} method")
results = []
page_num = 1
for page in query.pages():
    print(f"Reading result page #{page_num}", end= " ", flush=True)
    results.extend(page.items)
    print("Done!")
    page_num += 1
print(f"Query returned {len(results)} matches")

returns

Query is using GET method
Reading result page #1 Done!
Reading result page #2 Done!
Query returned 8 matches

but it doesn't seem like a good solution when there is existing logic in pystac-client to gracefully handle the HTTP-405 response from the endpoint.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions