Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add order parameter to changeset query API call #4158

Merged
merged 2 commits into from Aug 13, 2023

Conversation

AntonKhorev
Copy link
Contributor

When you want to get an unknown number of changesets matching some criteria, you use /api/0.6/changesets with the corresponding parameters. If you receive a number of changesets that is equal to the limit, you know there might be more matching changesets. To get the next page of changesets you set the upper limit of the time parameter to the last received changeset.* The lower limit is usually set to some date before osm existed. Then you run the query again and receive the next page of results. That next page should consist of changesets created earlier than the ones received previously.

But what if you want a previous page? You can't set the upper time limit to the first received changeset because you'll get the same result again.* You can set the lower limit, but then you'll still need to decide on the upper limit. If you don't set the upper limit, you'll receive the first page with the latest changesets, not the previous one. You'll have to guess the upper time limit of the previous page, and guessing it correctly may take several requests.

If you could get changesets in ascending order, specifying lower limit would be enough to get the previous page.*

This pull request adds the order parameter to /api/0.6/changesets, similar to /api/0.6/notes/search. order=oldest returns changesets in ascending order.

* some details about same-second and overlapping changesets omitted

@mattfbacon
Copy link

then you'll still need to decide on the upper limit.

Can't you just use the same approach as with the lower limit and set it to a value far far into the future?

@AntonKhorev
Copy link
Contributor Author

Can't you just use the same approach as with the lower limit and set it to a value far far into the future?

Setting the upper limit far into the future is the same as not setting it at all. You'll get the "first page" of changesets, not the previous page.

@mattfbacon
Copy link

Ah,I see, it always chooses the N most recent changes within the range.

@tomhughes tomhughes merged commit 6d74aa2 into openstreetmap:master Aug 13, 2023
15 checks passed
@AntonKhorev AntonKhorev deleted the changeset-query-order branch August 13, 2023 14:48
@AntonKhorev
Copy link
Contributor Author

Looks like there are performance problems now. When the query is limited by a user, it runs ok. When it asks for every changeset in a specified time window, it's slow. Apparently the db doesn't like WHERE (... and created_at <= $sometime) ORDER BY created_at ASC with the where condition and the order going in different directions.

@tomhughes
Copy link
Member

Are you saying existing queries have got slower? or new ones that weren't possible before are slow?

@AntonKhorev
Copy link
Contributor Author

AntonKhorev commented Aug 13, 2023

The new ones with order=oldest.
I have a possible fix with an extra condition on created_at.

@AntonKhorev
Copy link
Contributor Author

Example of slow request

https://api.openstreetmap.org/api/0.6/changesets.json?limit=10&time=20200501T000000Z&order=oldest

It probably goes through almost the entire created_at index because there's no condition created_at. The lower bound is checked only on closed_at. In the fix I added a condition that skips changesets that are definitely too old to be open at the lower bound (2 * max age).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants