Feature request: Possibility to fetch observations with obscured coordinates

[oolonek]

Feature description

In some case observation can have obscured coordinates (for ex. when the species is endangered). It would be nice to still be able to fetch all remaining information except the precise geolocation, without authenticating.

Use case

I need to fetch all observation from a project but I can't because some have their geolocation obscured by iNaturalist. Only "The person who made the observation, Individuals who the observer has trusted with their hidden coordinates and Curators of the projects" can fetch it.

Workarounds

• Logging at iNaturalist and use the https://www.inaturalist.org/observations/export functions.
• Authenticating https://pyinaturalist.readthedocs.io/en/stable/user_guide.html#authentication (didn't try at the moment because I didn't fulfil the requirements to build an app see screenshot)

[JWCook]

This is an interesting one. Most API searches (by user, taxon, etc.) will return observations with obscured or private coordinates, unless you exclude them by passing geoprivacy='open'. But in this case, you will need to be authenticated to see these results when searching by project.

I think the main issue is that the project is limited by location:

1. The project requirements include locations that the observations must be within
2. The resolution of obscured coordinates (~27km) are larger than those locations
3. The observations with obscured coordinates are not publicly listed on the project, because that would give away a more specific location for those observations. In other words, it would make it easier for someone else to potentially find the location of an endangered species. This likely isn't as much of a concern for you if these observations are all in a botanical center, but that's just a safety measure in iNaturalist to avoid giving away more information than you intended to.

I am only able to see 183 observations on that project (but I am guessing it shows 192 for you, if you're logged in):

And observation 112079159 doesn't show any associated projects for me:

Same in the API results:

from pyinaturalist import get_observations

obs = get_observations(id=112079159)
print(obs['results'][0]['project_ids'])
[]

The reason you can see those observations in the CSV from the export tool is because you're logged in on inaturalist.org, so I think you will need to use an access token to see the same results from the API.

Also, I found some related forum threads here:

• https://forum.inaturalist.org/t/what-is-the-size-of-obscured-area
• https://forum.inaturalist.org/t/include-obscured-observations-in-species-lists-for-place-based-projects

[JWCook]

By the way, you can manually test an authenticated API request by getting a token here: https://www.inaturalist.org/users/api_token

And passing it via access_token:

from pyinaturalist import get_observations

obs = get_observations(project_id=130644, access_token='eyjHA6OjE2...')

Or just open an API request URL in your browser: https://api.inaturalist.org/v1/observations?project_id=130644

[JWCook]

@oolonek Did that answer your question?

[oolonek]

Hi @JWCook ! Thanks for the detailed explanations. This all makes sense. However I just tested now with the authenticated API token and still get 183 entries (instead of the expected 192).

[oolonek]

By the way, the API request url (https://api.inaturalist.org/v1/observations?project_id=130644) also returns 183 entries only.
See first lines below :

{"total_results":183,"page":1,"per_page":30,"results":[{"quality_grade":"casual","time_observed_at":"2022-06-02T11:54:00+02:00","taxon_geoprivacy":null,"annotations":[],"uuid":"fc28ac57-b200-4701-aa71-39c4924d0584","observed_on_details":{"date":"2022-06-02","week":22,"month":6,"hour":11,"year":2022,"day":2},"id":120025587,"cached_votes_total":0,"identifications_most_agree":false,"created_at_details":{"date":"2022-06-03","week":22,"month":6,"hour":18,"year":2022,"day":3},"species_guess":"Ruta graveolens","identifications_most_disagree":false,"tags":[],"positional_accuracy":null,"comments_count":0,"site_id":1,"created_time_zone":"Europe/Zurich","license_code":"cc0","observed_time_zone":"Europe/Zurich","quality_metrics":

[oolonek]

Ohhh ! My bad it is working through the get_observations() with the authenticated API token
I was actually passing the token under api_token= instead of access_token= !
Strangely enough, no error message returned when using api_token=

Thanks @JWCook all set up on my side !

[abubelinha]

By the way, you can manually test an authenticated API request by getting a token here: https://www.inaturalist.org/users/api_token
And passing it via access_token:

from pyinaturalist import get_observations
obs = get_observations(project_id=130644, access_token='eyjHA6OjE2...')

@JWCook could you tell me why this is not working as I expected?

def pyinat_auth_test():
	from importlib.metadata import version
	print(version('pyinaturalist'))

	# I prefer to get the token inside my own application:
	token = get_access_token(
		username='usus', password='papa', 
		app_id='aiai', app_secret='asas'
	)
	print("you got a {} characters length code".format(len(token)))

	# now I pass it and request one of my own obscured observations
	obs = get_observations(id=xxxxxx, access_token=token)
	print(obs.get("taxon").get("name"), 
		obs.get("geojson"), 
		obs.get("geoprivacy"), 
		obs.get("private_geojson"))

if __name__ == "__main__":
	import sys
	from pyinaturalist import *
	pyinat_auth_test()

output:

0.13.0
you got a 43 characters length code
TypeError: get_observations() got an unexpected keyword argument 'access_token'

Thanks a lot in advance

EDIT:
Sorry. Indeed version 0.13.0 was too old.
Problem solved after upgrading.

New output:

0.17.4
you got a 192 characters length code
Castanea sativa {'type': 'Point', 'coordinates': [-0.0384499418, 44.8465039563]} obscured {'type': 'Point', 'coordinates': [-0.0349975184, 44.9475050275]}

[abubelinha]

BTW, I think both get_observation() and get_observations() documentation is not up to date (access_token parameter is not mentioned for any of them).
Or perhaps I am reading an old documentation page?

[JWCook]

@abubelinha You're right, thanks for pointing that out. I'll get that updated.

[abubelinha]

Thanks @JWCook
One question. Do you know why my token length changed from 43 to 192 after upgrading pyinaturalist in my code above?

That might help to explain why I am failing to get a valid token using the python code in this gist.
It's adapted from a 7 years old code, but I am trying to use it in an old machine with Python 3.4 where I can't install pyinaturalist.

That gist gives me a 43 characters token as well. But when I try to use it in another request, it doesn't return me the obscured coordinates. So I guess it is not a correct token.

I know this is not a pyinaturalist issue, but perhaps you could you give me a clue of what is failing?

[JWCook]

Do you know why my token length changed from 43 to 192 after upgrading pyinaturalist in my code above?

It's a different token format. In 0.17 I switched the default format to JWT, which according to iNat is now the preferred method: https://www.inaturalist.org/pages/api+recommended+practices

As far as I know, OAuth tokens (the 43-character ones) should still be working with the V1 API, so I'm not sure what's wrong in your example. If you want to try out JWT, just add another request to your example after line 22:

response = requests.get(
    'https://www.inaturalist.org/users/api_token',
    headers={'Authorization': f'Bearer {token}'},
)
token = response.json()['api_token']

That token then goes in your Authorization header, just like the OAuth tokens.

If that doesn't help, go ahead and create another issue and I can help look into it some more.

[abubelinha]

Thank you very much for this advice 😃 👍 👍 👍 !!
So one still needs the old OAuth token in order to get the new JWT token.

I upgraded my gist and I can confirm it now works in my old Python 3.4 machine.

BTW I can also confirm that in Python 3.8, setting pyinaturalist 0.17.4 get_access_token(jwt=False), further api requests do also fail to provide private information fields like private_geojson.