This is awesome! But how to efficiently query lots of data?

[guidocioni]

I really don't know how I didn't discover this API before...I was really looking for a European-aggregated database for weather observation. This is just awesome :D

I would like to make a map of daily tmax in Europe so I just tried to use bounds followed by a fetch

stations = Stations()
stations = stations.bounds((70, -25), (30, 50))
list_stations = stations.fetch()
start = datetime(2021, 5, 1)
end = datetime(2021, 5, 2)
data = Daily(list_stations, start, end)
data = data.fetch()

This is obviously taking a long time as from the code it seems it is querying individually every csv and extracting info to reconstruct the output df.

Is there a better way to obtain daily data over an area? Seems that all endpoints only support querying data by station ID which means that they will also work with a list but likely iterate over it.

Thanks again for making this lib available to everyone :)

[clampr]

Thanks @guidocioni 🙂

You're right. That's the way to go with what the lib offers at the moment. I'm working on gridded data access using XArray. That might be an alternative, but it will take a few more months until this is ready.

[guidocioni]

Thanks @guidocioni 🙂

You're right. That's the way to go with what the lib offers at the moment. I'm working on gridded data access using XArray. That might be an alternative, but it will take a few more months until this is ready.

Maybe out of curiosity but...aren't the observations also stored into a DB? Wouldn't it be better to use that source of data also in the Python library? Or isn't that used because the bulk API does not require a token?

[clampr]

Exactly. I wanted to provide an interface which provides unlimited data access. However, I can provide custom endpoint as part of the JSON API if required.