Support for limiting results

[derniwi]

Hi,

I'm going to test this API i a micro controller. Due to the limited RAM it would be nice to have an additional api call with a limited result set or returning just requested parameters.
As an example
https://api.brightsky.dev/weatherlimited?lat=52&lon=7.6&date=2020-04-21&params=timestamp,temperature,condition
should only return
{
"weather": [
{
"timestamp": "2020-04-21T00:00:00+00:00",
"temperature": 10.6,
"condition": "dry"
},
...
]
}

Best egards
Nils

[jdemaeyer]

Hi Nils,

I think this would indeed be a nice addition to the existing endpoints, and a chance to upgrade our falcon dependency along the way. I can't promise it, but I'll try to get it done sometime this week. :)

Notes to myself:

• the cheap way to do this would be to perform our database lookup as-always and just delete the unrequested fields before returning the response
• the cleaner approach would be to add a params argument to query.weather() and query.current_weather() and not retrieve the unrequested fields in the first place, thereby improving performance of the database lookup and - more importantly - reducing the chance of having to fill up incomplete weather records
• the latter way doesn't seem too much extra effort as long as it's easy to properly validate the user input - using request.get_param_as_list and checking the list contents against a whitelist should probably be enough?

[derniwi]

Hi Jakob,

don't hurry. Querying and receiving only the needed information should be easy, maybe there are some values which are mandatory? Or some single values are useless? So just requesting "wind" should deliver all wind related values without requesting all of them separately, querying only "wind_direction" seems currently useless for me...

Maybe it makes sense to cluster some values like "get=atmos" might query
pressure_msl, temperature, cloud_cover, dew_point, relative_humidity, visibility and condition

Which should not replace something like "get=temperature,condition" which will return this two values.

Thank you for the quick response
Nils

[rscmbbng]

Hi Jakob,

great project! I have a similar issue request to minimize data on the wire & ram: the possibility to query specific dates and times, rather than ranges.

For example I am interested only in 2020-04-21T13:00, 2020-04-22T13:00, 2020-04-23T13:00 and would like to write a query something like:
https://api.brightsky.dev/weather?date=2020-04-21T13%3A00%2C+2020-04-22T13%3A00%2C+2020-04-23T%3A13%3A00&lat=51.58&lon=7.38"

Currently the only way to get these dates is to either ask a range (https://api.brightsky.dev/weather?date=2020-04-21T13%3A00&last_date=2020-04-21T15%3A00&lat=51.58&lon=7.38)
or to make separate requests for a single date+time ( like so "https://api.brightsky.dev/weather?date=2020-04-21T13%3A00&last_date=2020-04-21T13%3A00&lat=51.58&lon=7.38").

The combined query would return 3 items in weather, rather than the 49 one gets when the range is supplied. Likewise the content length will go from 27899 to 2623. When doing three separate requests I have a total content length of 4449. These numbers are estimations on what the API returns now.

[jdemaeyer]

I am finally getting around to this as part of our new funding round (#138).

@derniwi @rscmbbng Maybe you can help me get a better feeling for what this feature should aim at. If the main goal is to reduce the footprint of the response, then it seems consistent to remove everything except the actual values. E.g.:

> GET /weather?lat=52&lon=7.6&date=2023-03-02&compress=temperature,icon
[[-2.9, "clear-night"], [-0.6, "clear-night"], ...]

(Where the ordering of the inner lists matches that of the supplied compress parameter.)

Would that work for your use cases? Also, do you need the sources part of the response? If so, is including source_id in the compress parameter and generating an extra request to the /sources endpoint a feasible option?

[derniwi]

Hi Jakob,
thank you and sorry for the late answer.
I'll check if I can easily switch to an other result format. JSON is currently easy because there exists many libraries which can be used. But your compressed format is not longer JSON so I have to check if parsing is easily possible.
Greetings
Nils

[jdemaeyer]

Hi Nils

the output is still JSON :) Just that the compressed format is an array and not an object. (If you remove the ... which of course would not be in any actual API response.)

$ echo '[[-2.9, "clear-night"], [-0.6, "clear-night"]]' | jq .

[
  [
    -2.9,
    "clear-night"
  ],
  [
    -0.6,
    "clear-night"
  ]
]