Allow Optimized searches for tiled requets. #39
Conversation
…l and sample data
…odify test data with ids that can be identified as testdata and to have attributes to test more filters
…, start/end dates and updated properties
|
FYI an integration of the |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
| @@ -31,7 +31,7 @@ services: | |||
| - PGHOST=localhost | |||
| - PGDATABASE=postgis | |||
| ports: | |||
| - "5432:5432" | |||
| - "5439:5432" | |||
There was a problem hiding this comment.
Hope this is ok, Basically using 5432 was in conflict with my local pg instance
| @@ -104,17 +104,17 @@ BEGIN | |||
|
|
|||
| prev_area := unionedgeom_area; | |||
|
|
|||
| RAISE NOTICE '% % % %', st_area(unionedgeom)/tilearea, counter, scancounter, ftime(); | |||
| RAISE NOTICE '% % % %', unionedgeom_area/tilearea, counter, scancounter, ftime(); | |||
There was a problem hiding this comment.
avoid unnecessary st_area computation
| IF fields IS NOT NULL THEN | ||
| out_records := out_records || filter_jsonb(iter_record.content, includes, excludes); | ||
| ELSE | ||
| out_records := out_records || iter_record.content; | ||
| END IF; | ||
|
|
||
| IF counter > _limit | ||
| IF counter >= _limit |
There was a problem hiding this comment.
because we set counter to 1 by default, if we set limit to 1, we will get 2 items
| OR scancounter > _scanlimit | ||
| OR ftime() > _timelimit | ||
| OR unionedgeom_area >= tilearea | ||
| OR (skipcovered AND unionedgeom_area >= tilearea) |
There was a problem hiding this comment.
should only check for tile coverage if skipcovered==True
There was a problem hiding this comment.
Actually this call should not include the skipcovered. This is the statement that returns as soon as we have covered the entire area of the geometry passed in. The skipcovered parameter is there to allow us to not include an individual item where that item would be "buried" by the other items that have already been included. I'll change this back.
Three SQL functions are provided that all return a FeatureCollection of items that are within the geometry that has been requested. The number of records returned is based on scanning through the data until the geometry passed in has been filled with tiles using the geometry included with the item record.
geoemtrysearch(geom, queryhash, fields, _scanlimit, _limit, _timelimit, skipcovered);
geojsonsearch(geojson, queryhash, fields, _scanlimit, _limit, _timelimit, skipcovered);
xyzsearch(x,y,z, queryhash, fields, _scanlimit, _limit, _timelimit, skipcovered)
queryhash => a hash of the CQL query that is created and passed to allow get requests with larger queries
fields => Json definition {"fields":{"include":[],"exclude":[]}} to filter what is included in the results
_scanlimit (default 10000) total number of rows to scan through before returning
_limit (default 10) max number of rows to return
_timelimit (default 5 seconds) scan through the data for up to 5 seconds and then return what has been accumulated to that point
exitwhenfull => exit as soon as the geometry passed in is completely covered by records
skipcovered => If the geom of a record is entirely covered by the previous (by order) records, don't include it in the output (when this is set is when _scanlimit and _limit can be different)
Adds create_items and upsert_items that will take a jsonb array of json stac items and bulk load.
Fixes #31
Fixes #16