Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mechanism to clear cache? #103

Open
cboettig opened this issue Jul 25, 2024 · 3 comments
Open

mechanism to clear cache? #103

cboettig opened this issue Jul 25, 2024 · 3 comments

Comments

@cboettig
Copy link

Is there a way to adjust the cache expiration for data urls?

In general, in maplibre data resources come from an external url, which can make it a bit more difficult to update maps dynamically. If I understand correctly, the typical solution (provided we can update the remote data source) is to expire the cache so that the new data loads, i.e. set

'Cache-Control: 'max-age=0'

(perhaps as discussed in mapbox/mapbox-gl-js#2633 (comment) ? I'm a bit out of my depth on this one).

@crazycapivara
Copy link
Contributor

crazycapivara commented Jul 26, 2024

@cboettig In most case you do not need to update your data with Map.set_data but use filters instead which makes it very comfortable:

m = Map(...)
m.add_layer(...)

# Update your layer using a filter on the 'growth' property of your source
filter_expression = [">=", ["get", "growth"], 0.5] # only show features where growth >= 0.5

m.set_filter("your_layer_id", filter_expression)

Furthermore, the upcoming release comes with seamless support for geopandas data frames for vector data sources.

See also changelog for unreleased version.

@cboettig
Copy link
Author

oh fantastic! native support for geopandas is much simpler than cache exploration.

Thanks for mentioning the filters, I need to spend more time exploring the performance implications there. Do you know if maplibre js is able to execute those as a range request without transferring all the data? For the moment I'm relying on duckdb to do my filters, it's quite good at filtering 150gb parquet file in seconds, and figured I that would be difficult purely client-side. But in at least some cases I'm probably better off using the filters.

That filter syntax is really nice too. @giswqs -- is something like m.set_filter() possible in leafmap.maplibre wrapper? Guess that feels a bit more pythonic than putting it all in the style dict (though I guess something about configuring the map entirely with a dict is also kinda elegant in it's own way).

@crazycapivara
Copy link
Contributor

crazycapivara commented Jul 28, 2024

@cboettig Yes, range requests are possible:

["all", [">=", ["get", "growth"], 0.2], ["<=", ["get", "growth"], 0.5]]

In the next release I also added some helper functions for creating these kind of expressions / filters as you can see in this example.

In general, for large datasets it is good practice to use url sources, because maplibre only loads / queries the features in the current viewport and for the filters set, it never loads all of the data.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants