Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

display a large amount of geometry #835

Open
shyoon87 opened this issue Jul 12, 2024 · 2 comments
Open

display a large amount of geometry #835

shyoon87 opened this issue Jul 12, 2024 · 2 comments

Comments

@shyoon87
Copy link

shyoon87 commented Jul 12, 2024

It's very interesting to manage geometry using gpu.

Is there any way to display a large amount of geometry except to convert it to "tf_geo_rasterize" and into a Raster image?

Is there a way to convert it to VectorTile and display it on Mapbox?

@cdessanti
Copy link
Contributor

It's very interesting to manage geometry using gpu.

Is there any way to display a large amount of geometry except to convert it to "tf_geo_rasterize" and into a Raster image?

Is there a way to convert it to VectorTile and display it on Mapbox?

Right now out software (not the os version) and draw millions of linestrings, polygons and multipolygons or points in less than a second.

If you have a ratser it's can be loaded and going to be converted into points that can be drawn quite easily. I loaded an Esa dataset containing 8 billions points being able to analyze data with the single grace hopper.

BTW I am not the beta person to reply this question so I am asking for the this question internally

@mflaxman10
Copy link
Contributor

There are multiple questions in here, so let’s separate out a bit.
In general, we’ve got the fastest geo rendering architecture on the market, offering orders of magnitude higher performance than conventional. This works by avoiding raw data movement across the network, and we can render complex maps with literally billions of features in a few hundred milliseconds.

So this avoids a whole series of workarounds required in conventional GIS and BI, most notably vector tiles and separating visualization formats from analytic ones. Vector tiles work by simplifying out and indexing vector fragments by scale. None of that is needed in our system, since the source of truth is near instantly available. You can apply cartographic scale generalization if you want for visualization purposes but don’t need it for performance. Also no reason to rasterize for such purposes.

Independent of all of that, we do have raster data handling and rasterization as you’ve discovered. That can be used for analytics typically for things like weighted overlays. We continue to invest in improving this, at last release with HeavyConnect Raster which supports linking to and managing huge raster file data stores. At next release (8.1) we’ve added push down bounding box filtering for further efficiency when dealing with slower external storage sources like s3. Upcoming releases will include enhancements to color management and raster RBG (aka true color) rendering.

Lastly, if you have some data already in vector tile and lack access to the original full fidelity data, our SQL version of COPY FROM supports all of the built in GDAL drivers. I’ve lost track of which version of GDAL supports reading of vector tiles, so feel free to post again on that specifically if you need. We’ve mostly been recommending “flat geo buffer” as a high efficiency geo transfer format when needed. That works today in Immerse. We are tracking rapid advances in geoparquet, and expect further announcements there soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants