Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cloud Optimized GeoTIFF tile size autodetect #408

Closed
wants to merge 9 commits into from

Conversation

@Komzpa
Copy link
Member

commented Jun 3, 2019

Use tile sizes aligned with GeoTIFF "Natural Blocks" to speed up import 10x.

@robe2

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

Looks fine to me

Komzpa added 3 commits Jun 3, 2019
@dbaston

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

I think some tests might be helpful here.

This isn't strictly related to COGs, is it? It's defaulting to multiples of the block size, whatever that is?

@Komzpa

This comment has been minimized.

Copy link
Member Author

commented Jun 4, 2019

It's not about cloud optimized only but it also saves much time on them:

kom@nucat:~/downloads$ time ~/proj/postgis/raster/loader/raster2pgsql /vsicurl/https://github.com/postgis/postgis/raw/690cdd624b6d5ad62c497481832bbf0425457687/raster/test/regress/loader/testraster2.tif  -t auto 0_fb_africa_population_raster >/dev/null
Processing 1/1: /vsicurl/https://github.com/postgis/postgis/raw/690cdd624b6d5ad62c497481832bbf0425457687/raster/test/regress/loader/testraster2.tif
INFO: Using computed tile size: 256x192

real    0m13,444s
user    0m1,901s
sys     0m0,105s
kom@nucat:~/downloads$ time raster2pgsql /vsicurl/https://github.com/postgis/postgis/raw/690cdd624b6d5ad62c497481832bbf0425457687/raster/test/regress/loader/testraster2.tif  -t auto > /dev/nullProcessing 1/1: /vsicurl/https://github.com/postgis/postgis/raw/690cdd624b6d5ad62c497481832bbf0425457687/raster/test/regress/loader/testraster2.tif
INFO: Using computed tile size: 49x49

real    3m12,630s
user    0m12,546s
sys     0m3,494s

Komzpa added 2 commits Jun 4, 2019
@dbaston

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

I guess my thought is that if it's defaulting to the block size, whatever that is, you might end up with pretty bad tile sizes for rasters that are written as scanlines. (By "bad" I mean that they would perform poorly for spatial queries.)

@Komzpa

This comment has been minimized.

Copy link
Member Author

commented Jun 4, 2019

@dbaston scanline case is guarded by taking image dimension instead of block dimension if it is 1.

@strk strk closed this in b6055a5 Jun 4, 2019

@Komzpa Komzpa deleted the Komzpa:ticket-4413 branch Jun 4, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.