Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Geopackage: handle raster drop in browser #5057

Merged
merged 10 commits into from
Aug 29, 2017

Conversation

elpaso
Copy link
Contributor

@elpaso elpaso commented Aug 22, 2017

Description

This implementation is using GDALTranslate directly, my previous attempt to use QgsRasterFileWriter was not working because the originated gpkg table was created ok but it was completely empty.

As a side note, the failed attempt to import a sample raster with the QgsRasterFileWriter task was at least 10 times slower than using GDALTranslate directly.

}
}
GDALClose( hSrcDS );
GDALTranslateOptionsFree( psOptions );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing GDALClose( hOutDS )


const char *args[] = { "-of", "gpkg", "-co", QStringLiteral( "RASTER_TABLE=%1" ).arg( u.name ).toUtf8().constData(), "-co", "APPEND_SUBDATASET=YES", nullptr };
GDALTranslateOptions *psOptions = GDALTranslateOptionsNew( ( char ** )args, nullptr );
GDALDatasetH hSrcDS = GDALOpen( u.uri.toUtf8().constData(), GA_ReadOnly );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be prudent to check hSrcDS != NULL just in case

GDALTranslateOptions *psOptions = GDALTranslateOptionsNew( ( char ** )args, nullptr );
GDALDatasetH hSrcDS = GDALOpen( u.uri.toUtf8().constData(), GA_ReadOnly );
CPLErrorReset();
GDALDatasetH hOutDS = GDALTranslate( mPath.toUtf8().constData(), hSrcDS, psOptions, NULL );
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GDALTranslate() requires GDAL >= 2.1. Currently only >= 2.0 is required : https://github.com/qgis/QGIS/blob/master/cmake/FindGDAL.cmake#L104

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jef-n and @rouault: How do you feel about requiring >= 2.1 ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the Debian/Ubuntu world, the jump was from 1.11 to 2.1, and people using for example Ubuntu 16.04 have to use UbuntuGIS to build QGIS that ship with at least 2.1 as well. OSGeo4W has GDAL 2.2. So would seem to be OK to me.

@Gustry
Copy link
Contributor

Gustry commented Aug 22, 2017

Thanks for your efforts on supporting geopackage @elpaso !

Is-it possible to implement some kind of helpers, especially for python plugin coders for importing vectors and rasters easily, without doing the same kind of R&D that you are doing by trying QgsRasterFileWriter, GDALTranslate, or how to remove layers from a geopackage using SQL.

It would be very nice if we can have a set of tool for plugins to create/update/delete layers in a geopackage, without switching between the QGIS and GDAL/OGR API. I was writing a small wrapper https://github.com/Gustry/geopackage-qgis but it could be nice to have something in QGIS.

@elpaso
Copy link
Contributor Author

elpaso commented Aug 22, 2017

@Gustry yes, my preferred way would be to rely on QGIS API instead of going upstream with GDAL.
Unfortunately I wasn't able to make the complex machinery of QgsRasterFileWriter work for this use case, that class was designed and tested to produce image rasters (geotiff etc.), not to work on gpkg rasters.

If you want to go that route (that would probably be ~ 10 times slower anyway), I think you need some help from the raster experts ( @rouault , @blazek etc. ), btw, I've not yet given up completely, I want to spend some more time to see if I can make it work.

If I don't succeed (that is very likely), I would go with this GDAL implementation and wrap it into a generic re-usable translate class (and a companion task), I think that it would be very handy for many use cases.

For the gpkg layer handling wrapper, in theory all needed functionality should be in the provider: I think we should implement raster delete in GDAL, I've added the deleteLayer in da6049e and createEmptyLayer was already added by @nyalldawson for OGR but it's missing for GDAL provider and would not work for gpkg in any case because is't not implemented in the GPKG GDAL provider.

@rouault
Copy link
Contributor

rouault commented Aug 22, 2017

that class was designed and tested to produce image rasters (geotiff etc.), not to work on gpkg rasters.

I don't know how QgsRasterFileWriter works, but the GDAL GPKG driver implements the GDAL Raster API, so in theory QgsRasterFileWriter should work with it. Well that's the theory. There must be some grain of sand somewhere...

I think we should implement raster delete in GDAL

A related question was asked this morning on GDAL ML. An enhancement ticket has been filed https://trac.osgeo.org/gdal/ticket/7013

@elpaso elpaso changed the title [WIP] Geopackage: handle raster drop in browser Geopackage: handle raster drop in browser Aug 24, 2017
@elpaso
Copy link
Contributor Author

elpaso commented Aug 24, 2017

@nyalldawson I've wrapped the importer into a QgsTask, can you please have a look?

Copy link
Collaborator

@nyalldawson nyalldawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! @rouault could you also take a look here?

Now that 2.1 is required
@elpaso
Copy link
Contributor Author

elpaso commented Aug 28, 2017

@nyalldawson I've removed some ifdefs, there is one inside test that cannot be removed because the build will error: https://github.com/qgis/QGIS/blob/master/tests/src/core/testqgsinvertedpolygonrenderer.cpp#L162 ( @mhugo would you mind having a look?)
Since I suspect that that particular test has never run, I don't consider this a blocker for this PR.
@rouault has already reviewed the main GDAL related part, from the time when he reviewed the code the only addition was the progress function: https://github.com/qgis/QGIS/pull/5057/files#diff-e9261e12bb7888ef998381ac7062df2fR41

@nyalldawson
Copy link
Collaborator

@elpaso that test is broken - I tried removing the ifdef a while back with no luck there.

@elpaso elpaso merged commit daa60d1 into qgis:master Aug 29, 2017
@elpaso elpaso deleted the gpkg-raster-import2 branch August 29, 2017 06:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants