Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Static data providers #149

Open
PeterPetrik opened this issue Jun 4, 2019 · 6 comments

Comments

@PeterPetrik
Copy link

commented Jun 4, 2019

QGIS Enhancement: Static data providers

Date 2019/06/04

Author Peter Petrik (@PeterPetrik)

Contact peter.petrik@lutraconsulting.co.uk

maintainer @PeterPetrik

Version QGIS 3.10

Summary

Currently all QGIS data providers written in C++ are compiled as separate modules (.so / .dll) which are loaded by traversing lib/qgis/plugins directory upon start. The idea of this QEP is to make it possible to compile code of data providers directly into qgis_core library. This would become a compilation option which may be turned on or off.

Reasons:

  • simplification of bootstrap of qgis_core (no need to walk through plugins directory to load plugins one by one)
  • formalise internal API of c++ data providers
  • speeding up the load time for desktop builds (probably marginally)
  • iOS platform does not support dynamic libraries
  • for Android development, it would be more convenient to have data providers (OGR, GDAL, MDAL, postgresql, ...) as part of QGIS core (fewer .so files to ship)

This proposal describes technical solution of how to allow static linkage of providers to the core library.

Proposed Solution

The existing internal API for C++ data providers has become convoluted and consists of a bunch of exported functions that are not properly documented and data provider modules implement a subset of those. Moreover, some exported functions are quite specific to a particular provider.

Here is the list of exported functions

Screenshot 2019-06-04 at 07 58 26

We propose to move OGR, GDAL and MDAL providers to src/core and force them to be always build similar to memory providers. Their extra functions used from qgisapp (e.g. directoryExtensions, isValidRasterFilename) will be outside the provider interface. QGIS cannot be built without OGR, GDAL and MDAL anyway and the number of functions that is exported only from these providers should not populate common provider's interface.

We propose to add the rest of the functions to QgsProviderMetadata similarly to CreateDataProviderFunction. We will deprecate use of createProviderLibrary and function() from QgsProviderRegistry, so it will not be possible to call custom functions, just functions from defined interface.

The interface can be divided to 5 function "groups":

  • core(description, type): core functions valid for any data provider
  • browser (dataItemProviders): functions related to browser panel
  • gui (selectWidget): functions related to GUI widgets
  • style (listStyles): functions for layer styles
  • data management(createTransaction): functions for database or data management handling

We introduce new CMAKE global define WITH_STATIC_PROVIDERS that will force all providers to be statically linked.

In dynamic-linked compilation, there will be just one C-exported function void* getQgsProviderMetadata() instead of current set of functions that will be checked by QgsProviderRegistry. In statically-linked compilation, the list of expected static providers will be hardcoded in the provider registry constructor.

Similarly to symbol layers (QgsSymbolLayerWidget), GUI components (widgets) will have metadata class. Registration of the GUI metadata will be done in qgis_gui initialisation routine.

Affected Files

many files all over the code that calls any of the custom OGR, GDAL and providers functions, most changes in

  • src/code/qgsproviderregistry.cpp
  • src/code/qgsprovidermetadata.cpp
  • src/providers
  • src/core/providers

Further Considerations/Improvements

  • remove dataItem() and dataCapabilities() functions
  • implement deleteStyleByID() for msqsl, spatialite and oracle providers
  • merge classFactory() and createProvider() functions

Backwards Compatibility

  • deprecate use of createProviderLibrary and function() from QgsProviderRegistry
  • custom/external C++ providers will need to implement new interface

Issue Tracking ID(s)

(optional)

Votes

(required)

@elpaso

This comment has been minimized.

Copy link

commented Jun 4, 2019

big +1!

@luipir

This comment has been minimized.

Copy link

commented Jun 4, 2019

generally +0, but +1 due to "iOS platform does not support dynamic libraries"

@nyalldawson

This comment has been minimized.

Copy link

commented Jun 4, 2019

+1 to moving gdal/ogr providers to core. It's impossible to build QGIS without gdal anyway, and I can't see any good reason why anyone would NOT want to use GDAL/OGR layers, so I can't see any reason for keeping these out of core.

@NathanW2

This comment has been minimized.

Copy link
Member

commented Jun 4, 2019

@nyalldawson

This comment has been minimized.

Copy link

commented Jun 5, 2019

I'd like to propose that alongside GDAL and OGR, the WMS provider ALSO be moved to core (since it enables the use of XYZ tiles).

@PeterPetrik

This comment has been minimized.

Copy link
Author

commented Jun 12, 2019

I suggest to migrate just OGR/GDAL to core to minimize the number of changes and keep other providers to be migrated if needed in later stages

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.