Skip to content
Permalink
Browse files

[processing] Non-filed based outputs (e.g. postgis, geopackage)

options should be available for certain model outputs and script
algorithm outputs

We do this by swapping the test for non-file based output support
from checking only the algorithm's provider to instead checking
on a parameter-by-parameter basis.

This is done in order to support models. For models, depending
on what child algorithm a model output is based off, an individual
model may or may not have support for non-file based outputs. E.g
a model may generate outputs from a native qgis alg (supporting
these outputs) AND an output from a GDAL alg (with no support
for these outputs). In this case we need to enable or disable
the ui controls for non-file based outputs on an individual
output basis.

For scripts (for now) we blindly just say all outputs support
non-file based formats. This is going to be the case most of
the time, since scripts will usually be written using PyQGIS
API. For the exceptions (e.g. scripts which call other algs
like GDAL algs) we probably should add some way for the script
to indicate whether an individual output supports this, but
for now we just say they all do.

Fixes #17949
  • Loading branch information
nyalldawson committed Jan 25, 2018
1 parent 723e0a1 commit bf19eb6f35afafe113cf3d43950e0538992d08fa
@@ -1775,20 +1775,20 @@ Returns a new QgsProcessingOutputDefinition corresponding to the definition of t
parameter.
%End

bool supportsNonFileBasedOutputs() const;
bool supportsNonFileBasedOutput() const;
%Docstring
Returns true if the destination parameter supports non filed-based outputs,
such as memory layers or direct database outputs.

.. seealso:: :py:func:`setSupportsNonFileBasedOutputs`
.. seealso:: :py:func:`setSupportsNonFileBasedOutput`
%End

void setSupportsNonFileBasedOutputs( bool supportsNonFileBasedOutputs );
void setSupportsNonFileBasedOutput( bool supportsNonFileBasedOutput );
%Docstring
Sets whether the destination parameter supports non filed-based outputs,
such as memory layers or direct database outputs.

.. seealso:: :py:func:`supportsNonFileBasedOutputs`
.. seealso:: :py:func:`supportsNonFileBasedOutput`
%End

virtual QString defaultFileExtension() const = 0;
@@ -143,7 +143,10 @@ Otherwise the first reported supported raster format will be used.
virtual bool supportsNonFileBasedOutput() const;
%Docstring
Returns true if the provider supports non-file based outputs (such as memory layers
or direct database outputs).
or direct database outputs). If a provider returns false for this method than it
indicates that none of the outputs from any of the provider's algorithms have
support for non-file based outputs. Returning true indicates that the algorithm's
parameters will each individually declare their non-file based support.

.. seealso:: :py:func:`supportedOutputVectorLayerExtensions`
%End
@@ -77,7 +77,7 @@ def __init__(self, parameter, alg):
self.leText.setPlaceholderText(self.SKIP_OUTPUT)
self.use_temporary = False
elif isinstance(self.parameter, QgsProcessingParameterFeatureSink) \
and alg.provider().supportsNonFileBasedOutput():
and self.parameter.supportsNonFileBasedOutput():
# use memory layers for temporary files if supported
self.leText.setPlaceholderText(self.SAVE_TO_TEMP_LAYER)
elif not isinstance(self.parameter, QgsProcessingParameterFolderDestination):
@@ -107,7 +107,7 @@ def selectOutput(self):
popupMenu.addAction(actionSkipOutput)

if isinstance(self.parameter, QgsProcessingParameterFeatureSink) \
and self.alg.provider().supportsNonFileBasedOutput():
and self.parameter.supportsNonFileBasedOutput():
# use memory layers for temporary layers if supported
actionSaveToTemp = QAction(
self.tr('Create temporary layer'), self.btnSelect)
@@ -123,7 +123,7 @@ def selectOutput(self):
popupMenu.addAction(actionSaveToFile)

if isinstance(self.parameter, QgsProcessingParameterFeatureSink) \
and self.alg.provider().supportsNonFileBasedOutput():
and self.parameter.supportsNonFileBasedOutput():
actionSaveToGpkg = QAction(
self.tr('Save to GeoPackage...'), self.btnSelect)
actionSaveToGpkg.triggered.connect(self.saveToGeopackage)
@@ -146,7 +146,7 @@ def selectOutput(self):
popupMenu.exec_(QCursor.pos())

def saveToTemporary(self):
if isinstance(self.parameter, QgsProcessingParameterFeatureSink) and self.alg.provider().supportsNonFileBasedOutput():
if isinstance(self.parameter, QgsProcessingParameterFeatureSink) and self.parameter.supportsNonFileBasedOutput():
self.leText.setPlaceholderText(self.SAVE_TO_TEMP_LAYER)
else:
self.leText.setPlaceholderText(self.SAVE_TO_TEMP_FILE)
@@ -188,7 +188,7 @@ def saveToGeopackage(self):
filename, filter = QFileDialog.getSaveFileName(self, self.tr("Save to GeoPackage"), path,
file_filter, options=QFileDialog.DontConfirmOverwrite)

if filename is None:
if not filename:
return

layer_name, ok = QInputDialog.getText(self, self.tr('Save to GeoPackage'), self.tr('Layer name'), text=self.parameter.name().lower())
@@ -92,6 +92,9 @@ def icon(self):
def svgIconPath(self):
return QgsApplication.iconPath("processingModel.svg")

def supportsNonFileBasedOutput(self):
return True

def loadAlgorithms(self):
self.algs = []
folders = ModelerUtils.modelsFolders()
@@ -99,3 +99,10 @@ def loadAlgorithms(self):

def addAlgorithmsFromFolder(self, folder):
self.folder_algorithms.extend(ScriptUtils.loadFromFolder(folder))

def supportsNonFileBasedOutput(self):
# TODO - this may not be strictly true. We probably need a way for scripts
# to indicate whether individual outputs support non-file based outputs,
# but for now allow it. At best we expose nice features to users, at worst
# they'll get an error if they use them with incompatible outputs...
return True
@@ -107,13 +107,16 @@ void QgsProcessingAlgorithm::setProvider( QgsProcessingProvider *provider )
{
mProvider = provider;

// need to update all destination parameters to set whether the provider supports non file based outputs
Q_FOREACH ( const QgsProcessingParameterDefinition *definition, mParameters )
if ( !mProvider->supportsNonFileBasedOutput() )
{
if ( definition->isDestination() && mProvider )
// need to update all destination parameters to turn off non file based outputs
Q_FOREACH ( const QgsProcessingParameterDefinition *definition, mParameters )
{
const QgsProcessingDestinationParameter *destParam = static_cast< const QgsProcessingDestinationParameter *>( definition );
const_cast< QgsProcessingDestinationParameter *>( destParam )->setSupportsNonFileBasedOutputs( mProvider->supportsNonFileBasedOutput() );
if ( definition->isDestination() && mProvider )
{
const QgsProcessingDestinationParameter *destParam = static_cast< const QgsProcessingDestinationParameter *>( definition );
const_cast< QgsProcessingDestinationParameter *>( destParam )->setSupportsNonFileBasedOutput( false );
}
}
}
}
@@ -246,7 +249,8 @@ bool QgsProcessingAlgorithm::addParameter( QgsProcessingParameterDefinition *def
if ( definition->isDestination() && mProvider )
{
QgsProcessingDestinationParameter *destParam = static_cast< QgsProcessingDestinationParameter *>( definition );
destParam->setSupportsNonFileBasedOutputs( mProvider->supportsNonFileBasedOutput() );
if ( !mProvider->supportsNonFileBasedOutput() )
destParam->setSupportsNonFileBasedOutput( false );
}

mParameters << definition;
@@ -3209,7 +3209,7 @@ bool QgsProcessingParameterFeatureSink::fromVariantMap( const QVariantMap &map )

QString QgsProcessingParameterFeatureSink::generateTemporaryDestination() const
{
if ( supportsNonFileBasedOutputs() )
if ( supportsNonFileBasedOutput() )
return QStringLiteral( "memory:%1" ).arg( description() );
else
return QgsProcessingDestinationParameter::generateTemporaryDestination();
@@ -1720,16 +1720,16 @@ class CORE_EXPORT QgsProcessingDestinationParameter : public QgsProcessingParame
/**
* Returns true if the destination parameter supports non filed-based outputs,
* such as memory layers or direct database outputs.
* \see setSupportsNonFileBasedOutputs()
* \see setSupportsNonFileBasedOutput()
*/
bool supportsNonFileBasedOutputs() const { return mSupportsNonFileBasedOutputs; }
bool supportsNonFileBasedOutput() const { return mSupportsNonFileBasedOutputs; }

/**
* Sets whether the destination parameter supports non filed-based outputs,
* such as memory layers or direct database outputs.
* \see supportsNonFileBasedOutputs()
* \see supportsNonFileBasedOutput()
*/
void setSupportsNonFileBasedOutputs( bool supportsNonFileBasedOutputs ) { mSupportsNonFileBasedOutputs = supportsNonFileBasedOutputs; }
void setSupportsNonFileBasedOutput( bool supportsNonFileBasedOutput ) { mSupportsNonFileBasedOutputs = supportsNonFileBasedOutput; }

/**
* Returns the default file extension for destination file paths
@@ -146,7 +146,10 @@ class CORE_EXPORT QgsProcessingProvider : public QObject

/**
* Returns true if the provider supports non-file based outputs (such as memory layers
* or direct database outputs).
* or direct database outputs). If a provider returns false for this method than it
* indicates that none of the outputs from any of the provider's algorithms have
* support for non-file based outputs. Returning true indicates that the algorithm's
* parameters will each individually declare their non-file based support.
* \see supportedOutputVectorLayerExtensions()
*/
virtual bool supportsNonFileBasedOutput() const { return false; }
@@ -275,6 +275,16 @@ class DummyAlgorithm : public QgsProcessingAlgorithm
QCOMPARE( asPythonCommand( params, context ), QStringLiteral( "processing.run(\"test\", {'p1':'a','p2':'b'})" ) );
}

void addDestParams()
{
QgsProcessingParameterFeatureSink *sinkParam1 = new QgsProcessingParameterFeatureSink( "supports" );
sinkParam1->setSupportsNonFileBasedOutput( true );
addParameter( sinkParam1 );
QgsProcessingParameterFeatureSink *sinkParam2 = new QgsProcessingParameterFeatureSink( "non_supports" );
sinkParam2->setSupportsNonFileBasedOutput( false );
addParameter( sinkParam2 );
}

};

//dummy provider for testing
@@ -300,7 +310,13 @@ class DummyProvider : public QgsProcessingProvider
return "pcx"; // next-gen raster storage
}

bool supportsNonFileBasedOutput() const override
{
return supportsNonFileOutputs;
}

bool *unloaded = nullptr;
bool supportsNonFileOutputs = false;

protected:

@@ -426,6 +442,7 @@ class TestQgsProcessing: public QObject
void combineFields();
void stringToPythonLiteral();
void defaultExtensionsForProvider();
void supportsNonFileBasedOutput();

private:

@@ -3973,7 +3990,7 @@ void TestQgsProcessing::parameterFeatureSink()

QCOMPARE( def->defaultFileExtension(), QStringLiteral( "shp" ) );
QCOMPARE( def->generateTemporaryDestination(), QStringLiteral( "memory:" ) );
def->setSupportsNonFileBasedOutputs( false );
def->setSupportsNonFileBasedOutput( false );
QVERIFY( def->generateTemporaryDestination().endsWith( QStringLiteral( ".shp" ) ) );
QVERIFY( def->generateTemporaryDestination().startsWith( QgsProcessingUtils::tempFolder() ) );

@@ -3985,7 +4002,7 @@ void TestQgsProcessing::parameterFeatureSink()
QCOMPARE( fromMap.flags(), def->flags() );
QCOMPARE( fromMap.defaultValue(), def->defaultValue() );
QCOMPARE( fromMap.dataType(), def->dataType() );
QCOMPARE( fromMap.supportsNonFileBasedOutputs(), def->supportsNonFileBasedOutputs() );
QCOMPARE( fromMap.supportsNonFileBasedOutput(), def->supportsNonFileBasedOutput() );
def.reset( dynamic_cast< QgsProcessingParameterFeatureSink *>( QgsProcessingParameters::parameterFromVariantMap( map ) ) );
QVERIFY( dynamic_cast< QgsProcessingParameterFeatureSink *>( def.get() ) );

@@ -4203,7 +4220,7 @@ void TestQgsProcessing::parameterRasterOut()
QCOMPARE( fromMap.description(), def->description() );
QCOMPARE( fromMap.flags(), def->flags() );
QCOMPARE( fromMap.defaultValue(), def->defaultValue() );
QCOMPARE( fromMap.supportsNonFileBasedOutputs(), def->supportsNonFileBasedOutputs() );
QCOMPARE( fromMap.supportsNonFileBasedOutput(), def->supportsNonFileBasedOutput() );
def.reset( dynamic_cast< QgsProcessingParameterRasterDestination *>( QgsProcessingParameters::parameterFromVariantMap( map ) ) );
QVERIFY( dynamic_cast< QgsProcessingParameterRasterDestination *>( def.get() ) );

@@ -4323,7 +4340,7 @@ void TestQgsProcessing::parameterFileOut()
QCOMPARE( fromMap.flags(), def->flags() );
QCOMPARE( fromMap.defaultValue(), def->defaultValue() );
QCOMPARE( fromMap.fileFilter(), def->fileFilter() );
QCOMPARE( fromMap.supportsNonFileBasedOutputs(), def->supportsNonFileBasedOutputs() );
QCOMPARE( fromMap.supportsNonFileBasedOutput(), def->supportsNonFileBasedOutput() );
def.reset( dynamic_cast< QgsProcessingParameterFileDestination *>( QgsProcessingParameters::parameterFromVariantMap( map ) ) );
QVERIFY( dynamic_cast< QgsProcessingParameterFileDestination *>( def.get() ) );

@@ -4396,7 +4413,7 @@ void TestQgsProcessing::parameterFolderOut()
QCOMPARE( fromMap.description(), def->description() );
QCOMPARE( fromMap.flags(), def->flags() );
QCOMPARE( fromMap.defaultValue(), def->defaultValue() );
QCOMPARE( fromMap.supportsNonFileBasedOutputs(), def->supportsNonFileBasedOutputs() );
QCOMPARE( fromMap.supportsNonFileBasedOutput(), def->supportsNonFileBasedOutput() );
def.reset( dynamic_cast< QgsProcessingParameterFolderDestination *>( QgsProcessingParameters::parameterFromVariantMap( map ) ) );
QVERIFY( dynamic_cast< QgsProcessingParameterFolderDestination *>( def.get() ) );

@@ -5868,5 +5885,25 @@ void TestQgsProcessing::defaultExtensionsForProvider()
QCOMPARE( provider.defaultRasterFileExtension(), QStringLiteral( "mig" ) );
}

void TestQgsProcessing::supportsNonFileBasedOutput()
{
DummyAlgorithm alg( QStringLiteral( "test" ) );
DummyProvider p( QStringLiteral( "test_provider" ) );
alg.addDestParams();
// provider has no support for file based outputs, so both output parameters should deny support
alg.setProvider( &p );
QVERIFY( !static_cast< const QgsProcessingDestinationParameter * >( alg.destinationParameterDefinitions().at( 0 ) )->supportsNonFileBasedOutput() );
QVERIFY( !static_cast< const QgsProcessingDestinationParameter * >( alg.destinationParameterDefinitions().at( 1 ) )->supportsNonFileBasedOutput() );

DummyAlgorithm alg2( QStringLiteral( "test" ) );
DummyProvider p2( QStringLiteral( "test_provider" ) );
p2.supportsNonFileOutputs = true;
alg2.addDestParams();
// provider has support for file based outputs, but only first output parameter should indicate support (since the second has support explicitly denied)
alg2.setProvider( &p2 );
QVERIFY( static_cast< const QgsProcessingDestinationParameter * >( alg2.destinationParameterDefinitions().at( 0 ) )->supportsNonFileBasedOutput() );
QVERIFY( !static_cast< const QgsProcessingDestinationParameter * >( alg2.destinationParameterDefinitions().at( 1 ) )->supportsNonFileBasedOutput() );
}

QGSTEST_MAIN( TestQgsProcessing )
#include "testqgsprocessing.moc"

0 comments on commit bf19eb6

Please sign in to comment.
You can’t perform that action at this time.