Skip to content
Permalink
Browse files
[feature] virtual raster data provider (#44195)
* try to commit some changes in the branch

* addedd 3 file, cmake, vrp .h and .cpp

* updating VRP, no good results

* try to solve some issue to the virtual raster provider

* metadata class added, it should be developed

* registred in qgsproviderregistry the new VRP

* on line to test debug in the  new VRP

* chenge of 1_06

* minor change 2 06

* updating VRP, it can add some useless color to map canvas

* add baseclass test fro new provider

* Delete testqgsvirtualrasterprovider.cpp

* move VRP to src/providers

* Delete src/core/providers/virtualrasterprovider directory

* little change to cmake

* added as dynamic provider

* move the VRP to src/providers and let it work

* move the VRP to src/providers and let it work

* including qgis analysis

* updating block function with some calc tools

* delete block function, updating readblock function

* modified block method, used setValue --> to update with bits

* modified block method

* modified block method, adding takedata to the resultmatrix

* modified test for the new provider

* Add some checks in the block method, for raster entries

* tries for the calculator tool

* work on the hardcoded part and test

* work on the hardcoded test

* minor change in order to make the calc functionaltiy work

* minor change to the loop in which the computation occurs

* try to understand how qgsrastermatrixworks

* work on debugging new class

* work on debugging new class

* updating test and block method

* updating test and and block, now it works running in test mode

* updating test and and block, now it works running in test mode with bits

* updating test and and block, now it works running in test mode with bits

* update test

* update test and provider class

* add decodeuri and encodeui in metadata, at the moment they do nothing

* modify qgsrasterdataprovider by adding decoding and encoding uri methods

* workin on encode and decode funct

* work on encode/decode after Martin advices

* work on encode/decode after Martin advices, and update unit test

* work on encode/decode and update Cmakelist of provider test

* work on encode/decode and update Cmakelist and provider test

* work on encode/decode and update test + constructor

* work on encode/decode and constructor

* decode is now non trivial, he same as constructor

* working on encoding

* working on encoding/decoding

* change a little the code/decode mehods in order to return and get a struct as output/input

* change of decode method

* change the encode method, I should polish the code and optimize the encode for raster layers

* add flag ptr bool to decode mehod

* add flag ptr and modified the constructor of vrprovider

* work on the constructor and deconstructor, the flag does not work properly

* work on the constructor anf decoder

* work on the constructor and decoder

* work on the constructor and add some lines to popoluta mRasterEntries

* delete hardcoded data

* work on constructor and copy constructor 2

* add the parsing of the formula in the constructor, I should make a private var with calcnode

* add the parsing of the formula in the constructor, I should make a private var with calcnode2

* add some checks in the constructor2

* modify the copy constructor, add some code to adjust the qgsrasterdataprovider.sip

* modify checks in the constructor and in the decode method, in order to avoid nested if stat

* delete the logic in the decode method and add in the constructor

* delete the logic in the decode method and add in the constructor, add some redundant check in the constructor

* delete the logic in the decode method and add in the constructor, add some redundant check in the constructor 2

* test is now more general

* test is now more general2

* delete of a comment

* banned keywords check

* little error

* add copyright

* add copyright2

* add assignment operator= delete

* add documentation in qgsrasterprovider

* documentation

* modify qgsstringutils

* little modifications

* add the checkbox to the ui

* come back to old ui for raster calc

* adjust some qt::endl in the test of vrprovider

* start to work on the ui dialog

* layot of the code

* add some logic to the dialog of the raster calc

* add some test to qgisapp, need to modify the encoder and maybe put it in the qgrastercalcdialog

* modify some logic in rastercalcdialog and in the app, I should modify and move the encode method

* modify the comment

* add method to calcnode, modify the constructor of the provider and the open of the dialog

* add documentation

* some modifications according to the first review

* sadd some changes according to Martin's review

* change qgsrastercalcnode methods that returns raster names, small change to ui

* change the ui

* change the ui - using QString instead of QStringLiteral

* change the ui - using QString instead of QStringLiteral

* change name of subdirectory containing the virtual raster provider

* change name of subdirectory containing the virtual raster provider2

* change name of subdirectory containing the virtual raster provider3

* minor changes to makefiles

* minor change to dialog, to hide and show the output format and path

* provider key changed to the shorter virtualraster

* add some code to QgsRasterLayer::encodedSource and decodedSource

* delete some comments

* updating VRP, no good results

* made some change after the the second review

* delete the comments

* change some part of the code (delete of comments and cleaning the code)

* correct some layout errors

* change the mail contatct in my prvider

* change details on the method in qgsrastercalcnode

* change some method of the test class of my provider

* delete some leakage in the virtual raster provider

* modify some parts according to the review + change two lines of code for encoding/decoding methods

* modify a line of qgisapp.cpp

* small change according to the comments on th PR

* add a test method for second generation virtual raster and modify checks in qgisapp

* correction of the test method

Co-authored-by: franc <Franc-Brs>
  • Loading branch information
Franc-Brs committed Aug 17, 2021
1 parent 9a4f0f5 commit 179cb1d9517dab8ea4303fe2a69b3a74adf77b79
@@ -98,6 +98,25 @@ Returns a list of nodes of a specific ``type``

static QgsRasterCalcNode *parseRasterCalcString( const QString &str, QString &parserErrorMsg ) /Factory/;

QStringList referencedLayerNames( );
%Docstring
Returns a list of raster layer names that are referenced in the formula without the quotation marks.
It uses QgsRasterCalcNode.cleanRasterReferences

.. note::

since QGIS 3.22
%End

QStringList cleanRasterReferences();
%Docstring
Returns a list of raster layer references that are addressed in the formula, without quotation marks.

.. note::

since QGIS 3.22
%End

private:
QgsRasterCalcNode( const QgsRasterCalcNode &rh );
};
@@ -474,6 +474,50 @@ Returns a list of pyramid resampling method name and label pairs
for given provider
%End

struct VirtualRasterInputLayers
{
QString name;
QString uri;
QString provider;
};

struct VirtualRasterParameters
{
QgsCoordinateReferenceSystem crs;
QgsRectangle extent;
int width;
int height;
QString formula;
QList <QgsRasterDataProvider::VirtualRasterInputLayers> rInputLayers;

};

static QgsRasterDataProvider::VirtualRasterParameters decodeVirtualRasterProviderUri( const QString &uri, bool *ok = 0 );
%Docstring
Decodes the URI returning a struct with all the parameters for :py:class:`QgsVirtualRasterProvider` class

.. note::

used by Virtual Raster Provider only

.. note::

since QGIS 3.22
%End

static QString encodeVirtualRasterProviderUri( const VirtualRasterParameters &parts );
%Docstring
Encodes the URI starting from the struct .

.. note::

used by Virtual Raster Provider only

.. note::

since QGIS 3.22
%End

virtual QString validateCreationOptions( const QStringList &createOptions, const QString &format );
%Docstring
Validates creation options for a specific dataset and destination format.
@@ -382,3 +382,38 @@ QgsRasterCalcNode *QgsRasterCalcNode::parseRasterCalcString( const QString &str,
return localParseRasterCalcString( str, parserErrorMsg );
}

QStringList QgsRasterCalcNode::referencedLayerNames()
{
QStringList referencedRasters;

QStringList rasterRef = this->cleanRasterReferences();
for ( const auto &i : rasterRef )
{
if ( referencedRasters.contains( i.mid( 0, i.lastIndexOf( "@" ) ) ) ) continue;
referencedRasters << i.mid( 0, i.lastIndexOf( "@" ) );
}

return referencedRasters;
}

QStringList QgsRasterCalcNode::cleanRasterReferences()
{
QStringList rasterReferences;
const QList<const QgsRasterCalcNode *> rasterRefNodes = this->findNodes( QgsRasterCalcNode::Type::tRasterRef );

for ( const QgsRasterCalcNode *r : rasterRefNodes )
{

QString layerRef( r->toString() );
if ( layerRef.at( 0 ) == QStringLiteral( "\"" ) && layerRef.at( layerRef.size() - 1 ) == QStringLiteral( "\"" ) )
{
layerRef.remove( 0, 1 );
layerRef.chop( 1 );

}
layerRef.remove( QChar( '\\' ), Qt::CaseInsensitive );
rasterReferences << layerRef;
}

return rasterReferences;
}
@@ -125,6 +125,19 @@ class ANALYSIS_EXPORT QgsRasterCalcNode

static QgsRasterCalcNode *parseRasterCalcString( const QString &str, QString &parserErrorMsg ) SIP_FACTORY;

/**
* Returns a list of raster layer names that are referenced in the formula without the quotation marks.
* It uses QgsRasterCalcNode::cleanRasterReferences
* \note since QGIS 3.22
*/
QStringList referencedLayerNames( );

/**
* Returns a list of raster layer references that are addressed in the formula, without quotation marks.
* \note since QGIS 3.22
*/
QStringList cleanRasterReferences();

private:
#ifdef SIP_RUN
QgsRasterCalcNode( const QgsRasterCalcNode &rh );
@@ -6562,7 +6562,47 @@ void QgisApp::newGpxLayer()
void QgisApp::showRasterCalculator()
{
QgsRasterCalcDialog d( qobject_cast<QgsRasterLayer *>( activeLayer() ), this );
if ( d.exec() == QDialog::Accepted )
if ( d.exec() != QDialog::Accepted )
{
return;
}
if ( d.useVirtualProvider() )
{
QgsRasterDataProvider::VirtualRasterParameters virtualCalcParams;
virtualCalcParams.crs = d.outputCrs();
virtualCalcParams.extent = d.outputRectangle();
virtualCalcParams.width = d.numberOfColumns();
virtualCalcParams.height = d.numberOfRows();
virtualCalcParams.formula = d.formulaString();

QString errorString;
std::unique_ptr< QgsRasterCalcNode > calcNodeApp( QgsRasterCalcNode::parseRasterCalcString( d.formulaString(), errorString ) );
if ( !calcNodeApp )
{
return;
}
QStringList rLayerDictionaryRef = calcNodeApp->cleanRasterReferences();
QSet<QPair<QString, QString>> uniqueRasterUriTmp;

for ( const auto &r : QgsRasterCalculatorEntry::rasterEntries() )
{
if ( ( ! rLayerDictionaryRef.contains( r.ref ) ) ||
uniqueRasterUriTmp.contains( QPair( r.raster->source(), r.ref.mid( 0, r.ref.lastIndexOf( "@" ) ) ) ) ) continue;
uniqueRasterUriTmp.insert( QPair( r.raster->source(), r.ref.mid( 0, r.ref.lastIndexOf( "@" ) ) ) );

QgsRasterDataProvider::VirtualRasterInputLayers projectRLayer;
projectRLayer.name = r.ref.mid( 0, r.ref.lastIndexOf( "@" ) );
projectRLayer.provider = r.raster->dataProvider()->name();
projectRLayer.uri = r.raster->source();

virtualCalcParams.rInputLayers.append( projectRLayer );
}

addRasterLayer( QgsRasterDataProvider::encodeVirtualRasterProviderUri( virtualCalcParams ),
d.virtualLayerName().isEmpty() ? d.formulaString() : d.virtualLayerName(),
QStringLiteral( "virtualraster" ) );
}
else
{
//invoke analysis library
QgsRasterCalculator rc( d.formulaString(),
@@ -6589,7 +6629,7 @@ void QgisApp::showRasterCalculator()
case QgsRasterCalculator::Success:
if ( d.addLayerToProject() )
{
addRasterLayer( d.outputFile(), QFileInfo( d.outputFile() ).completeBaseName(), QString() );
addRasterLayer( d.outputFile(), QFileInfo( d.outputFile() ).completeBaseName(), QStringLiteral( "gdal" ) );
}
visibleMessageBar()->pushMessage( tr( "Raster calculator" ),
tr( "Calculation complete." ),
@@ -95,6 +95,14 @@ QgsRasterCalcDialog::QgsRasterCalcDialog( QgsRasterLayer *rasterLayer, QWidget *
mOutputLayer->setDialogTitle( tr( "Enter result file" ) );
mOutputLayer->setDefaultRoot( settings.value( QStringLiteral( "/RasterCalculator/lastOutputDir" ), QDir::homePath() ).toString() );
connect( mOutputLayer, &QgsFileWidget::fileChanged, this, [ = ]() { setAcceptButtonState(); } );

connect( mUseVirtualProviderCheckBox, &QCheckBox::clicked, this, &QgsRasterCalcDialog::setOutputToVirtual );

if ( ! useVirtualProvider() )
{
setOutputToVirtual();
}

}

QString QgsRasterCalcDialog::formulaString() const
@@ -149,6 +157,16 @@ bool QgsRasterCalcDialog::addLayerToProject() const
return mAddResultToProjectCheckBox->isChecked();
}

bool QgsRasterCalcDialog::useVirtualProvider() const
{
return mUseVirtualProviderCheckBox->isChecked();
}

QString QgsRasterCalcDialog::virtualLayerName() const
{
return mVirtualLayerName->text();
}

QVector<QgsRasterCalculatorEntry> QgsRasterCalcDialog::rasterEntries() const
{
QVector<QgsRasterCalculatorEntry> entries;
@@ -305,7 +323,7 @@ void QgsRasterCalcDialog::mExpressionTextEdit_textChanged()
if ( expressionValid() )
{
mExpressionValidLabel->setText( tr( "Expression valid" ) );
if ( filePathValid() )
if ( filePathValid() || useVirtualProvider() )
{
mButtonBox->button( QDialogButtonBox::Ok )->setEnabled( true );
return;
@@ -320,7 +338,7 @@ void QgsRasterCalcDialog::mExpressionTextEdit_textChanged()

void QgsRasterCalcDialog::setAcceptButtonState()
{
if ( expressionValid() && filePathValid() )
if ( ( expressionValid() && filePathValid() ) || ( expressionValid() && useVirtualProvider() ) )
{
mButtonBox->button( QDialogButtonBox::Ok )->setEnabled( true );
}
@@ -330,6 +348,33 @@ void QgsRasterCalcDialog::setAcceptButtonState()
}
}

void QgsRasterCalcDialog::setOutputToVirtual()
{
if ( useVirtualProvider() )
{
mOutputFormatComboBox->hide();
mOutputLayer->hide();
mOutputLayerLabel->hide();
mOutputFormatLabel->hide();
mAddResultToProjectCheckBox->isChecked();
mAddResultToProjectCheckBox->setEnabled( false );
mVirtualLayerLabel->show();
mVirtualLayerName->show();
setAcceptButtonState();
}
else
{
mOutputFormatComboBox->show();
mOutputLayer->show();
mOutputLayerLabel->show();
mOutputFormatLabel->show();
mAddResultToProjectCheckBox->setEnabled( true );
mVirtualLayerLabel->hide();
mVirtualLayerName->hide();
setAcceptButtonState();
}
}

bool QgsRasterCalcDialog::expressionValid() const
{
QString errorString;
@@ -355,6 +400,8 @@ bool QgsRasterCalcDialog::filePathValid() const
void QgsRasterCalcDialog::mRasterBandsListWidget_itemDoubleClicked( QListWidgetItem *item )
{
mExpressionTextEdit->insertPlainText( quoteBandEntry( item->text() ) );
//to enable the "ok" button if someone checks the virtual provider checkbox before adding a valid expression,
if ( expressionValid() && useVirtualProvider() ) setAcceptButtonState();
}

void QgsRasterCalcDialog::mPlusPushButton_clicked()
@@ -42,6 +42,10 @@ class APP_EXPORT QgsRasterCalcDialog: public QDialog, private Ui::QgsRasterCalcD
QString outputFormat() const;
QgsCoordinateReferenceSystem outputCrs() const;
bool addLayerToProject() const;
//! True if Virtual is checked
bool useVirtualProvider() const;
//! Return the name written in the qlineedit
QString virtualLayerName() const;

//! Bounding box for output raster
QgsRectangle outputRectangle() const;
@@ -64,6 +68,8 @@ class APP_EXPORT QgsRasterCalcDialog: public QDialog, private Ui::QgsRasterCalcD
void mExpressionTextEdit_textChanged();
//! Enables OK button if calculator expression is valid and output file path exists
void setAcceptButtonState();
//! Disables some options that are not required if using Virtual Provider
void setOutputToVirtual();
void showHelp();

//calculator buttons

0 comments on commit 179cb1d

Please sign in to comment.