diff --git a/CHANGELOG.md b/CHANGELOG.md index b7cc7d4e8..db491f6cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,25 +1,61 @@ # CHANGELOG -## 4.7.0 +## 4.8.0 + +Melhorias: + +- Adicionada a opção de suprimir o formulário de feição no modo reclassificação do menu de aquisição (particularmente útil quando se está corrigindo flags de áreas sem centroide na construção de polígonos utilizando linha e centroide); +- Novo processo de identificar inconsistências entre os elementos da rede de drenagem; + +## 4.7.1 - 2023-05-10 + +Correção de bug: + +- Correção de bug no menu (filtro de geometria estava quebrado); + +## 4.7.0 - 2023-05-09 Novas funcionalidades: + - Novo processo de selecionar feições no canvas de camadas selecionadas; - Novo processo de filtrar lista de camadas no processing por tipo geométrico; - Novo processo de remover holes pequenos de camadas de cobertura; - Novo processo de dissolver polígonos para vizinhos (heurística pelo maior comprimento da intersecção); -- Novo processo de identificar inconsistências entre os elementos da rede de drenagem; +- Novo processo de construir grid de pontos dentro de polígonos; +- Novo processo de dividir polígonos; +- Novo processo de dividir polígonos por grid; +- Novo processo de selecionar por DE9IM; +- Novo processo de extrair feições por DE9IM; +- Processo de converter linha para multilinha portado do ferramentas experimentais; + Melhorias: + - Adicionada a opção de dar pan na barra de ferramentas de revisão; - Adicionada mudanca de ferramenta atual nos icones das ferramentas de filtro; +- Processing de construção do diagrama de elevação portado para o Ferramentas de Edição; +- Adicionado o comportamento no seletor genérico de selecionar somente na camada ativa quando a tecla Alt estiver selecionada; +- Adicionada a opção de rodar a construção de polígonos por polígono de área geográfica (por MI); +- Melhoria de desempenho na construção de polígonos (adicionado paralelismo em thread); +- Melhoria de desempenho na verificação de delimitadores não utilizados no processo de construção de polígonos; +- Adicionada a opção de verificar ou não delimitadores não utilizados no processo de construção de polígonos; +- Melhoria de desempenho na identificação de erros de construção do terreno (roda em thread por área geográfica); +- A ferramenta de verificação de erros de relacionamentos espaciais agora permite regras com de9im e relacionamentos espaciais simultaneamente; +- Adicionada a opção de desligar todas as imagens ativas na ferramenta de seleção de raster; +- Adicionado o id da geometria na flag do identificar geometrias inválidas; +- O menu de aquisição agora permite reclassificação de polígono para ponto (particularmente útil quando se está corrigindo flags de áreas sem centroide na construção de polígonos utilizando linha e centroide); Correção de bug: + - Corrigido o bug de sempre apontar flags quando a geometria tem buraco do processo de identificar geometrias com densidade incorreta de vértices; - Correção de bug no processo de adicionar vértice em segmento compartilhado; +- Correção de bug no processo de dissolver polígonos com mesmo conjunto de atributos quando é passada uma área mínima para o dissolve; +- Correção de bug no acesso ao BDGEx (a url do serviço mudou e o código teve de ser atualizado, mudando a url do serviço de https para http); -## 4.6.0 +## 4.6.0 - 2022-12-19 Novas funcionalidades: + - Novo processo de estender linhas próximas da moldura; - Novo algoritmo de detecção de geometrias nulas; - Novo processo de adicionar vértices não compartilhados nas intersecções (processo de correção associado ao processo de Identificar vértices não compartilhados na intersecção); @@ -32,6 +68,7 @@ Novas funcionalidades: - Nova funcionalidade de copiar geometrias selecionadas como WKT (portado do ferramentas experimentais); Melhorias: + - Adicionada a opção de atribuir um id de atividade para o grid de revisão criado no processo de criar grid de edição; - Melhorado o estilo do grid utilizado pela barra de ferramentas de revisão; - Adicionada a funcionalidade de resetar o grid na barra ferramentas de revisão; @@ -39,6 +76,7 @@ Melhorias: - Barra de atalhos refatorada. Alguns atalhos não utilizados frequentemente foram retirados e foram criadas novas barras para dar a opção do usuário escolher quais ele quer ativar. Correção de bug: + - Correção de bug no identificar pontas soltas (o algoritmo estava levantando flag em vértice ocupado dentro do raio de busca); - Correção de bug no identificar erros no terreno (o algoritmo estava levantando a geometria da flag confusa); - Correção de crash ao rodar o snap hierárquico (o algoritmo agora só transmite as mudanças para o banco ao final do processo, mantendo os cálculos intermediários em camada de cache gravadas em camada temporária do processing do QGIS, ativado por meio da flag is_child_algorithm=True ao rodar o processo); @@ -46,6 +84,7 @@ Correção de bug: ## 4.5.0 - 2022-09-08 Novas funcionalidades: + - Novo processo de identificar undershoot de polígonos; - Novo processo de identificar erros de construção de redes (linhas que compartilham vértices não segmentadas dentro da camada, linhas não segmentadas com as camadas de filtro); - Novo processo de identificar linhas com mesmo conjunto de atributos não unidas; @@ -61,12 +100,14 @@ Novas funcionalidades: - Novo processo de construir grid de revisão; Melhorias: + - Melhoria de desempenho no identificar Z; - Melhoria de desempenho no identificar geometrias inválidas; - Melhoria de desempenho no identificar dangles; - Melhoria no processo de validação do terreno (removidos os falso-positivos com a moldura); Correção de bug: + - Tratamento de geometria nula no Identify Out Of Bounds Angles in Coverage; ## 4.4.0 - 2022-07-12 @@ -79,6 +120,7 @@ Novas funcionalidades: - Novo processo de identificar feições com densidade alta de vértices; Melhorias: + - Refatoração da interface de carregamento de camadas (remoção de funcionalidades não utilizadas e melhoria no filtro de camadas); - Adicionadas flags de delimitador não utilizado no algoritmo Construir Polígonos com Delimitadores e Centroides; - Adicionada a opção de verificar geometrias inválidas nos polígonos montados no algoritmo Construir Polígonos com Delimitadores e Centroides; @@ -105,11 +147,13 @@ Correção de bugs: ## 4.3.1 - 2022-05-30 Novas funcionalidades: + - Adicionado processo de verificação de caracteres unicode; - Adicionados parâmetros de densidade de pontos na criação de molduras; - Adicionados novos casos no processo de identificação de geometrias inválidas (buraco intersectando fronteira de polígono); Correção de bugs: + - Correção no template da EDGV 3.0; - Correção nos endereços do BDGEx; - Correção na janela de opções do DSGTools; @@ -119,9 +163,11 @@ Correção de bugs: ## 4.3.0 - 2022-01-20 Novas funcionalidades: + - Novo menu de classificação Novos algoritmos: + - Corretor ortográfico - Verifica o UUID das feições - Verifica a sobreposição de curvas de nível @@ -130,12 +176,14 @@ Novos algoritmos: - Carrega um shapefile Melhorias: + - Adequação dos processings de camadas para ser compatível com o SAP - Compatibilidade com QGIS 3.22 Correção de bugs: + - Ferramenta de inspeção de feições, agora mostra a aproximação correta quando utilizado em linha ou áreas em latlong com porcentagem inferior a 100% - O problema onde a Ferramenta de Aquisição com Ângulos Retos e a Ferramenta de Aquisição à Mão Livre não atribuíam os valores padrões nos formulários da feição foi corrigido - Correção nos processings de geração de MI: remover MI que não existem -Changelog completo: https://github.com/dsgoficial/DsgTools/wiki/Changelog-4.3 +Changelog completo: diff --git a/DsgTools/Modules/acquisitionMenu/controllers/acquisitionMenuCtrl.py b/DsgTools/Modules/acquisitionMenu/controllers/acquisitionMenuCtrl.py index 3bc3ece62..35d084669 100644 --- a/DsgTools/Modules/acquisitionMenu/controllers/acquisitionMenuCtrl.py +++ b/DsgTools/Modules/acquisitionMenu/controllers/acquisitionMenuCtrl.py @@ -2,6 +2,7 @@ from PyQt5 import QtCore, uic, QtWidgets, QtGui from DsgTools.Modules.qgis.controllers.qgisCtrl import QgisCtrl import json +from qgis.core import QgsWkbTypes class AcquisitionMenuCtrl: @@ -169,7 +170,7 @@ def openReclassifyDialog(self, buttonConfig, callback): if len(layers) > 1: raise Exception("Há camadas repetidas!") layer = layers[0] - layerName = layer.dataProvider().uri().table() + layerName = layer.dataProvider().uri().table() if layer.providerType() == "postgres" else layer.name() layersToReclassification = self.getLayersForReclassification( layerName, layer.geometryType() ) @@ -180,6 +181,7 @@ def openReclassifyDialog(self, buttonConfig, callback): self.reclassifyDialog = self.widgetFactory.createWidget( "ReclassifyDialog", self ) + suppressReclassificationDialog = buttonConfig.get("buttonSuppressReclassificationForm", False) self.reclassifyDialog.setAttributeTableWidget(self.getAttributeTableWidget()) self.reclassifyDialog.loadAttributes( self.getAttributesConfigByLayerName(buttonConfig["buttonLayer"]) @@ -187,6 +189,10 @@ def openReclassifyDialog(self, buttonConfig, callback): self.reclassifyDialog.setAttributesValues(buttonConfig["buttonAttributes"]) self.reclassifyDialog.loadLayersStatus(layersToReclassification) self.reclassifyDialog.success.connect(callback) + if suppressReclassificationDialog: + self.reclassifyDialog.hide() + self.reclassifyDialog.on_saveBtn_clicked() + return self.reclassifyDialog.showTopLevel() def reclassify(self, buttonConfig, reclassifyData): @@ -205,10 +211,15 @@ def reclassify(self, buttonConfig, reclassifyData): def getLayersForReclassification(self, layerName, geometryType): layers = self.qgis.getLoadedVectorLayers() + geometryFilterDict = { + QgsWkbTypes.PointGeometry: (QgsWkbTypes.PointGeometry,), + QgsWkbTypes.LineGeometry: (QgsWkbTypes.LineGeometry,), + QgsWkbTypes.PolygonGeometry: (QgsWkbTypes.PointGeometry, QgsWkbTypes.PolygonGeometry) + } return [ l for l in layers - if l.geometryType() == geometryType and l.selectedFeatureCount() > 0 + if l.selectedFeatureCount() > 0 and l.geometryType() in geometryFilterDict[l.geometryType()] ] def activeMenuButton(self, buttonConfig): diff --git a/DsgTools/Modules/acquisitionMenu/uis/addButtonDialog.ui b/DsgTools/Modules/acquisitionMenu/uis/addButtonDialog.ui index 31a675a05..b6bbfa270 100644 --- a/DsgTools/Modules/acquisitionMenu/uis/addButtonDialog.ui +++ b/DsgTools/Modules/acquisitionMenu/uis/addButtonDialog.ui @@ -6,18 +6,34 @@ 0 0 - 559 - 725 + 495 + 626 Botão - - + + + + Botão ( preview ): + + + + + + + + + + + + + + - + 0 @@ -31,12 +47,12 @@ - Palavras Chaves (;): + Nome: - + @@ -44,10 +60,10 @@ - - + + - + 0 @@ -61,23 +77,81 @@ - Nome: + Aba: - + + + + 0 + + + 0 + + + 0 + + + 0 + + + 0 + + + + + + + + + + + + + 0 + 0 + + + + + 165 + 32 + + - + Ferramenta de Aquisição: + + + + + 0 + + + 0 + + + 0 + + + 0 + + + 0 + + + + - - + + - + 0 @@ -91,19 +165,19 @@ - Cor de Fundo: + Cor do Texto: - + - + Qt::Horizontal @@ -117,53 +191,10 @@ - - - - - 0 - - - 0 - - - - - - - - - - Qt::Horizontal - - - - 168 - 20 - - - - - - - - Cancelar - - - - - - - Salvar - - - - - - - + + - + 0 @@ -177,28 +208,36 @@ - Aba: + Cor de Fundo: - - - - 0 - - - 0 - - + + + + + + + + Qt::Horizontal + + + + 336 + 0 + + + + - - + + - + 0 @@ -212,17 +251,26 @@ - Ferramenta de Aquisição: + Camada: - - + + 0 - + + 0 + + + 0 + + + 0 + + 0 @@ -230,10 +278,10 @@ - - + + - + 0 @@ -247,31 +295,19 @@ - Camada: + Palavras Chaves (;): - - - - 0 - - - 0 - - + + + + - - - - Atributos: - - - @@ -314,10 +350,16 @@ - 165 + 240 32 + + + 240 + 16777215 + + Suprimir Formulário: @@ -330,28 +372,25 @@ + + + + Qt::Horizontal + + + + 336 + 20 + + + + - - - - Botão ( preview ): - - - - - - - - - - - - - - + + - + 0 @@ -360,37 +399,102 @@ - 165 + 240 32 + + + 240 + 16777215 + + - Cor do Texto: + Suprimir Formulário de Relassificação: - + - + Qt::Horizontal 336 - 0 + 20 + + + + Atributos: + + + + + + + + 0 + + + 0 + + + 0 + + + 0 + + + 0 + + + + + + + + + + Qt::Horizontal + + + + 168 + 20 + + + + + + + + Cancelar + + + + + + + Salvar + + + + + diff --git a/DsgTools/Modules/acquisitionMenu/widgets/addButtonDialog.py b/DsgTools/Modules/acquisitionMenu/widgets/addButtonDialog.py index df161121c..cbfa059ef 100644 --- a/DsgTools/Modules/acquisitionMenu/widgets/addButtonDialog.py +++ b/DsgTools/Modules/acquisitionMenu/widgets/addButtonDialog.py @@ -125,6 +125,7 @@ def setData(self, buttonConfig): self.keyWordsLe.setText(buttonConfig["buttonKeyWords"]) self.tooltipLe.setText(buttonConfig["buttonTooltip"]) self.suppressFormCkb.setChecked(buttonConfig["buttonSuppressForm"]) + self.suppressReclassificationFormCkb.setChecked(buttonConfig.get("buttonSuppressReclassificationForm", False)) self.previewBtn.setText(buttonConfig["buttonName"]) self.setColorPreviewButton( @@ -160,6 +161,7 @@ def getData(self): "buttonKeyWords": self.keyWordsLe.text(), "buttonTooltip": self.tooltipLe.text(), "buttonSuppressForm": self.suppressFormCkb.isChecked(), + "buttonSuppressReclassificationForm": self.suppressReclassificationFormCkb.isChecked(), } def validData(self): diff --git a/DsgTools/Modules/qgis/controllers/qgisCtrl.py b/DsgTools/Modules/qgis/controllers/qgisCtrl.py index c285c9620..36241dee9 100644 --- a/DsgTools/Modules/qgis/controllers/qgisCtrl.py +++ b/DsgTools/Modules/qgis/controllers/qgisCtrl.py @@ -21,6 +21,10 @@ def getLoadedVectorLayerNames(self): layerName = ( l.dataProvider().uri().uri().split("|")[-1].split("=")[-1][1:-1] ) + if layerName == '': + layerName = l.name() + else: + layerName = l.name() if not layerName: continue layerNames.append(layerName) @@ -38,6 +42,10 @@ def getVectorLayerNames(self, layers): layerName = ( l.dataProvider().uri().uri().split("|")[-1].split("=")[-1][1:-1] ) + if layerName == '': + layerName = l.name() + else: + layerName = l.name() if not layerName: continue layerNames.append(layerName) @@ -62,6 +70,10 @@ def getVectorLayersByName(self, name): layerName = ( l.dataProvider().uri().uri().split("|")[-1].split("=")[-1][1:-1] ) + if layerName == '': + layerName = l.name() + else: + layerName = l.name() if not layerName or layerName != name: continue layers.append(l) @@ -210,7 +222,11 @@ def cutAndPasteSelectedFeatures(self, layer, destinatonLayer, attributes): for feature in features: newFeat = core.QgsFeature() newFeat.setFields(destinatonLayer.fields()) - newFeat.setGeometry(feature.geometry()) + newGeom = feature.geometry().pointOnSurface() \ + if destinatonLayer.geometryType() == core.QgsWkbTypes.PointGeometry \ + and layer.geometryType() == core.QgsWkbTypes.PolygonGeometry \ + else feature.geometry() + newFeat.setGeometry(newGeom) self.attributeFeature(newFeat, destinatonLayer, attributes) newFeatures.append(newFeat) layer.deleteSelectedFeatures() diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/buildTerrainSlicingFromContoursAlgorihtm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/buildTerrainSlicingFromContoursAlgorihtm.py deleted file mode 100644 index ee36661f9..000000000 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/buildTerrainSlicingFromContoursAlgorihtm.py +++ /dev/null @@ -1,528 +0,0 @@ -# -*- coding: utf-8 -*- -""" -/*************************************************************************** - DsgTools - A QGIS plugin - Brazilian Army Cartographic Production Tools - ------------------- - begin : 2022-08-16 - git sha : $Format:%H$ - copyright : (C) 2022 by Philipe Borba - Cartographic Engineer @ Brazilian Army - email : borba.philipe@eb.mil.br - ***************************************************************************/ - -/*************************************************************************** - * * - * This program is free software; you can redistribute it and/or modify * - * it under the terms of the GNU General Public License as published by * - * the Free Software Foundation; either version 2 of the License, or * - * (at your option) any later version. * - * * - ***************************************************************************/ -""" - -import numpy as np -import processing -from osgeo import gdal -from PyQt5.QtCore import QCoreApplication, QVariant -from qgis.core import ( - QgsFeature, - QgsFeatureSink, - QgsField, - QgsFields, - QgsGeometry, - QgsProcessing, - QgsProcessingAlgorithm, - QgsProcessingMultiStepFeedback, - QgsProcessingParameterFeatureSink, - QgsProcessingParameterFeatureSource, - QgsProcessingParameterNumber, - QgsProcessingParameterRasterDestination, - QgsProcessingParameterRasterLayer, - QgsProcessingUtils, - QgsProject, - QgsVectorLayer, - QgsWkbTypes, -) - -from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner -from DsgTools.core.GeometricTools.geometryHandler import GeometryHandler - - -class BuildTerrainSlicingFromContoursAlgorihtm(QgsProcessingAlgorithm): - - INPUT = "INPUT" - CONTOUR_INTERVAL = "CONTOUR_INTERVAL" - GEOGRAPHIC_BOUNDARY = "GEOGRAPHIC_BOUNDARY" - AREA_WITHOUT_INFORMATION_POLYGONS = "AREA_WITHOUT_INFORMATION_POLYGONS" - WATER_BODIES_POLYGONS = "WATER_BODIES_POLYGONS" - MIN_PIXEL_GROUP_SIZE = "MIN_PIXEL_GROUP_SIZE" - SMOOTHING_PARAMETER = "SMOOTHING_PARAMETER" - OUTPUT_POLYGONS = "OUTPUT_POLYGONS" - OUTPUT_RASTER = "OUTPUT_RASTER" - - def initAlgorithm(self, config=None): - self.addParameter( - QgsProcessingParameterRasterLayer( - self.INPUT, - self.tr("Input DEM"), - ) - ) - self.addParameter( - QgsProcessingParameterNumber( - self.CONTOUR_INTERVAL, - self.tr("Equidistance value"), - type=QgsProcessingParameterNumber.Integer, - minValue=0, - defaultValue=10, - ) - ) - - self.addParameter( - QgsProcessingParameterFeatureSource( - self.GEOGRAPHIC_BOUNDARY, - self.tr("Geographic bounds layer"), - [QgsProcessing.TypeVectorPolygon], - optional=False, - ) - ) - - self.addParameter( - QgsProcessingParameterFeatureSource( - self.AREA_WITHOUT_INFORMATION_POLYGONS, - self.tr("Area without information layer"), - [QgsProcessing.TypeVectorPolygon], - optional=True, - ) - ) - - self.addParameter( - QgsProcessingParameterFeatureSource( - self.WATER_BODIES_POLYGONS, - self.tr("Water bodies layer"), - [QgsProcessing.TypeVectorPolygon], - optional=True, - ) - ) - - self.addParameter( - QgsProcessingParameterNumber( - self.MIN_PIXEL_GROUP_SIZE, - self.tr("Minimum pixel group size"), - type=QgsProcessingParameterNumber.Integer, - minValue=0, - defaultValue=100, - ) - ) - - self.addParameter( - QgsProcessingParameterNumber( - self.SMOOTHING_PARAMETER, - self.tr("Smoothing parameter"), - type=QgsProcessingParameterNumber.Double, - minValue=0, - defaultValue=0.001, - ) - ) - - self.addParameter( - QgsProcessingParameterFeatureSink( - self.OUTPUT_POLYGONS, self.tr("Output polygons") - ) - ) - - self.addParameter( - QgsProcessingParameterRasterDestination( - self.OUTPUT_RASTER, self.tr("Output slicing") - ) - ) - - def processAlgorithm(self, parameters, context, feedback): - algRunner = AlgRunner() - self.geometryHandler = GeometryHandler() - inputRaster = self.parameterAsRasterLayer(parameters, self.INPUT, context) - threshold = self.parameterAsInt(parameters, self.CONTOUR_INTERVAL, context) - geoBoundsSource = self.parameterAsSource( - parameters, self.GEOGRAPHIC_BOUNDARY, context - ) - areaWithoutInformationSource = self.parameterAsSource( - parameters, self.AREA_WITHOUT_INFORMATION_POLYGONS, context - ) - waterBodiesSource = self.parameterAsSource( - parameters, self.WATER_BODIES_POLYGONS, context - ) - minPixelGroupSize = self.parameterAsInt( - parameters, self.MIN_PIXEL_GROUP_SIZE, context - ) - smoothingThreshold = self.parameterAsDouble( - parameters, self.SMOOTHING_PARAMETER, context - ) - outputRaster = self.parameterAsOutputLayer( - parameters, self.OUTPUT_RASTER, context - ) - outputFields = self.getOutputFields() - (output_sink, output_sink_id) = self.getOutputSink( - inputRaster, outputFields, parameters, context - ) - - multiStepFeedback = QgsProcessingMultiStepFeedback( - 15, feedback - ) # ajustar depois - currentStep = 0 - multiStepFeedback.setCurrentStep(currentStep) - - geographicBounds = ( - self.overlayPolygonLayer( - inputLyr=parameters[self.GEOGRAPHIC_BOUNDARY], - polygonLyr=parameters[self.AREA_WITHOUT_INFORMATION_POLYGONS], - crs=inputRaster.crs() - if inputRaster is not None - else QgsProject.instance().crs(), - context=context, - feedback=multiStepFeedback, - operator=2, - ) - if areaWithoutInformationSource is not None - and areaWithoutInformationSource.featureCount() > 0 - else parameters[self.GEOGRAPHIC_BOUNDARY] - ) - - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - geographicBounds = ( - self.overlayPolygonLayer( - inputLyr=geographicBounds, - polygonLyr=parameters[self.WATER_BODIES_POLYGONS], - crs=inputRaster.crs() - if inputRaster is not None - else QgsProject.instance().crs(), - context=context, - feedback=multiStepFeedback, - operator=2, - ) - if waterBodiesSource is not None and waterBodiesSource.featureCount() > 0 - else geographicBounds - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - clippedRaster = algRunner.runClipRasterLayer( - inputRaster, - mask=geographicBounds, - context=context, - feedback=multiStepFeedback, - noData=-9999, - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - slicedDEM = algRunner.runGrassMapCalcSimple( - inputA=clippedRaster, - expression=f"{threshold} * floor(A / {threshold})", - context=context, - feedback=multiStepFeedback, - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - slicingThresholdDict = self.findSlicingThresholdDict(slicedDEM) - expression = "\n".join( - [f"{a} thru {b} = {i}" for i, (a, b) in slicingThresholdDict.items()] - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - bufferedGeographicBounds = algRunner.runBuffer( - parameters[self.GEOGRAPHIC_BOUNDARY], - distance=10 * smoothingThreshold, - context=context, - feedback=multiStepFeedback, - ) - currentStep += 1 - multiStepFeedback.setCurrentStep(currentStep) - clippedRaster = algRunner.runClipRasterLayer( - inputRaster, - mask=bufferedGeographicBounds, - context=context, - feedback=multiStepFeedback, - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - classifiedRaster = algRunner.runGrassReclass( - clippedRaster, expression, context=context, feedback=multiStepFeedback - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - sieveOutput = algRunner.runSieve( - classifiedRaster, - threshold=minPixelGroupSize, - context=context, - feedback=multiStepFeedback, - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - finalRaster = algRunner.runClipRasterLayer( - sieveOutput, - mask=geographicBounds, - context=context, - feedback=multiStepFeedback, - outputRaster=outputRaster, - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - polygonLayer = algRunner.runGdalPolygonize( - sieveOutput, - context=context, - feedback=multiStepFeedback, - is_child_algorithm=True, - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - smoothPolygons = ( - algRunner.runChaikenSmoothing( - polygonLayer, - threshold=smoothingThreshold, - context=context, - feedback=multiStepFeedback, - is_child_algorithm=True, - ) - if smoothingThreshold > 0 - else polygonLayer - ) - currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - overlayedPolygons = self.overlayPolygonLayer( - inputLyr=smoothPolygons, - polygonLyr=geographicBounds, - crs=inputRaster.crs() - if inputRaster is not None - else QgsProject.instance().crs(), - context=context, - feedback=multiStepFeedback, - ) - currentStep += 1 - # multiStepFeedback.setCurrentStep(currentStep) - # cleanedPolygons = self.cleanPolyonLayer() - # currentStep += 1 - multiStepFeedback.setCurrentStep(currentStep) - algRunner.runCreateSpatialIndex( - overlayedPolygons, context, feedback=multiStepFeedback - ) - currentStep += 1 - - featCount = overlayedPolygons.featureCount() - if featCount == 0: - return { - "OUTPUT_POLYGONS": output_sink_id, - "OUTPUT_RASTER": finalRaster, - } - - multiStepFeedback.setCurrentStep(currentStep) - stepSize = 100 / featCount - valueSet = set(int(feat["a_DN"]) for feat in overlayedPolygons.getFeatures()) - diff = valueSet.difference(set(range(len(valueSet)))) - - def classLambda(x): - x = int(x) - if x == 0: - return x - return x - 1 if diff != set() else x - - for current, feat in enumerate(overlayedPolygons.getFeatures()): - if multiStepFeedback.isCanceled(): - break - newFeat = QgsFeature(outputFields) - newFeat["class"] = classLambda(feat["a_DN"]) - newFeat["class_min"], newFeat["class_max"] = slicingThresholdDict[ - feat["a_DN"] - ] - geom = self.validatePolygon(feat, overlayedPolygons) - newFeat.setGeometry(geom) - output_sink.addFeature(newFeat, QgsFeatureSink.FastInsert) - multiStepFeedback.setProgress(current * stepSize) - - return { - "OUTPUT_POLYGONS": output_sink_id, - "OUTPUT_RASTER": finalRaster, - } - - def validatePolygon(self, feat, overlayerPolygons): - geom = feat.geometry() - _, donutholes = self.geometryHandler.getOuterShellAndHoles(geom, False) - filteredHoles = [] - holesIdsToDelete = set() - if donutholes == []: - return geom - - def holeWithValue(centerPoint): - centerPointBB = centerPoint.boundingBox() - for polygonFeat in overlayerPolygons.getFeatures(centerPointBB): - polygonGeom = polygonFeat.geometry() - if polygonGeom.equals(geom): - continue - if polygonGeom.intersects(centerPoint): - return True - return False - - for idx, hole in enumerate(donutholes): - centerPoint = hole.pointOnSurface() - hasValue = holeWithValue(centerPoint) - if not hasValue: - holesIdsToDelete.add(idx + 1) - continue - filteredHoles.append(hole) - if donutholes == filteredHoles: - return geom - geom = QgsGeometry(geom) - for idx in holesIdsToDelete: - geom.deleteRing(idx) - return geom - - def overlayPolygonLayer( - self, inputLyr, polygonLyr, crs, context, feedback, operator=0 - ): - parameters = { - "ainput": inputLyr, - "atype": 0, - "binput": polygonLyr, - "btype": 0, - "operator": operator, - "snap": 0, - "-t": False, - "output": "TEMPORARY_OUTPUT", - "GRASS_REGION_PARAMETER": None, - "GRASS_SNAP_TOLERANCE_PARAMETER": -1, - "GRASS_MIN_AREA_PARAMETER": 1e-15, - "GRASS_OUTPUT_TYPE_PARAMETER": 3, - "GRASS_VECTOR_DSCO": "", - "GRASS_VECTOR_LCO": "", - "GRASS_VECTOR_EXPORT_NOCAT": False, - } - x = processing.run( - "grass7:v.overlay", parameters, context=context, feedback=feedback - ) - lyr = QgsProcessingUtils.mapLayerFromString(x["output"], context) - lyr.setCrs(crs) - return lyr - - def findSlicingThresholdDict(self, inputRaster): - ds = gdal.Open(inputRaster) - npRaster = np.array(ds.GetRasterBand(1).ReadAsArray()) - npRaster = npRaster[~np.isnan(npRaster)] # removes nodata values - minValue = np.amin(npRaster) - maxValue = np.amax(npRaster) - numberOfElevationBands = self.getNumberOfElevationBands(maxValue - minValue) - areaRatioList = self.getAreaRatioList(numberOfElevationBands) - uniqueValues, uniqueCount = np.unique(npRaster, return_counts=True) - cumulativePercentage = np.cumsum(uniqueCount) / np.prod(npRaster.shape) - areaPercentageValues = uniqueCount / np.prod(npRaster.shape) - if any(areaPercentageValues >= 0.48) and numberOfElevationBands > 2: - """ - The MTM spec states that if there is an elevation slice that covers more than - 50% of the map, there must only be 2 elevation bands. - """ - idx = np.argmax(areaPercentageValues >= 0.5) - if idx == 0: - return { - 0: (int(uniqueValues[0]), int(uniqueValues[1])), - 1: (int(uniqueValues[1]), int(uniqueValues[-1])), - } - elif idx == len(areaPercentageValues): - return { - 0: (int(uniqueValues[0]), int(uniqueValues[-2])), - 1: (int(uniqueValues[-2]), int(uniqueValues[-1])), - } - else: - return { - 0: (int(uniqueValues[0]), int(uniqueValues[idx])), - 1: (int(uniqueValues[idx]), int(uniqueValues[idx + 1])), - } - - if numberOfElevationBands == 2 and np.argmax(areaPercentageValues >= 0.5) == 0: - return { - 0: (int(uniqueValues[0]), int(uniqueValues[1])), - 1: (int(uniqueValues[1]), int(uniqueValues[-1])), - } - - classThresholds = list( - uniqueValues[ - np.searchsorted(cumulativePercentage, np.cumsum(areaRatioList)) - ] - ) - classDict = dict() - lowerBounds = ( - [minValue] + classThresholds - if minValue not in classThresholds - else classThresholds - ) - for i, (a, b) in enumerate(zip(lowerBounds, classThresholds)): - classDict[i] = (int(a), int(b)) - return classDict - - def getAreaRatioList(self, numberOfElevationBands): - bandDict = { - 2: [0.6, 0.4], - 3: [0.3, 0.4, 0.3], - 4: [0.2, 0.3, 0.3, 0.2], - } - return bandDict[numberOfElevationBands] - - def getOutputSink(self, inputRaster, outputFields, parameters, context): - return self.parameterAsSink( - parameters, - self.OUTPUT_POLYGONS, - context, - outputFields, - QgsWkbTypes.Polygon, - inputRaster.crs() - if inputRaster is not None - else QgsProject.instance().crs(), - ) - - def getNumberOfElevationBands(self, range): - if range <= 100: - return 2 - elif range <= 600: - return 3 - else: - return 4 - - def getOutputFields(self): - fields = QgsFields() - fields.append(QgsField("class", QVariant.Int)) - fields.append(QgsField("class_min", QVariant.Int)) - fields.append(QgsField("class_max", QVariant.Int)) - - return fields - - def tr(self, string): - return QCoreApplication.translate( - "BuildTerrainSlicingFromContoursAlgorihtm", string - ) - - def createInstance(self): - return BuildTerrainSlicingFromContoursAlgorihtm() - - def name(self): - return "buildterrainslicingfromcontours" - - def displayName(self): - return self.tr("Build Terrain Slicing from Contours") - - def group(self): - return self.tr("Geometric Algorithms") - - def groupId(self): - return "DSGTools: Geometric Algorithms" - - def shortHelpString(self): - return self.tr( - "O algoritmo constrói o fatiamento do terreno baseado nas curvas de nível." - ) diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/extractByDE9IM.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/extractByDE9IM.py new file mode 100644 index 000000000..9875090a4 --- /dev/null +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/extractByDE9IM.py @@ -0,0 +1,229 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-04-28 + git sha : $Format:%H$ + copyright : (C) 2023 by Philipe Borba - Cartographic Engineer @ Brazilian Army + email : borba.philipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +from collections import defaultdict +import itertools +import json +import os + +import concurrent.futures + +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.validationAlgorithm import ( + ValidationAlgorithm, +) +from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner +from DsgTools.core.GeometricTools.featureHandler import FeatureHandler +from DsgTools.core.GeometricTools.layerHandler import LayerHandler + +from qgis.PyQt.Qt import QVariant +from PyQt5.QtCore import QCoreApplication, QRegExp, QCoreApplication +from qgis.PyQt.QtGui import QRegExpValidator + +from qgis.core import ( + QgsProcessing, + QgsProcessingParameterFeatureSink, + QgsProcessingParameterVectorLayer, + QgsProcessingParameterBoolean, + QgsProcessingParameterField, + QgsProcessingException, + QgsProcessingParameterDistance, + QgsProcessingMultiStepFeedback, + QgsProcessingFeatureSourceDefinition, + QgsGeometry, + QgsProcessingParameterString, + QgsProcessingParameterNumber, + QgsProcessingParameterExpression, + QgsFeatureRequest, + QgsProcessingContext, + QgsProcessingAlgorithm, + QgsProcessingParameterFeatureSource, + QgsSpatialIndex, +) + + +class ValidationString(QgsProcessingParameterString): + """ + Auxiliary class for pre validation on measurer's names. + """ + + # __init__ not necessary + + def __init__(self, name, description=""): + super().__init__(name, description) + + def checkValueIsAcceptable(self, value, context=None): + regex = QRegExp("[FfTt012\*]{9}") + acceptable = QRegExpValidator.Acceptable + return ( + isinstance(value, str) + and QRegExpValidator(regex).validate(value, 9)[0] == acceptable + ) + + +class ExtractByDE9IMAlgorithm(QgsProcessingAlgorithm): + INPUT = "INPUT" + INTERSECT = "INTERSECT" + DE9IM = "DE9IM" + OUTPUT = "OUTPUT" + + def initAlgorithm(self, config): + """ + Parameter setting. + """ + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INPUT, + self.tr("Select features from"), + [QgsProcessing.TypeVectorAnyGeometry], + ) + ) + + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INTERSECT, + self.tr("By comparing features from"), + [QgsProcessing.TypeVectorAnyGeometry], + ) + ) + + param = ValidationString(self.DE9IM, description=self.tr("DE9IM")) + self.addParameter(param) + + self.addParameter( + QgsProcessingParameterFeatureSink(self.OUTPUT, self.tr("Output")) + ) + + def processAlgorithm(self, parameters, context, feedback): + """ + Here is where the processing itself takes place. + """ + self.layerHandler = LayerHandler() + self.algRunner = AlgRunner() + source = self.parameterAsSource(parameters, self.INPUT, context) + layer = self.parameterAsVectorLayer(parameters, self.INPUT, context) + intersectSource = self.parameterAsSource(parameters, self.INTERSECT, context) + de9im = self.parameterAsString(parameters, self.DE9IM, context) + + (sink, dest_id) = self.parameterAsSink( + parameters, + self.OUTPUT, + context, + source.fields(), + source.wkbType(), + source.sourceCrs(), + ) + + nFeats = intersectSource.featureCount() + if nFeats == 0: + return {self.OUTPUT: dest_id} + if de9im == "FF1FF0102": + return self.algRunner.runExtractByLocation( + inputLyr=parameters[self.INPUT], + intersectLyr=parameters[self.INTERSECT], + context=context, + feedback=feedback, + predicate=[2], + method=0, + is_child_algorithm=False, + ) + nSteps = 2 + multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + selectedLyr = self.algRunner.runExtractByLocation( + inputLyr=parameters[self.INPUT], + intersectLyr=parameters[self.INTERSECT], + context=context, + feedback=multiStepFeedback, + predicate=[2] if de9im == "FF1FF0102" else [0], + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + nFeats = selectedLyr.featureCount() + if nFeats == 0: + return {self.OUTPUT: dest_id} + stepSize = 100 / nFeats + + def compute(feat): + returnSet = set() + geom = feat.geometry() + bbox = geom.boundingBox() + engine = QgsGeometry.createGeometryEngine(geom.constGet()) + engine.prepareGeometry() + for f in selectedLyr.getFeatures(bbox): + if multiStepFeedback.isCanceled(): + return {} + intersectGeom = f.geometry() + if intersectGeom.isEmpty() or intersectGeom.isNull(): + continue + if engine.relatePattern(intersectGeom.constGet(), de9im): + returnSet.add(f) + return returnSet + + for current, feat in enumerate(intersectSource.getFeatures()): + if multiStepFeedback.isCanceled(): + return {} + outputSet = compute(feat) + sink.addFeatures(list(outputSet)) + multiStepFeedback.setProgress(current * stepSize) + + return {self.OUTPUT: dest_id} + + def name(self): + """ + Returns the algorithm name, used for identifying the algorithm. This + string should be fixed for the algorithm, and must not be localised. + The name should be unique within each provider. Names should contain + lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "extractbyde9im" + + def displayName(self): + """ + Returns the translated algorithm name, which should be used for any + user-visible display of the algorithm name. + """ + return self.tr("Extract features by DE9IM") + + def group(self): + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Geometric Algorithms") + + def groupId(self): + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Geometric Algorithms" + + def tr(self, string): + return QCoreApplication.translate("ExtractByDE9IMAlgorithm", string) + + def createInstance(self): + return ExtractByDE9IMAlgorithm() diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/line2Multiline.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/line2Multiline.py new file mode 100644 index 000000000..91a2326d0 --- /dev/null +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/line2Multiline.py @@ -0,0 +1,224 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-05-01 + git sha : $Format:%H$ + copyright : (C) 2023 by Jossan + email : + ***************************************************************************/ + +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +from collections import defaultdict +from qgis.PyQt.QtCore import QCoreApplication, QVariant +from qgis.core import ( + QgsProcessing, + QgsFeatureSink, + QgsProcessingAlgorithm, + QgsProcessingParameterFeatureSink, + QgsFeature, + QgsProcessingParameterFeatureSource, + QgsGeometry, + QgsLineString, + QgsProcessingMultiStepFeedback, + QgsWkbTypes, + QgsFields, + QgsField, + QgsMultiLineString, +) + +from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner + + +class Line2Multiline(QgsProcessingAlgorithm): + + INPUT = "INPUT" + OUTPUT = "OUTPUT" + + def initAlgorithm(self, config=None): + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INPUT, self.tr("Select line layer"), [QgsProcessing.TypeVectorLine] + ) + ) + + self.addParameter( + QgsProcessingParameterFeatureSink(self.OUTPUT, self.tr("Output")) + ) + + def processAlgorithm(self, parameters, context, feedback): + self.algRunner = AlgRunner() + lines = self.parameterAsSource(parameters, self.INPUT, context) + + fields = QgsFields() + fields.append(QgsField("length", QVariant.String)) + (sink_l, sinkId_l) = self.parameterAsSink( + parameters, + self.OUTPUT, + context, + fields, + QgsWkbTypes.MultiLineString, + lines.sourceCrs(), + ) + multiStepFeedback = QgsProcessingMultiStepFeedback(6, feedback) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + + lines = self.algRunner.runAddAutoIncrementalField( + inputLyr=parameters[self.INPUT], + context=context, + feedback=multiStepFeedback, + fieldName="AUTO", + ) + currentStep += 1 + + multiStepFeedback.setCurrentStep(currentStep) + self.algRunner.runCreateSpatialIndex( + inputLyr=lines, + context=context, + feedback=multiStepFeedback, + ) + currentStep += 1 + + multiStepFeedback.setCurrentStep(currentStep) + spatialJoinOutput = self.algRunner.runJoinAttributesByLocation( + inputLyr=lines, + joinLyr=lines, + context=context, + feedback=multiStepFeedback, + ) + currentStep += 1 + + multiStepFeedback.setCurrentStep(currentStep) + self.populateAuxStructure(lines, feedback=multiStepFeedback) + currentStep += 1 + + multiStepFeedback.setCurrentStep(currentStep) + self.buildMatchingFeaturesDict(spatialJoinOutput, feedback=multiStepFeedback) + currentStep += 1 + + multiStepFeedback.setCurrentStep(currentStep) + self.processFeatures(fields, sink_l, feedback=multiStepFeedback) + + return {self.OUTPUT: sinkId_l} + + def processFeatures(self, fields, sink_l, feedback): + nFeats = len(self.ids_in_stack) + if nFeats == 0: + return + stepSize = 100 / nFeats + current = 0 + while len(self.ids_in_stack) > 0: + if feedback.isCanceled(): + break + currentid = self.ids_in_stack.pop() + current = nFeats - len(self.ids_in_stack) + + mls_array = self.aggregate(currentid, feedback=feedback) + + mls = QgsMultiLineString() + for el in mls_array: + mls.addGeometry(QgsLineString(list(el.vertices()))) + self.addSink(QgsGeometry(mls), sink_l, fields) + feedback.setProgress(current * stepSize) + + def buildMatchingFeaturesDict(self, spatialJoinOutput, feedback): + self.matching_features = defaultdict(list) + nFeats = spatialJoinOutput.featureCount() + if nFeats == 0: + return + stepSize = 100 / nFeats + for current, feat in enumerate(spatialJoinOutput.getFeatures()): + if feedback.isCanceled(): + break + if feat["AUTO"] == feat["AUTO_2"]: + continue + self.matching_features[feat["AUTO"]].append(feat["AUTO_2"]) + feedback.setProgress(current * stepSize) + + def populateAuxStructure(self, lines, feedback): + self.id_to_feature = {} + self.ids_in_stack = set() + nFeats = lines.featureCount() + if nFeats == 0: + return + stepSize = 100 / nFeats + for current, currentFeature in enumerate(lines.getFeatures()): + if feedback.isCanceled(): + break + self.id_to_feature[currentFeature["AUTO"]] = currentFeature + self.ids_in_stack.add(currentFeature["AUTO"]) + feedback.setCurrentStep(current * stepSize) + + def aggregate(self, featureId, feedback): + stack = [featureId] + mls_array = [] + + while stack: + current_id = stack.pop() + currentfeature = self.id_to_feature[current_id] + currentgeom = currentfeature.geometry() + mls_array.append(currentgeom) + + if feedback.isCanceled(): + return mls_array + + matching_features_ids = set( + el for el in self.matching_features[current_id] if el in self.ids_in_stack + ) + + self.ids_in_stack = self.ids_in_stack - matching_features_ids + + stack.extend(matching_features_ids) + + return mls_array + + def addSink(self, geom, sink, fields): + newFeat = QgsFeature(fields) + newFeat.setGeometry(geom) + newFeat["length"] = geom.length() + sink.addFeature(newFeat, QgsFeatureSink.FastInsert) + + def tr(self, string): + return QCoreApplication.translate("Processing", string) + + def createInstance(self): + return Line2Multiline() + + def name(self): + return "line2multiline" + + def displayName(self): + return self.tr("Convert Line to Multiline") + + def group(self): + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Geometric Algorithms") + + def groupId(self): + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Geometric Algorithms" + + def shortHelpString(self): + return self.tr("O algoritmo converte linhas que se tocam para multilinha") diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/pointsInPolygonGridAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/pointsInPolygonGridAlgorithm.py new file mode 100644 index 000000000..d333eaad6 --- /dev/null +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/pointsInPolygonGridAlgorithm.py @@ -0,0 +1,150 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-04-15 + git sha : $Format:%H$ + copyright : (C) 2023 by Felipe Diniz - Cartographic Engineer @ Brazilian Army + email : diniz.felipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +from qgis.PyQt.QtCore import QCoreApplication +from qgis.core import ( + QgsProcessing, + QgsProcessingAlgorithm, + QgsProcessingParameterFeatureSource, + QgsProcessingParameterDistance, + QgsProcessingParameterFeatureSink, +) +import math +from qgis.core import QgsFeature, QgsGeometry, QgsPointXY, QgsWkbTypes +from itertools import product + + +class PointsInPolygonGridAlgorithm(QgsProcessingAlgorithm): + INPUT = "INPUT" + X_DISTANCE = "X_DISTANCE" + Y_DISTANCE = "Y_DISTANCE" + OUTPUT = "OUTPUT" + + def tr(self, string): + return QCoreApplication.translate("Processing", string) + + def createInstance(self): + return PointsInPolygonGridAlgorithm() + + def name(self): + return "points_in_polygon_grid_old" + + def displayName(self): + return self.tr("Points In Polygon Grid old") + + def group(self): + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Geometric Algorithms") + + def groupId(self): + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Geometric Algorithms" + + def shortHelpString(self): + return self.tr( + "Create a layer of points evenly spaced with X and Y distances for each polygon in the input layer." + ) + + def initAlgorithm(self, config=None): + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INPUT, self.tr("Input layer"), [QgsProcessing.TypeVectorPolygon] + ) + ) + self.addParameter( + QgsProcessingParameterDistance( + self.X_DISTANCE, self.tr("X Distance"), defaultValue=0.01 + ) + ) + self.addParameter( + QgsProcessingParameterDistance( + self.Y_DISTANCE, self.tr("Y Distance"), defaultValue=0.01 + ) + ) + self.addParameter( + QgsProcessingParameterFeatureSink( + self.OUTPUT, self.tr("Output point layer") + ) + ) + + def processAlgorithm(self, parameters, context, feedback): + source = self.parameterAsSource(parameters, self.INPUT, context) + x_distance = self.parameterAsDouble(parameters, self.X_DISTANCE, context) + y_distance = self.parameterAsDouble(parameters, self.Y_DISTANCE, context) + + fields = source.fields() + (sink, dest_id) = self.parameterAsSink( + parameters, + self.OUTPUT, + context, + fields, + QgsWkbTypes.Point, + source.sourceCrs(), + ) + + total = 100.0 / source.featureCount() if source.featureCount() else 0 + + for current, feature in enumerate(source.getFeatures()): + if feedback.isCanceled(): + break + + geometry = feature.geometry() + if geometry.isEmpty() or geometry.isNull(): + continue + + bbox = geometry.boundingBox() + x_min, y_min, x_max, y_max = ( + bbox.xMinimum(), + bbox.yMinimum(), + bbox.xMaximum(), + bbox.yMaximum(), + ) + + point_coordinates = ( + QgsGeometry.fromPointXY( + QgsPointXY(x_min + i * x_distance, y_min + j * y_distance) + ) + for i, j in product( + range(int(math.ceil((x_max - x_min) / x_distance)) + 1), + range(int(math.ceil((y_max - y_min) / y_distance)) + 1), + ) + ) + + for point_geom in point_coordinates: + if geometry.contains(point_geom): + new_feature = QgsFeature(fields) + new_feature.setGeometry(point_geom) + new_feature.setAttributes(feature.attributes()) + sink.addFeature(new_feature) + + feedback.setProgress(int(current * total)) + + return {self.OUTPUT: dest_id} diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/reclassifyAdjecentPolygonsAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/reclassifyAdjecentPolygonsAlgorithm.py index 077344257..48e4892cb 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/reclassifyAdjecentPolygonsAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/reclassifyAdjecentPolygonsAlgorithm.py @@ -20,6 +20,9 @@ ***************************************************************************/ """ +from collections import defaultdict +import itertools +import json import os import concurrent.futures @@ -29,7 +32,9 @@ ) from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner from DsgTools.core.GeometricTools.featureHandler import FeatureHandler +from DsgTools.core.GeometricTools.layerHandler import LayerHandler +from qgis.PyQt.Qt import QVariant from PyQt5.QtCore import QCoreApplication from qgis.core import ( @@ -43,13 +48,20 @@ QgsProcessingMultiStepFeedback, QgsProcessingFeatureSourceDefinition, QgsGeometry, + QgsProcessingParameterString, + QgsProcessingParameterNumber, + QgsProcessingParameterExpression, + QgsFeatureRequest, + QgsProcessingContext ) class ReclassifyAdjacentPolygonsAlgorithm(ValidationAlgorithm): INPUT = "INPUT" SELECTED = "SELECTED" - MAX_AREA = "MAX_AREA" + FILTER_EXPRESSION = "FILTER_EXPRESSION" + LABEL_FIELD = "LABEL_FIELD" + LABEL_ORDER = "LABEL_RULES" DISSOLVE_ATTRIBUTE_LIST = "DISSOLVE_ATTRIBUTE_LIST" DISSOLVE_OUTPUT = "DISSOLVE_OUTPUT" OUTPUT = "OUTPUT" @@ -72,14 +84,36 @@ def initAlgorithm(self, config): ) ) - param = QgsProcessingParameterDistance( - self.MAX_AREA, - self.tr("Maximum area"), - parentParameterName=self.INPUT, - defaultValue=0.0001, + self.addParameter( + QgsProcessingParameterField( + self.LABEL_FIELD, + self.tr("Class label field on input polygons"), + None, + self.INPUT, + QgsProcessingParameterField.Any, + allowMultiple=False, + ) + ) + + self.addParameter( + QgsProcessingParameterExpression( + self.FILTER_EXPRESSION, + self.tr("Filter expression for input"), + None, + self.INPUT, + optional=True + ) + ) + + self.addParameter( + QgsProcessingParameterString( + self.LABEL_ORDER, + description=self.tr("Label order"), + multiLine=False, + defaultValue="", + optional=True, + ) ) - param.setMetadata({"widget_wrapper": {"decimals": 10}}) - self.addParameter(param) self.addParameter( QgsProcessingParameterBoolean( @@ -98,7 +132,6 @@ def initAlgorithm(self, config): optional=True, ) ) - self.addParameter( QgsProcessingParameterFeatureSink(self.OUTPUT, self.tr("Output")) ) @@ -115,18 +148,30 @@ def processAlgorithm(self, parameters, context, feedback): "This algorithm requires the Python networkx library. Please install this library and try again." ) ) - algRunner = AlgRunner() + self.algRunner = AlgRunner() + self.layerHandler = LayerHandler() inputLyr = self.parameterAsVectorLayer(parameters, self.INPUT, context) if inputLyr is None: raise QgsProcessingException( self.invalidSourceError(parameters, self.INPUT) ) onlySelected = self.parameterAsBool(parameters, self.SELECTED, context) - maxAreaToDissolve = self.parameterAsDouble(parameters, self.MAX_AREA, context) + filterExpression = self.parameterAsExpression(parameters, self.FILTER_EXPRESSION, context) + if filterExpression == '': + filterExpression = None + classFieldName = self.parameterAsFields(parameters, self.LABEL_FIELD, context)[0] + labelListStr = self.parameterAsString(parameters, self.LABEL_ORDER, context) + classOrderList = None if labelListStr == '' else labelListStr.split(",") + field = [f for f in inputLyr.fields() if f.name() == classFieldName][0] + if field.type() == QVariant.Int: + classOrderList = list(map(int, classOrderList)) + elif field.type() == QVariant.Double: + classOrderList = list(map(float, classOrderList)) dissolveOutput = self.parameterAsBool(parameters, self.DISSOLVE_OUTPUT, context) dissolveFields = self.parameterAsFields( parameters, self.DISSOLVE_ATTRIBUTE_LIST, context ) + dissolveFields = list(set(dissolveFields).union({classFieldName})) (output_sink, output_sink_id) = self.parameterAsSink( parameters, self.OUTPUT, @@ -137,13 +182,12 @@ def processAlgorithm(self, parameters, context, feedback): ) if output_sink is None: raise QgsProcessingException(self.invalidSinkError(parameters, self.OUTPUT)) - nSteps = 7 if dissolveOutput else 6 + nSteps = 6 + (dissolveOutput is True) multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) currentStep = 0 - multiStepFeedback.setCurrentStep(currentStep) multiStepFeedback.setProgressText(self.tr("Creating cache layer")) - cacheLyr = algRunner.runCreateFieldWithExpression( + cacheLyr = self.algRunner.runCreateFieldWithExpression( inputLyr=inputLyr if not onlySelected else QgsProcessingFeatureSourceDefinition(inputLyr.id(), True), @@ -157,13 +201,13 @@ def processAlgorithm(self, parameters, context, feedback): multiStepFeedback.setCurrentStep(currentStep) multiStepFeedback.setProgressText(self.tr("Creating spatial index on cache")) - algRunner.runCreateSpatialIndex(cacheLyr, context, feedback=multiStepFeedback) + self.algRunner.runCreateSpatialIndex(cacheLyr, context, feedback=multiStepFeedback) currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) multiStepFeedback.setProgressText(self.tr("Building aux structures")) G, featDict, idSet = self.buildAuxStructures( - nx, cacheLyr, maxAreaToDissolve, multiStepFeedback + nx, cacheLyr, context, filterExpression=filterExpression, feedback=multiStepFeedback ) currentStep += 1 @@ -171,33 +215,35 @@ def processAlgorithm(self, parameters, context, feedback): anchorIdsSet = set(featDict.keys()).difference(idSet) fieldNames = [field.name() for field in inputLyr.fields()] multiStepFeedback.setProgressText(self.tr("Performing reclassification")) - featuresToUpdateSet = self.reclassifyPolygons( - G, featDict, anchorIdsSet, idSet, dissolveFields, multiStepFeedback + featureIdsToUpdateSet = self.reclassifyPolygons( + G=G, + featDict=featDict, + anchorIdsSet=anchorIdsSet, + candidateIdSet=idSet, + fieldNames=dissolveFields, + feedback=multiStepFeedback, + classFieldName=classFieldName, + classOrderList=classOrderList, ) currentStep += 1 multiStepFeedback.setProgressText(self.tr("Changing attributes from cache")) multiStepFeedback.setCurrentStep(currentStep) - nFeatsToUpdate = len(featuresToUpdateSet) + nFeatsToUpdate = len(featureIdsToUpdateSet) if nFeatsToUpdate == 0: return {self.OUTPUT: output_sink_id} stepSize = 100 / nFeatsToUpdate cacheLyr.startEditing() cacheLyr.beginEditCommand("Updating features") cacheLyrDataProvider = cacheLyr.dataProvider() - indexDict = { - fieldName: cacheLyrDataProvider.fields().indexFromName(fieldName) - for fieldName in fieldNames - } - for current, featid in enumerate(idSet): + fieldIdx = cacheLyrDataProvider.fields().indexFromName(classFieldName) + for current, (featid, classValue) in enumerate(featureIdsToUpdateSet): if multiStepFeedback.isCanceled(): break - # featid = feat['featid'] cacheLyrDataProvider.changeAttributeValues( { featid: { - indexDict[fieldName]: featDict[featid][fieldName] - for fieldName in dissolveFields + fieldIdx: classValue } } ) @@ -208,7 +254,7 @@ def processAlgorithm(self, parameters, context, feedback): multiStepFeedback.setCurrentStep(currentStep) if dissolveOutput: multiStepFeedback.setProgressText(self.tr("Dissolving Polygons")) - mergedLyr = algRunner.runDissolve( + mergedLyr = self.algRunner.runDissolve( inputLyr=cacheLyr, context=context, feedback=multiStepFeedback, @@ -217,7 +263,7 @@ def processAlgorithm(self, parameters, context, feedback): ) currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) - mergedLyr = algRunner.runMultipartToSingleParts( + mergedLyr = self.algRunner.runMultipartToSingleParts( inputLayer=mergedLyr, context=context, feedback=multiStepFeedback ) currentStep += 1 @@ -240,114 +286,172 @@ def processAlgorithm(self, parameters, context, feedback): return {self.OUTPUT: output_sink_id} - def buildAuxStructures(self, nx, inputLyr, tol, feedback): + def buildAuxStructures(self, nx, inputLyr, context, filterExpression=None, feedback=None): G = nx.Graph() featDict = dict() idSet = set() - nFeats = inputLyr.featureCount() - if nFeats == 0: - return G, featDict, idSet - stepSize = 100 / nFeats - - multiStepFeedback = QgsProcessingMultiStepFeedback(2, feedback) + multiStepFeedback = QgsProcessingMultiStepFeedback(3, feedback) multiStepFeedback.setCurrentStep(0) - multiStepFeedback.setProgressText(self.tr("Submiting processes to thread")) - pool = concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count() - 1) - futures = set() + candidatesLayer = self.algRunner.runFilterExpression( + inputLyr=inputLyr, expression=filterExpression, context=context, feedback=multiStepFeedback + ) + nFeats = candidatesLayer.featureCount() + multiStepFeedback.setCurrentStep(1) - def compute(feat): - itemList = [] - geom = feat.geometry() - featId = feat.id() - bbox = geom.boundingBox() - engine = QgsGeometry.createGeometryEngine(geom.constGet()) - engine.prepareGeometry() - for candidateFeat in inputLyr.getFeatures(bbox): - candidateFeatId = candidateFeat.id() + def compute(feat, featLayer): + context = QgsProcessingContext() + algRunner = AlgRunner() + itemSet = set() + featId = feat['featid'] + if multiStepFeedback.isCanceled(): + return itemSet + extractedFeaturesLayer = algRunner.runExtractByLocation( + inputLyr=inputLyr, + intersectLyr=featLayer, + context=context, + is_child_algorithm=True + ) + if multiStepFeedback.isCanceled(): + return itemSet + extractedBoundariesLayer = algRunner.runPolygonsToLines( + inputLyr=extractedFeaturesLayer, context=context + ) + if multiStepFeedback.isCanceled(): + return itemSet + algRunner.runCreateSpatialIndex( + inputLyr=extractedBoundariesLayer, + context=context + ) + if multiStepFeedback.isCanceled(): + return itemSet + splitBounds = algRunner.runClip(extractedBoundariesLayer, featLayer, context=context) + if multiStepFeedback.isCanceled(): + return itemSet + for candidateFeat in splitBounds.getFeatures(): + if multiStepFeedback.isCanceled(): + return set((None, None, None, -1)) + candidateFeatId = candidateFeat['featid'] if candidateFeatId == featId: continue - candidateGeom = candidateFeat.geometry() - candidateGeomConstGet = candidateGeom.constGet() - if not engine.intersects(candidateGeomConstGet): + candidateLength = candidateFeat.geometry().length() + if candidateLength <= 0: continue - intersectionGeom = engine.intersection(candidateGeomConstGet) - if intersectionGeom.length() <= 0: - continue - itemList.append( - (featId, candidateFeatId, candidateFeat, intersectionGeom.length()) + itemSet.add( + (featId, candidateFeatId, candidateFeat, candidateLength) ) - return itemList + return itemSet + + def build_graph(item): + featId, candidateFeatId, candidateFeat, intersectionLength = item + if featId is None: + return + if candidateFeatId not in featDict: + featDict[candidateFeatId] = candidateFeat + G.add_edge(featId, candidateFeatId) + G[featId][candidateFeatId]["length"] = intersectionLength - for current, feat in enumerate(inputLyr.getFeatures()): + if nFeats == 0: + return G, featDict, idSet + multiStepFeedback.pushInfo(f"Submitting {nFeats} features to thread.") + stepSize = 100 / nFeats + logInterval = 1000 + futures = set() + executor = concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count()-1) + for current, feat in enumerate(candidatesLayer.getFeatures()): if multiStepFeedback.isCanceled(): break - if feat.geometry().area() > tol: - continue - featId = feat.id() + featId = feat['featid'] featDict[featId] = feat idSet.add(featId) - futures.add(pool.submit(compute, feat)) + featLayer = self.layerHandler.createMemoryLayerWithFeature(inputLyr, feat, context) + # result = compute(feat, featLayer) + # list(map(build_graph, result)) + futures.add(executor.submit(compute, feat, featLayer)) multiStepFeedback.setProgress(current * stepSize) - multiStepFeedback.setCurrentStep(1) - multiStepFeedback.setProgressText(self.tr("Building graph with thread results")) - nFeats = len(futures) - stepSize = 100 / nFeats - for current, future in enumerate(concurrent.futures.as_completed(futures)): + nFeats = len(featDict) + stepSize = 100/nFeats + if nFeats == 0: + return G, featDict, idSet + + multiStepFeedback.setProgressText(self.tr(f"Starting the processess of building graph using parallel computing. Evaluating {nFeats:n} features.")) + candidateCount = 0 + for candidateCount, future in enumerate(concurrent.futures.as_completed(futures)): if multiStepFeedback.isCanceled(): break - for item in future.result(): - featId, candidateFeatId, candidateFeat, intersectionLength = item - if candidateFeatId not in featDict: - featDict[candidateFeatId] = candidateFeat - G.add_edge(featId, candidateFeatId) - G[featId][candidateFeatId]["length"] = intersectionLength - multiStepFeedback.setProgress(current * stepSize) + result = future.result() + list(map(build_graph, result)) + if candidateCount % logInterval == 0: + multiStepFeedback.setProgressText(self.tr(f"Evaluated {candidateCount:n} / {nFeats:n} features.")) + multiStepFeedback.setProgress(candidateCount * stepSize) + multiStepFeedback.pushInfo(self.tr(f"{nFeats:n} evaluated. Found {candidateCount:n} candidates to evaluate in next step.")) return G, featDict, idSet def reclassifyPolygons( - self, G, featDict, anchorIdsSet, idSet, fieldNames, feedback + self, G, featDict, anchorIdsSet, candidateIdSet, fieldNames, classFieldName, feedback, classOrderList=None ): visitedSet = set() - featuresToUpdateSet = set() - nIds = len(idSet) + featureIdsToUpdateSet = set() + nIds = len(candidateIdSet) if nIds == 0: - return featuresToUpdateSet + return featureIdsToUpdateSet stepSize = 100 / nIds processedFeats = 0 - def updateAttributes(feat, anchorFeat): - for fieldName in fieldNames: - feat[fieldName] = anchorFeat[fieldName] - return feat - + def chooseId(G, id, candidateIdSet): + if classOrderList is None: + return max(candidateIdSet, key=lambda x: G[id][x]["length"]) + auxDict = defaultdict(list) + sortedIdsByLength = sorted(candidateIdSet, key=lambda x:G[id][x]["length"], reverse=True) + if len(sortedIdsByLength) == 1: + return sortedIdsByLength[0] + for i in sortedIdsByLength: + auxDict[featDict[i][classFieldName]].append(i) + for key in classOrderList: + if key not in auxDict or len(auxDict[key]) == 0: + continue + return auxDict[key][0] + for id in set(node for node in G.nodes if G.degree(node) == 1) - anchorIdsSet: + if feedback.isCanceled(): + return featureIdsToUpdateSet + anchorId = set(G.neighbors(id)).pop() + featDict[id][classFieldName] = featDict[anchorId][classFieldName] + featureIdsToUpdateSet.add((id, featDict[id][classFieldName])) + visitedSet.add(id) + processedFeats += 1 + feedback.setProgress(processedFeats * stepSize) + visitedHolesSet = set(visitedSet) + originalAnchorIdSet = anchorIdsSet while True: newAnchors = set() for anchorId in anchorIdsSet: + if feedback.isCanceled(): + return featureIdsToUpdateSet + anchorNeighborIdsSet = set(G.neighbors(anchorId)) - originalAnchorIdSet for id in sorted( - set(G.neighbors(anchorId)) - visitedSet, + anchorNeighborIdsSet - visitedSet - anchorIdsSet - originalAnchorIdSet, key=lambda x: featDict[x].geometry().area(), ): if feedback.isCanceled(): - return featuresToUpdateSet + return featureIdsToUpdateSet # d = set(G.neighbors(id)) - anchorIdsSet - visitedSet - d = set(G.neighbors(id)).intersection(anchorIdsSet) - if d == set(): + candidateIdSet = set(G.neighbors(id)).intersection(anchorIdsSet) + if candidateIdSet == set(): continue - newAnchors.add(id) + if len(candidateIdSet) > 1: + newAnchors.add(id) visitedSet.add(id) processedFeats += 1 - chosenId = max(d, key=lambda x: G[id][x]["length"]) - feat = updateAttributes(featDict[id], featDict[chosenId]) - featDict[id] = feat - featuresToUpdateSet.add(feat) + chosenId = chooseId(G, id, candidateIdSet) + featDict[id][classFieldName] = featDict[chosenId][classFieldName] + featureIdsToUpdateSet.add((id, featDict[chosenId][classFieldName])) newAnchors.add(id) feedback.setProgress(processedFeats * stepSize) anchorIdsSet = newAnchors if anchorIdsSet == set(): break - return featuresToUpdateSet + return featureIdsToUpdateSet def getAttributesFromFeature(self, newFeat, originalFeat): pass diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/selectByDE9IM.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/selectByDE9IM.py new file mode 100644 index 000000000..055100b1b --- /dev/null +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/selectByDE9IM.py @@ -0,0 +1,228 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-04-28 + git sha : $Format:%H$ + copyright : (C) 2023 by Philipe Borba - Cartographic Engineer @ Brazilian Army + email : borba.philipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + + +from PyQt5.QtCore import QCoreApplication, QRegExp +from qgis.core import (QgsGeometry, QgsProcessing, + QgsProcessingAlgorithm, QgsProcessingMultiStepFeedback, + QgsProcessingParameterEnum, + QgsProcessingParameterFeatureSource, + QgsProcessingParameterString, Qgis) +from qgis.PyQt.QtGui import QRegExpValidator + +from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner +from DsgTools.core.GeometricTools.layerHandler import LayerHandler + + +class ValidationString(QgsProcessingParameterString): + """ + Auxiliary class for pre validation on measurer's names. + """ + + # __init__ not necessary + + def __init__(self, name, description=""): + super().__init__(name, description) + + def checkValueIsAcceptable(self, value, context=None): + regex = QRegExp("[FfTt012\*]{9}") + acceptable = QRegExpValidator.Acceptable + return ( + isinstance(value, str) + and QRegExpValidator(regex).validate(value, 9)[0] == acceptable + ) + + +class SelectByDE9IMAlgorithm(QgsProcessingAlgorithm): + INPUT = "INPUT" + INTERSECT = "INTERSECT" + DE9IM = "DE9IM" + METHOD = "METHOD" + + def initAlgorithm(self, config): + """ + Parameter setting. + """ + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INPUT, + self.tr("Select features from"), + [QgsProcessing.TypeVectorAnyGeometry], + ) + ) + + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INTERSECT, + self.tr("By comparing features from"), + [QgsProcessing.TypeVectorAnyGeometry], + ) + ) + + param = ValidationString(self.DE9IM, description=self.tr("DE9IM")) + self.addParameter(param) + self.method = [ + self.tr("creating new selection"), + self.tr("adding to current selection"), + self.tr("selecting within current selection"), + self.tr("removing from current selection"), + ] + + self.addParameter( + QgsProcessingParameterEnum( + self.METHOD, self.tr("Modify current selection by"), options=self.method, defaultValue=0 + ) + ) + self.selectionIdDict = { + 0: Qgis.SelectBehavior.SetSelection, + 1: Qgis.SelectBehavior.AddToSelection, + 2: Qgis.SelectBehavior.IntersectSelection, + 3: Qgis.SelectBehavior.RemoveFromSelection, + } + + def processAlgorithm(self, parameters, context, feedback): + """ + Here is where the processing itself takes place. + """ + self.layerHandler = LayerHandler() + self.algRunner = AlgRunner() + source = self.parameterAsSource(parameters, self.INPUT, context) + layer = self.parameterAsVectorLayer(parameters, self.INPUT, context) + intersectSource = self.parameterAsSource(parameters, self.INTERSECT, context) + method = self.parameterAsEnum(parameters, self.METHOD, context) + de9im = self.parameterAsString(parameters, self.DE9IM, context) + nFeats = intersectSource.featureCount() + if nFeats == 0: + return {} + if de9im == "FF1FF0102": + self.algRunner.runSelectByLocation( + inputLyr=parameters[self.INPUT], + intersectLyr=parameters[self.INTERSECT], + context=context, + feedback=feedback, + predicate=[2], + method=self.selectionIdDict[method], + is_child_algorithm=True, + ) + return + nSteps = 4 + multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + lyrWithId = self.algRunner.runCreateFieldWithExpression( + inputLyr=parameters[self.INPUT], + expression="$id", + fieldType=1, + fieldName="featid", + feedback=multiStepFeedback, + context=context, + is_child_algorithm=False, + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + self.algRunner.runCreateSpatialIndex( + lyrWithId, + context=context, + feedback=multiStepFeedback, + is_child_algorithm=True, + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + selectedLyr = self.algRunner.runExtractByLocation( + inputLyr=lyrWithId, + intersectLyr=parameters[self.INTERSECT], + context=context, + feedback=multiStepFeedback, + predicate=[2] if de9im == "FF1FF0102" else [0], + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + nFeats = selectedLyr.featureCount() + if nFeats == 0: + return {} + selectedSet = set() + stepSize = 100 / nFeats + + def compute(feat): + returnSet = set() + geom = feat.geometry() + bbox = geom.boundingBox() + engine = QgsGeometry.createGeometryEngine(geom.constGet()) + engine.prepareGeometry() + for f in selectedLyr.getFeatures(bbox): + if multiStepFeedback.isCanceled(): + return {} + intersectGeom = f.geometry() + if intersectGeom.isEmpty() or intersectGeom.isNull(): + continue + if engine.relatePattern(intersectGeom.constGet(), de9im): + returnSet.add(f["featid"]) + return returnSet + + for current, feat in enumerate(intersectSource.getFeatures()): + if multiStepFeedback.isCanceled(): + return {} + selectedSet.update(compute(feat)) + multiStepFeedback.setProgress(current * stepSize) + layer.selectByIds(list(selectedSet), self.selectionIdDict[method]) + + return {} + + def name(self): + """ + Returns the algorithm name, used for identifying the algorithm. This + string should be fixed for the algorithm, and must not be localised. + The name should be unique within each provider. Names should contain + lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "selectbyde9im" + + def displayName(self): + """ + Returns the translated algorithm name, which should be used for any + user-visible display of the algorithm name. + """ + return self.tr("Select features by DE9IM") + + def group(self): + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Geometric Algorithms") + + def groupId(self): + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Geometric Algorithms" + + def tr(self, string): + return QCoreApplication.translate("SelectByDE9IMAlgorithm", string) + + def createInstance(self): + return SelectByDE9IMAlgorithm() diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/splitPolygonsAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/splitPolygonsAlgorithm.py new file mode 100644 index 000000000..e9d36e76f --- /dev/null +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/splitPolygonsAlgorithm.py @@ -0,0 +1,271 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-04-15 + git sha : $Format:%H$ + copyright : (C) 2023 by Felipe Diniz - Cartographic Engineer @ Brazilian Army + email : diniz.felipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +import math +from itertools import product + +from PyQt5.QtCore import QCoreApplication, QVariant +from qgis.core import ( + QgsFeature, + QgsFeatureRequest, + QgsFeatureSink, + QgsField, + QgsGeometry, + QgsProcessing, + QgsProcessingAlgorithm, + QgsProcessingParameterEnum, + QgsProcessingParameterFeatureSink, + QgsProcessingParameterFeatureSource, + QgsProcessingParameterNumber, + QgsRectangle, + QgsSpatialIndex, + QgsVectorLayer +) + + +class SplitPolygons(QgsProcessingAlgorithm): + INPUT = "INPUT" + PARAM = "PARAM" + OVERLAP = "OVERLAP" + OUTPUT = "OUTPUT" + SPLIT_FACTORS = ["1/1", "1/4", "1/9", "1/16", "1/25"] + + def initAlgorithm(self, config=None): + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INPUT, "Input polygon layer", [QgsProcessing.TypeVectorPolygon] + ) + ) + + self.addParameter( + QgsProcessingParameterEnum( + self.PARAM, + "Splitting factor", + options=self.SPLIT_FACTORS, + defaultValue=0, + ) + ) + + self.addParameter( + QgsProcessingParameterNumber( + self.OVERLAP, + "Overlap value", + QgsProcessingParameterNumber.Double, + defaultValue=0.002, + ) + ) + + self.addParameter( + QgsProcessingParameterFeatureSink(self.OUTPUT, "Output split polygons") + ) + + def splitPol(self, features, overlap, fields, side_length, col_steps, row_steps, idToFeature, index, feedback): + polygons = [] + for feature in features: + if feedback.isCanceled(): + break + + geometry = feature.geometry() + source_fid = feature.id() + + xmin, ymin, xmax, ymax = geometry.boundingBox().toRectF().getCoords() + width = xmax - xmin + height = ymax - ymin + + for i, j in product(range(col_steps), range(row_steps)): + if feedback.isCanceled(): + break + x1 = xmin + (width * i * side_length) + y1 = ymin + (height * j * side_length) + x2 = xmin + (width * (i + 1) * side_length) + y2 = ymin + (height * (j + 1) * side_length) + + new_geom = QgsGeometry.fromRect(QgsRectangle(x1, y1, x2, y2)) + + # Buffer the new geometry + buffered_geom = new_geom.buffer( + overlap, 5 + ) # 5 is the default number of segments per quarter circle + + # Use the spatial index to find intersecting features + candidate_ids = index.intersects(buffered_geom.boundingBox()) + if not candidate_ids: + continue + + # Dissolve only intersecting features + dissolved_geometry = QgsGeometry.unaryUnion( + [idToFeature[cid].geometry() for cid in candidate_ids] + ) + + # Clip the buffered geometry by the dissolved geometry + intersected_geom = buffered_geom.intersection(dissolved_geometry) + + if intersected_geom.isEmpty(): + continue + + new_feature = QgsFeature(feature) + new_feature.setGeometry(intersected_geom) + new_feature.setFields(fields) + polygons.append(new_feature) + return polygons + + def processAlgorithm(self, parameters, context, feedback): + source = self.parameterAsSource(parameters, self.INPUT, context) + split_factor = self.parameterAsEnum(parameters, self.PARAM, context) + overlap = self.parameterAsDouble(parameters, self.OVERLAP, context) + + # Add a new field called "priority" to the output layer's fields + fields = source.fields() + fields.append(QgsField("priority", QVariant.Int)) + + (sink, dest_id) = self.parameterAsSink( + parameters, + self.OUTPUT, + context, + fields, + source.wkbType(), + source.sourceCrs(), + ) + + features = list(source.getFeatures()) + idToFeature = {} + for feat in features: + idToFeature[feat.id()] = feat + + # Create a spatial index + index = QgsSpatialIndex(source) + + parts = [1, 4, 9, 16, 25][split_factor] + side_length = math.sqrt(1 / parts) + col_steps = int(math.sqrt(parts)) + row_steps = col_steps + + polygons = self.splitPol(features, overlap, fields, side_length, col_steps, row_steps, idToFeature, index, feedback) + base_polygons = self.splitPol(features, overlap, fields, 1, 1, 1, idToFeature, index, feedback) + + tile_layer = QgsVectorLayer("Polygon?crs=" + source.sourceCrs().authid(), "tile_polygons", "memory") + tile_data_provider = tile_layer.dataProvider() + tile_data_provider.addAttributes(fields) + tile_layer.updateFields() + tile_data_provider.addFeatures(polygons) + tile_layer.updateExtents() + index_tile = QgsSpatialIndex(tile_layer) + idToFeatureTile = {} + for feat in tile_layer.getFeatures(): + idToFeatureTile[feat.id()] = feat + + base_layer = QgsVectorLayer("Polygon?crs=" + source.sourceCrs().authid(), "base_polygons", "memory") + base_data_provider = base_layer.dataProvider() + base_data_provider.addAttributes(fields) + base_layer.updateFields() + base_data_provider.addFeatures(base_polygons) + base_layer.updateExtents() + index_base = QgsSpatialIndex(base_layer) + idToFeatureBase = {} + for feat in base_layer.getFeatures(): + idToFeatureBase[feat.id()] = feat + + + priority = 1 + final_features = {} + base_used = [] + + sorted_source_features = sorted( + source.getFeatures(), + key=lambda x: ( + round(x.geometry().centroid().asPoint().y(),5), + -round(x.geometry().centroid().asPoint().x(),5), + ), + reverse=True, + ) + for feat in sorted_source_features: + # Use the spatial index to find intersecting features + geometry = feat.geometry() + candidate_ids = index_base.intersects(geometry.boundingBox()) + intersecting_features_base = [idToFeatureBase[cid] for cid in candidate_ids] + intersecting_features_base.sort( + key=lambda x: ( + round(x.geometry().centroid().asPoint().y(),5), + -round(x.geometry().centroid().asPoint().x(),5), + ) + if not x.geometry().isNull() + else (0, 0), + reverse=True, + ) + for base_polygon in intersecting_features_base: + if base_polygon.id() in base_used: + continue + base_used.append(base_polygon.id()) + + base_geometry = base_polygon.geometry() + candidate_ids = index_tile.intersects(base_geometry.boundingBox()) + intersecting_features = [idToFeatureTile[cid] for cid in candidate_ids] + + intersecting_features.sort( + key=lambda x: ( + round(x.geometry().centroid().asPoint().y(),5), + -round(x.geometry().centroid().asPoint().x(),5), + ) + if not x.geometry().isNull() + else (0, 0), + reverse=True, + ) + for polygon in intersecting_features: + if polygon.id() in final_features: + continue + polygon.setAttribute("priority", priority) + final_features[polygon.id()] = polygon + priority += 1 + + for polygon in final_features.values(): + sink.addFeature(polygon, QgsFeatureSink.FastInsert) + + return {self.OUTPUT: dest_id} + + def name(self): + return "splitpolygons" + + def displayName(self): + return "Split Polygons" + + def group(self): + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Geometric Algorithms") + + def groupId(self): + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Geometric Algorithms" + + def tr(self, string): + return QCoreApplication.translate("SplitPolygons", string) + + def createInstance(self): + return SplitPolygons() diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/splitPolygonsByGrid.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/splitPolygonsByGrid.py new file mode 100644 index 000000000..948862c34 --- /dev/null +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/GeometricAlgs/splitPolygonsByGrid.py @@ -0,0 +1,439 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-04-15 + git sha : $Format:%H$ + copyright : (C) 2023 by Felipe Diniz - Cartographic Engineer @ Brazilian Army + email : diniz.felipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +import concurrent.futures +import os +from DsgTools.core.Utils.threadingTools import concurrently + +from qgis.core import ( + QgsProcessing, + QgsProcessingAlgorithm, + QgsProcessingContext, + QgsProcessingMultiStepFeedback, + QgsProcessingParameterDistance, + QgsProcessingParameterFeatureSink, + QgsProcessingParameterFeatureSource, + QgsProcessingParameterField, + QgsSpatialIndex, + QgsProcessingParameterNumber, + QgsFields, + QgsVectorLayer, + QgsFeatureRequest, +) +from qgis.PyQt.QtCore import QCoreApplication + +from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner +from DsgTools.core.GeometricTools.layerHandler import LayerHandler + + +class SplitPolygonsByGrid(QgsProcessingAlgorithm): + INPUT = "INPUT" + X_DISTANCE = "X_DISTANCE" + Y_DISTANCE = "Y_DISTANCE" + MIN_AREA = "MIN_AREA" + NEIGHBOUR = "NEIGHBOUR" + CLASS_FIELD = "CLASS_FIELD" + MAX_CONCURRENCY = "MAX_CONCURRENCY" + OUTPUT = "OUTPUT" + + def tr(self, string): + return QCoreApplication.translate("SplitPolygonsByGrid", string) + + def createInstance(self): + return SplitPolygonsByGrid() + + def name(self): + return "polygon_split_by_grid" + + def displayName(self): + return self.tr("Polygon Split by Grid") + + def group(self): + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Geometric Algorithms") + + def groupId(self): + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Geometric Algorithms" + + def shortHelpString(self): + return self.tr( + "Splits input polygon layer by regular grid with user-defined X and Y distances and dissolves the intersection polygons based on the nearest neighbor's ID attribute." + ) + + def initAlgorithm(self, config=None): + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INPUT, + self.tr("Input Polygon Layer"), + [QgsProcessing.TypeVectorPolygon], + ) + ) + self.addParameter( + QgsProcessingParameterDistance( + self.X_DISTANCE, + self.tr("X Distance"), + parentParameterName=self.INPUT, + minValue=0.0, + defaultValue=0.0001, + ) + ) + self.addParameter( + QgsProcessingParameterDistance( + self.Y_DISTANCE, + self.tr("Y Distance"), + parentParameterName=self.INPUT, + minValue=0.0, + defaultValue=0.0001, + ) + ) + self.addParameter( + QgsProcessingParameterDistance( + self.Y_DISTANCE, + self.tr("Y Distance"), + parentParameterName=self.INPUT, + minValue=0.0, + defaultValue=0.0001, + ) + ) + param = QgsProcessingParameterDistance( + self.MIN_AREA, + self.tr( + "Minimun area to process. If feature's area is smaller than this value, " + "the feature will not be split, but only reclassified to the nearest neighbour." + ), + parentParameterName=self.INPUT, + defaultValue=1e-8, + ) + param.setMetadata({"widget_wrapper": {"decimals": 10}}) + self.addParameter(param) + self.addParameter( + QgsProcessingParameterFeatureSource( + self.NEIGHBOUR, + self.tr("Neighbour Polygon Layer"), + [QgsProcessing.TypeVectorPolygon], + ) + ) + self.addParameter( + QgsProcessingParameterField( + self.CLASS_FIELD, + self.tr("Class attribute field"), + parentLayerParameterName=self.NEIGHBOUR, + type=QgsProcessingParameterField.Any, + ) + ) + self.addParameter( + QgsProcessingParameterNumber( + self.MAX_CONCURRENCY, + self.tr("Max Concurrency"), + type=QgsProcessingParameterNumber.Integer, + defaultValue=1, + minValue=1, + ) + ) + self.addParameter( + QgsProcessingParameterFeatureSink(self.OUTPUT, self.tr("Output Layer")) + ) + + def processAlgorithm(self, parameters, context, feedback): + self.algRunner = AlgRunner() + self.layerHandler = LayerHandler() + source = self.parameterAsSource(parameters, self.INPUT, context) + x_distance = self.parameterAsDouble(parameters, self.X_DISTANCE, context) + y_distance = self.parameterAsDouble(parameters, self.Y_DISTANCE, context) + min_area = self.parameterAsDouble(parameters, self.MIN_AREA, context) + neighbour_source = self.parameterAsSource(parameters, self.NEIGHBOUR, context) + classFieldName = self.parameterAsString(parameters, self.CLASS_FIELD, context) + max_concurrency = self.parameterAsInt(parameters, self.MAX_CONCURRENCY, context) + + (sink, dest_id) = self.parameterAsSink( + parameters, + self.OUTPUT, + context, + source.fields(), + source.wkbType(), + source.sourceCrs(), + ) + + nFeats = source.featureCount() + if nFeats == 0: + return {self.OUTPUT: dest_id} + request = QgsFeatureRequest() + clause = QgsFeatureRequest.OrderByClause("$area") + orderby = QgsFeatureRequest.OrderBy([clause]) + request.setOrderBy(orderby) + iterator = source.getFeatures(request) + nSteps = nFeats + 2 + multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + multiStepFeedback.setCurrentStep(0) + multiStepFeedback.setProgressText(self.tr("Extracting vertexes...")) + verticesLyr = self.algRunner.runExtractVertices( + inputLyr=parameters[self.NEIGHBOUR], + context=context, + feedback=multiStepFeedback, + ) + multiStepFeedback.setCurrentStep(1) + multiStepFeedback.setProgressText(self.tr("Creating spatial index...")) + self.algRunner.runCreateSpatialIndex( + verticesLyr, context=context, feedback=multiStepFeedback + ) + multiStepFeedback.setProgressText(self.tr("Processing features...")) + + def prepare_data(feature): + geom = feature.geometry() + bbox = geom.boundingBox() + try: + featureLayer = self.layerHandler.createMemoryLayerWithFeature( + source, feature, context=context, isSource=True + ) + except: + return None, None + if bbox.isEmpty() or bbox.isNull() or not bbox.isFinite(): + return None, None + try: + localNeighborVertexes = self.algRunner.runExtractByExtent( + inputLayer=verticesLyr, extent=bbox, context=context, clip=True + ) + except: + return None, None + return featureLayer, localNeighborVertexes + + if max_concurrency == 1: + for current, feature in enumerate(iterator, start=2): + if multiStepFeedback.isCanceled(): + return {self.OUTPUT: dest_id} + multiStepFeedback.setCurrentStep(current) + geom = feature.geometry() + if geom.isNull() or geom.isEmpty(): + continue + featureLayer, localNeighborVertexes = prepare_data(feature) + if featureLayer is None: + continue + if multiStepFeedback.isCanceled(): + return {self.OUTPUT: dest_id} + outputFeatures = self.compute( + localNeighborVertexes=localNeighborVertexes, + feature=feature, + featureLayer=featureLayer, + x_distance=x_distance, + y_distance=y_distance, + classFieldName=classFieldName, + source_fields=source.fields(), + min_area=min_area, + feedback=multiStepFeedback, + ) + if outputFeatures is None or outputFeatures == set(): + if current % 500 == 0: + multiStepFeedback.pushInfo( + self.tr(f"Processed {current}/{nFeats}.") + ) + continue + sink.addFeatures(list(outputFeatures)) + if current % 500 == 0: + multiStepFeedback.pushInfo( + self.tr(f"Processed {current}/{nFeats}.") + ) + return {self.OUTPUT: dest_id} + + def compute_in_paralel(feature): + featureLayer, localNeighborVertexes = prepare_data(feature) + return self.compute( + localNeighborVertexes=localNeighborVertexes, + feature=feature, + featureLayer=featureLayer, + x_distance=x_distance, + y_distance=y_distance, + classFieldName=classFieldName, + source_fields=QgsFields(source.fields()), + min_area=min_area, + feedback=feedback, + ) + + for current, outputFeatures in enumerate( + concurrently(compute_in_paralel, iterator, max_concurrency=max_concurrency), + start=2, + ): + multiStepFeedback.setCurrentStep(current) + if multiStepFeedback.isCanceled(): + return {self.OUTPUT: dest_id} + if outputFeatures is None or outputFeatures == set(): + continue + sink.addFeatures(list(outputFeatures)) + if current % 500 == 0: + multiStepFeedback.pushInfo(self.tr(f"Processed {current}/{nFeats}.")) + + return {self.OUTPUT: dest_id} + + def compute( + self, + localNeighborVertexes, + feature, + featureLayer, + x_distance, + y_distance, + classFieldName, + source_fields, + min_area, + feedback=None, + ): + context = QgsProcessingContext() + algRunner = AlgRunner() + if feedback is not None and feedback.isCanceled(): + return set() + if ( + (feedback is not None and feedback.isCanceled()) + or feature is None + or localNeighborVertexes is None + or featureLayer is None + or isinstance(localNeighborVertexes, str) + or localNeighborVertexes.featureCount() == 0 + ): + return set() + neighbour_idx = QgsSpatialIndex(localNeighborVertexes.getFeatures()) + neighbourFeatDict = { + feat.id(): feat for feat in localNeighborVertexes.getFeatures() + } + geometry = feature.geometry() + if geometry.isEmpty() or geometry.isNull(): + return set() + bbox = geometry.boundingBox() + xmin, ymin, xmax, ymax = bbox.toRectF().getCoords() + xSpacing = ( + x_distance + if abs(xmax - xmin) > x_distance + else min(abs(xmax - xmin) / 2, abs(ymax - ymin) / 2) + ) + ySpacing = ( + y_distance + if abs(ymax - ymin) > y_distance + else min(abs(xmax - xmin) / 2, abs(ymax - ymin) / 2) + ) + if geometry.area() <= min_area or geometry.area() <= xSpacing * ySpacing: + nearest_neighbor_ids = neighbour_idx.nearestNeighbor( + geometry.centroid().asPoint(), 1 + ) + if nearest_neighbor_ids == []: + return set() + nearest_neighbor_id = nearest_neighbor_ids[0] + if nearest_neighbor_id not in neighbourFeatDict: + return set() + feature[classFieldName] = neighbourFeatDict[nearest_neighbor_id][ + classFieldName + ] + returnSet = set() + returnSet.add(feature) + return returnSet + gridLayer = algRunner.runCreateGrid( + extent=bbox, + crs=featureLayer.crs(), + hSpacing=xSpacing, + vSpacing=ySpacing, + context=context, + ) + if feedback is not None and feedback.isCanceled(): + return set() + algRunner.runCreateSpatialIndex(gridLayer, context=context) + if feedback is not None and feedback.isCanceled(): + return set() + try: + clippedPolygons = algRunner.runClip( + gridLayer, featureLayer, context=context + ) + except: + clippedPolygons = None + if ( + not isinstance(clippedPolygons, QgsVectorLayer) + or clippedPolygons.featureCount() < 4 + ): + nearest_neighbors = neighbour_idx.nearestNeighbor( + geometry.centroid().asPoint(), 1 + ) + if nearest_neighbors == []: + return set() + nearest_neighbor_id = nearest_neighbors[0] + if nearest_neighbor_id not in neighbourFeatDict: + return set() + feature[classFieldName] = neighbourFeatDict[nearest_neighbor_id][ + classFieldName + ] + returnSet = set() + returnSet.add(feature) + return returnSet + clippedPolygons.startEditing() + if feedback is not None and feedback.isCanceled(): + return set() + algRunner.runCreateSpatialIndex(clippedPolygons, context=context) + nFeats = clippedPolygons.featureCount() + if nFeats == 0: + return None + clippedPolygons.beginEditCommand("Updating features") + clippedPolygonsDataProvider = clippedPolygons.dataProvider() + if not any(i.name() == classFieldName for i in clippedPolygons.fields()): + clippedPolygonsDataProvider.addAttributes( + [i for i in source_fields if i.name() == classFieldName] + ) + clippedPolygons.updateFields() + fieldIdx = clippedPolygons.fields().indexFromName(classFieldName) + for feat in clippedPolygons.getFeatures(): + if feedback is not None and feedback.isCanceled(): + return set() + geom = feat.geometry() + if geom.isEmpty() or geom.isNull(): + continue + nearest_neighbor_ids = neighbour_idx.nearestNeighbor( + geom.centroid().asPoint(), 1 + ) + if nearest_neighbor_ids == []: + continue + nearest_neighbor_id = nearest_neighbor_ids[0] + destinationAttr = neighbourFeatDict[nearest_neighbor_id][classFieldName] + clippedPolygonsDataProvider.changeAttributeValues( + {feat.id(): {fieldIdx: destinationAttr}} + ) + clippedPolygons.endEditCommand() + if feedback is not None and feedback.isCanceled(): + return set() + snappedToGrid = algRunner.runSnapToGrid( + inputLayer=clippedPolygons, tol=1e-15, context=context + ) + if feedback is not None and feedback.isCanceled(): + return set() + dissolvedLyr = algRunner.runDissolve( + inputLyr=snappedToGrid, + field=[classFieldName], + context=context, + ) + if feedback is not None and feedback.isCanceled(): + return set() + retainedFields = algRunner.runRetainFields( + dissolvedLyr, [field.name() for field in source_fields], context=context + ) + return set(feat for feat in retainedFields.getFeatures()) diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnIntersectionsAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnIntersectionsAlgorithm.py index d0b5511a2..eda36ff65 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnIntersectionsAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnIntersectionsAlgorithm.py @@ -232,7 +232,7 @@ def groupId(self): contain lowercase alphanumeric characters only and no spaces or other formatting characters. """ - return "Quality Assurance Tools (Correction Processes)" + return "DSGTools: Quality Assurance Tools (Correction Processes)" def tr(self, string): return QCoreApplication.translate( diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnSharedEdgesAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnSharedEdgesAlgorithm.py index 0502a22ba..849eb4bd8 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnSharedEdgesAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/addUnsharedVertexOnSharedEdgesAlgorithm.py @@ -227,7 +227,7 @@ def groupId(self): contain lowercase alphanumeric characters only and no spaces or other formatting characters. """ - return "Quality Assurance Tools (Correction Processes)" + return "DSGTools: Quality Assurance Tools (Correction Processes)" def tr(self, string): return QCoreApplication.translate( diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/buildPolygonsFromCenterPointsAndBoundariesAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/buildPolygonsFromCenterPointsAndBoundariesAlgorithm.py index 2d8cd91f2..118db30e9 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/buildPolygonsFromCenterPointsAndBoundariesAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/buildPolygonsFromCenterPointsAndBoundariesAlgorithm.py @@ -20,8 +20,12 @@ ***************************************************************************/ """ +import concurrent.futures +import os +from uuid import uuid4 from DsgTools.core.GeometricTools.spatialRelationsHandler import SpatialRelationsHandler import processing +from processing.tools import dataobjects from PyQt5.QtCore import QCoreApplication from DsgTools.core.GeometricTools.layerHandler import LayerHandler @@ -44,6 +48,9 @@ QgsProcessingParameterVectorLayer, QgsWkbTypes, QgsProcessingUtils, + QgsVectorLayer, + QgsProcessingFeatureSourceDefinition, + QgsProcessingContext, ) from ...algRunner import AlgRunner @@ -59,10 +66,12 @@ class BuildPolygonsFromCenterPointsAndBoundariesAlgorithm(ValidationAlgorithm): CONSTRAINT_POLYGON_LAYERS = "CONSTRAINT_POLYGON_LAYERS" GEOGRAPHIC_BOUNDARY = "GEOGRAPHIC_BOUNDARY" SUPPRESS_AREA_WITHOUT_CENTROID_FLAG = "SUPPRESS_AREA_WITHOUT_CENTROID_FLAG" + CHECK_UNUSED_BOUDARY_LINES = "CHECK_UNUSED_BOUNDARY_LINES" CHECK_INVALID_GEOMETRIES_ON_OUTPUT_POLYGONS = ( "CHECK_INVALID_GEOMETRIES_ON_OUTPUT_POLYGONS" ) MERGE_OUTPUT_POLYGONS = "MERGE_OUTPUT_POLYGONS" + GROUP_BY_SPATIAL_PARTITION = "GROUP_BY_SPATIAL_PARTITION" OUTPUT_POLYGONS = "OUTPUT_POLYGONS" INVALID_POLYGON_LOCATION = "INVALID_POLYGON_LOCATION" UNUSED_BOUNDARY_LINES = "UNUSED_BOUNDARY_LINES" @@ -141,6 +150,13 @@ def initAlgorithm(self, config): defaultValue=True, ) ) + self.addParameter( + QgsProcessingParameterBoolean( + self.CHECK_UNUSED_BOUDARY_LINES, + self.tr("Check unused boundary lines"), + defaultValue=True, + ) + ) self.addParameter( QgsProcessingParameterBoolean( self.SUPPRESS_AREA_WITHOUT_CENTROID_FLAG, @@ -148,6 +164,12 @@ def initAlgorithm(self, config): defaultValue=False, ) ) + self.addParameter( + QgsProcessingParameterBoolean( + self.GROUP_BY_SPATIAL_PARTITION, + self.tr("Run algothimn grouping by spatial partition"), + ) + ) self.addParameter( QgsProcessingParameterFeatureSink( self.OUTPUT_POLYGONS, self.tr("Output Polygons") @@ -177,8 +199,8 @@ def processAlgorithm(self, parameters, context, feedback): """ Here is where the processing itself takes place. """ - layerHandler = LayerHandler() - algRunner = AlgRunner() + self.layerHandler = LayerHandler() + self.algRunner = AlgRunner() inputCenterPointLyr = self.parameterAsVectorLayer( parameters, self.INPUT_CENTER_POINTS, context ) @@ -208,9 +230,12 @@ def processAlgorithm(self, parameters, context, feedback): attributeBlackList = self.parameterAsFields( parameters, self.ATTRIBUTE_BLACK_LIST, context ) - fields = layerHandler.getFieldsFromAttributeBlackList( + fields = self.layerHandler.getFieldsFromAttributeBlackList( inputCenterPointLyr, attributeBlackList ) + groupBySpatialPartition = self.parameterAsBool( + parameters, self.GROUP_BY_SPATIAL_PARTITION, context + ) (output_polygon_sink, output_polygon_sink_id) = self.parameterAsSink( parameters, self.OUTPUT_POLYGONS, @@ -225,6 +250,9 @@ def processAlgorithm(self, parameters, context, feedback): checkInvalidOnOutput = self.parameterAsBool( parameters, self.CHECK_INVALID_GEOMETRIES_ON_OUTPUT_POLYGONS, context ) + checkUnusedBoundaries = self.parameterAsBool( + parameters, self.CHECK_UNUSED_BOUDARY_LINES, context + ) mergeOutput = self.parameterAsBool( parameters, self.MERGE_OUTPUT_POLYGONS, context ) @@ -246,24 +274,37 @@ def processAlgorithm(self, parameters, context, feedback): else inputCenterPointLyr.sourceCrs(), ) nSteps = ( - 3 + (mergeOutput + 1) + checkInvalidOnOutput + 3 + (mergeOutput + 1) + checkInvalidOnOutput + checkUnusedBoundaries ) # boolean sum, if true, sums 1 to each term currentStep = 0 multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) multiStepFeedback.setCurrentStep(currentStep) - polygonFeatList, flagDict = self.computePolygonsFromCenterPointAndBoundaries( - context, - layerHandler, - algRunner, - inputCenterPointLyr, - boundaryLineLyr, - constraintLineLyrList, - constraintPolygonLyrList, - onlySelected, - geographicBoundaryLyr, - attributeBlackList, - suppressPolygonWithoutCenterPointFlag, - multiStepFeedback, + polygonFeatList, flagDict = ( + self.computePolygonsFromCenterPointAndBoundaries( + context, + inputCenterPointLyr, + boundaryLineLyr, + constraintLineLyrList, + constraintPolygonLyrList, + onlySelected, + geographicBoundaryLyr, + attributeBlackList, + suppressPolygonWithoutCenterPointFlag, + multiStepFeedback, + ) + if not groupBySpatialPartition or geographicBoundaryLyr.featureCount() <= 1 + else self.computePolygonsFromCenterPointAndBoundariesGroupingBySpatialPartition( + context, + inputCenterPointLyr, + boundaryLineLyr, + constraintLineLyrList, + constraintPolygonLyrList, + onlySelected, + geographicBoundaryLyr, + attributeBlackList, + suppressPolygonWithoutCenterPointFlag, + multiStepFeedback, + ) ) currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) @@ -280,21 +321,22 @@ def processAlgorithm(self, parameters, context, feedback): ) sink.addFeatures(polygonFeatList, QgsFeatureSink.FastInsert) - multiStepFeedback.setCurrentStep(currentStep) - self.checkUnusedBoundariesAndWriteOutput( - context, - boundaryLineLyr, - geographicBoundaryLyr, - sink_id, - unused_boundary_flag_sink, - multiStepFeedback, - ) - currentStep += 1 + if checkUnusedBoundaries: + multiStepFeedback.setCurrentStep(currentStep) + self.checkUnusedBoundariesAndWriteOutput( + context, + boundaryLineLyr, + geographicBoundaryLyr, + sink_id, + unused_boundary_flag_sink, + multiStepFeedback, + ) + currentStep += 1 if mergeOutput: multiStepFeedback.setCurrentStep(currentStep) multiStepFeedback.setProgressText(self.tr("Dissolving output...")) - dissolvedLyr = algRunner.runDissolve( + dissolvedLyr = self.algRunner.runDissolve( sink_id, context, feedback=multiStepFeedback, @@ -302,7 +344,7 @@ def processAlgorithm(self, parameters, context, feedback): ) currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) - dissolvedLyr = algRunner.runMultipartToSingleParts( + dissolvedLyr = self.algRunner.runMultipartToSingleParts( dissolvedLyr, context=context, feedback=multiStepFeedback ) polygonFeatList = [feat for feat in dissolvedLyr.getFeatures()] @@ -313,8 +355,8 @@ def processAlgorithm(self, parameters, context, feedback): currentStep += 1 if checkInvalidOnOutput: multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Checking invalid geometries...")) self.checkInvalidOnOutput( - layerHandler, inputCenterPointLyr, multiStepFeedback, polygonFeatList, @@ -345,7 +387,8 @@ def checkUnusedBoundariesAndWriteOutput( multiStepFeedback.setProgressText(self.tr("Checking unused boundaries...")) currentStep = 0 multiStepFeedback.setCurrentStep(currentStep) - lyr = processing.run( + multiStepFeedback.setProgressText(self.tr("Building cache...")) + builtPolygonsLyr = processing.run( "native:addautoincrementalfield", parameters={ "INPUT": output_polygon_sink_id, @@ -359,102 +402,111 @@ def checkUnusedBoundariesAndWriteOutput( }, context=context, feedback=multiStepFeedback, + is_child_algorithm=True, )["OUTPUT"] currentStep += 1 + multiStepFeedback.setProgressText(self.tr("Converting built polygons to lines...")) multiStepFeedback.setCurrentStep(currentStep) - processing.run( - "native:createspatialindex", {"INPUT": lyr}, feedback=multiStepFeedback + polygonLines = self.algRunner.runPolygonsToLines( + inputLyr=builtPolygonsLyr, + context=context, + feedback=multiStepFeedback, + is_child_algorithm=True, ) currentStep += 1 + multiStepFeedback.setProgressText(self.tr("Exploding lines...")) multiStepFeedback.setCurrentStep(currentStep) - segments = AlgRunner().runExplodeLines( - boundaryLineLyr, context, feedback=multiStepFeedback + explodedPolygonLines = self.algRunner.runExplodeLines( + inputLyr=polygonLines, + context=context, + feedback=multiStepFeedback, + is_child_algorithm=True, ) currentStep += 1 + multiStepFeedback.setProgressText(self.tr("Building spatial index...")) multiStepFeedback.setCurrentStep(currentStep) - segments = processing.run( - "native:addautoincrementalfield", - parameters={ - "INPUT": segments, - "FIELD_NAME": "featid", - "START": 1, - "GROUP_FIELDS": [], - "SORT_EXPRESSION": "", - "SORT_ASCENDING": True, - "SORT_NULLS_FIRST": False, - "OUTPUT": "TEMPORARY_OUTPUT", - }, + self.algRunner.runCreateSpatialIndex( + inputLyr=explodedPolygonLines, context=context, feedback=multiStepFeedback, - )["OUTPUT"] - currentStep += 1 + is_child_algorithm=True, + ) + multiStepFeedback.setProgressText(self.tr("Exploding boudary lines...")) multiStepFeedback.setCurrentStep(currentStep) - processing.run( - "native:createspatialindex", {"INPUT": segments}, feedback=multiStepFeedback + segments = self.algRunner.runExplodeLines( + boundaryLineLyr, context, feedback=multiStepFeedback, is_child_algorithm=True, + ) + currentStep += 1 + + self.algRunner.runCreateSpatialIndex( + inputLyr=segments, + context=context, + feedback=multiStepFeedback, + is_child_algorithm=True, ) currentStep += 1 if geographicBoundaryLyr is not None: multiStepFeedback.setCurrentStep(currentStep) - segments = AlgRunner().runClip( + segments = self.algRunner.runClip( segments, geographicBoundaryLyr, context=context, feedback=multiStepFeedback, + is_child_algorithm=True, ) currentStep += 1 - - multiStepFeedback.setCurrentStep(currentStep) - processing.run( - "native:createspatialindex", - {"INPUT": segments}, + self.algRunner.runCreateSpatialIndex( + inputLyr=segments, + context=context, feedback=multiStepFeedback, + is_child_algorithm=True, ) currentStep += 1 + + multiStepFeedback.setProgressText(self.tr("Running spatial join...")) multiStepFeedback.setCurrentStep(currentStep) - flags = SpatialRelationsHandler().checkDE9IM( - layerA=segments, - layerB=lyr, - mask="*1*******", - cardinality="1..*", + unmatchedLines = processing.run( + "native:joinattributesbylocation", + { + 'INPUT':segments, + 'PREDICATE':[2], + 'JOIN':polygonLines, + 'JOIN_FIELDS':[], + 'METHOD':0, + 'DISCARD_NONMATCHING':False, + 'PREFIX':'', + 'NON_MATCHING':'memory:', + }, + context=context, feedback=multiStepFeedback, - ctx=context, - ) - currentStep += 1 - - featidList = list( - set(i["featid"] for i in segments.getFeatures(list(flags.keys()))) - ) - if len(featidList) == 0: - multiStepFeedback.setCurrentStep(8 if geographicBoundaryLyr is None else 10) - return - expressionStr = f"featid in {tuple(featidList)}" - if ",)" in expressionStr: - expressionStr = expressionStr.replace(",)", ")") + is_child_algorithm=True + )['NON_MATCHING'] + multiStepFeedback.setCurrentStep(currentStep) - segmentedFlags = AlgRunner().runFilterExpression( - segments, - expression=expressionStr, + multiStepFeedback.setProgressText(self.tr("Preparing unused boundaries flags...")) + self.algRunner.runCreateSpatialIndex( + inputLyr=unmatchedLines, context=context, feedback=multiStepFeedback, + is_child_algorithm=True, ) - currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) mergedSegments = processing.run( "native:dissolve", - {"INPUT": segmentedFlags, "OUTPUT": "TEMPORARY_OUTPUT"}, + {"INPUT": unmatchedLines, "OUTPUT": "memory:"}, context=context, feedback=multiStepFeedback, )["OUTPUT"] currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) - flagLyr = AlgRunner().runMultipartToSingleParts( + flagLyr = self.algRunner.runMultipartToSingleParts( mergedSegments, context, feedback=multiStepFeedback ) unused_boundary_flag_sink.addFeatures( @@ -463,7 +515,6 @@ def checkUnusedBoundariesAndWriteOutput( def checkInvalidOnOutput( self, - layerHandler, inputCenterPointLyr, feedback, polygonFeatList, @@ -474,7 +525,7 @@ def checkInvalidOnOutput( self.tr("Checking for invalid geometries on output polygons...") ) multiStepFeedback.setCurrentStep(0) - invalidGeomFlagDict, _ = layerHandler.identifyInvalidGeometries( + invalidGeomFlagDict, _ = self.layerHandler.identifyInvalidGeometries( polygonFeatList, len(polygonFeatList), inputCenterPointLyr, @@ -517,8 +568,6 @@ def writeOutputPolygons( def computePolygonsFromCenterPointAndBoundaries( self, context, - layerHandler, - algRunner, inputCenterPointLyr, boundaryLineLyr, constraintLineLyrList, @@ -538,7 +587,7 @@ def computePolygonsFromCenterPointAndBoundaries( ( polygonFeatList, flagDict, - ) = layerHandler.getPolygonsFromCenterPointsAndBoundaries( + ) = self.layerHandler.getPolygonsFromCenterPointsAndBoundaries( inputCenterPointLyr, geographicBoundaryLyr=geographicBoundaryLyr, constraintLineLyrList=constraintLineLyrList + [boundaryLineLyr] @@ -550,43 +599,354 @@ def computePolygonsFromCenterPointAndBoundaries( context=context, feedback=multiStepFeedback, attributeBlackList=attributeBlackList, - algRunner=algRunner, + algRunner=self.algRunner, ) return polygonFeatList, flagDict - def checkUnusedBoundaries( - self, boundaryLineLyr, output_polygon_sink_id, feedback=None, context=None + def computePolygonsFromCenterPointAndBoundariesGroupingBySpatialPartition( + self, + context, + inputCenterPointLyr, + boundaryLineLyr, + constraintLineLyrList, + constraintPolygonLyrList, + onlySelected, + geographicBoundaryLyr, + attributeBlackList, + suppressPolygonWithoutCenterPointFlag, + feedback, ): - multiStepFeedback = QgsProcessingMultiStepFeedback(3, feedback) - multiStepFeedback.setCurrentStep(0) - lyr = processing.run( - "native:addautoincrementalfield", - parameters={ - "INPUT": output_polygon_sink_id, - "FIELD_NAME": "featid", - "START": 1, - "GROUP_FIELDS": [], - "SORT_EXPRESSION": "", - "SORT_ASCENDING": True, - "SORT_NULLS_FIRST": False, - "OUTPUT": "TEMPORARY_OUTPUT", - }, + polygonFeatList = [] + flagDict = dict() + nSteps = 5 + 2 + multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Splitting geographic bounds")) + geographicBoundaryLayerList = self.layerHandler.createMemoryLayerForEachFeature( + layer=geographicBoundaryLyr, context=context, feedback=multiStepFeedback + ) + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Preparing constraint lines")) + constraintLinesLyr = ( + self.algRunner.runMergeVectorLayers( + inputList=constraintLineLyrList, + context=context, + feedback=multiStepFeedback, + ) + if len(constraintLineLyrList) > 0 + else None + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + if constraintLinesLyr is not None: + self.algRunner.runCreateSpatialIndex( + constraintLinesLyr, context, multiStepFeedback + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Preparing constraint polygons")) + constraintPolygonsLyr = ( + self.algRunner.runMergeVectorLayers( + inputList=constraintPolygonLyrList, + context=context, + feedback=multiStepFeedback, + ) + if len(constraintPolygonLyrList) > 0 + else None + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + if constraintPolygonsLyr is not None: + self.algRunner.runCreateSpatialIndex( + constraintPolygonsLyr, context, multiStepFeedback + ) + currentStep += 1 + + def compute(localGeographicBoundsLyr): + context = QgsProcessingContext() + if multiStepFeedback.isCanceled(): + return [], {} + localInputCenterPointLyr = self.extractFeaturesUsingGeographicBounds( + inputLyr=inputCenterPointLyr, + geographicBounds=localGeographicBoundsLyr, + feedback=None, + context=context, + onlySelected=onlySelected, + ) + if multiStepFeedback.isCanceled(): + return [], {} + localBoundaryLineLyr = self.extractFeaturesUsingGeographicBounds( + inputLyr=boundaryLineLyr, + geographicBounds=localGeographicBoundsLyr, + feedback=None, + context=context, + onlySelected=onlySelected, + ) + if multiStepFeedback.isCanceled(): + return [], {} + localLinesConstraintLyr = ( + self.extractFeaturesUsingGeographicBounds( + inputLyr=constraintLinesLyr, + geographicBounds=localGeographicBoundsLyr, + feedback=None, + context=context, + onlySelected=onlySelected, + ) + if constraintLinesLyr is not None + else None + ) + if multiStepFeedback.isCanceled(): + return [], {} + localPolygonsConstraintLyr = ( + self.extractFeaturesUsingGeographicBounds( + inputLyr=constraintPolygonsLyr, + geographicBounds=localGeographicBoundsLyr, + feedback=None, + context=context, + onlySelected=onlySelected, + ) + if constraintPolygonsLyr is not None + else None + ) + if multiStepFeedback.isCanceled(): + return [], {} + return self.layerHandler.getPolygonsFromCenterPointsAndBoundariesAlt( + localInputCenterPointLyr, + geographicBoundaryLyr=localGeographicBoundsLyr, + constraintLineLyrList=[localLinesConstraintLyr, localBoundaryLineLyr] + if localLinesConstraintLyr is not None + else [localBoundaryLineLyr], + constraintPolygonLyrList=[localPolygonsConstraintLyr] + if localPolygonsConstraintLyr is not None + else [], + onlySelected=False, # the selected features were already filtered + suppressPolygonWithoutCenterPointFlag=suppressPolygonWithoutCenterPointFlag, + context=context, + feedback=None, + attributeBlackList=attributeBlackList, + algRunner=AlgRunner(), + ) + + multiStepFeedback.setCurrentStep(currentStep) + nRegions = len(geographicBoundaryLayerList) + if nRegions == 0: + return polygonFeatList, flagDict + stepSize = 100 / nRegions + futures = set() + pool = concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count() - 1) + multiStepFeedback.setProgressText( + self.tr("Submitting building polygon tasks to thread...") + ) + for current, localGeographicBoundsLyr in enumerate( + geographicBoundaryLayerList, start=0 + ): + if multiStepFeedback.isCanceled(): + pool.shutdown(cancel_futures=True) + break + futures.add(pool.submit(compute, localGeographicBoundsLyr)) + multiStepFeedback.setProgress(current * stepSize) + + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + + multiStepFeedback.setProgressText(self.tr("Evaluating results...")) + for current, future in enumerate(concurrent.futures.as_completed(futures)): + if multiStepFeedback.isCanceled(): + pool.shutdown(cancel_futures=True) + break + localPolygonFeatList, localFlagDict = future.result() + multiStepFeedback.pushInfo( + self.tr( + f"Building polygons from region {current+1}/{nRegions} is done." + ) + ) + multiStepFeedback.setProgress(current * stepSize) + polygonFeatList += localPolygonFeatList + flagDict.update(localFlagDict) + return polygonFeatList, flagDict + + def checkUnusedBoundariesAndWriteOutputGroupingBySpatialPartition( + self, + context, + boundaryLineLyr, + geographicBoundaryLyr, + output_polygon_sink_id, + unused_boundary_flag_sink, + feedback, + ): + if boundaryLineLyr is None: + return + nRegions = geographicBoundaryLyr.featureCount() + if nRegions == 0: + return + nSteps = 5 + multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + multiStepFeedback.setProgressText(self.tr("Checking unused boundaries...")) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Building aux structures: creating local cache...")) + polygonLyr = self.algRunner.runAddAutoIncrementalField( + inputLyr=output_polygon_sink_id, + fieldName="featid", context=context, - feedback=multiStepFeedback, - )["OUTPUT"] - processing.run( - "native:createspatialindex", {"INPUT": lyr}, feedback=multiStepFeedback + feedback=multiStepFeedback, ) - multiStepFeedback.setCurrentStep(1) - flags = SpatialRelationsHandler().checkDE9IM( - layerA=boundaryLineLyr, - layerB=lyr, - mask="*1*******", - cardinality="1..*", + currentStep += 1 + + multiStepFeedback.setCurrentStep(currentStep) + self.algRunner.runCreateSpatialIndex( + inputLyr=polygonLyr, context=context, feedback=multiStepFeedback + ) + currentStep += 1 + + + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Splitting geographic bounds")) + geographicBoundaryLayerList = self.layerHandler.createMemoryLayerForEachFeature( + layer=geographicBoundaryLyr, context=context, feedback=multiStepFeedback + ) + currentStep += 1 + + def compute(localGeographicBoundsLyr): + context = QgsProcessingContext() + algRunner = AlgRunner() + if multiStepFeedback.isCanceled(): + return + localBoundaries = algRunner.runClip( + boundaryLineLyr, + localGeographicBoundsLyr, + context=context, + feedback=None, + is_child_algorithm=True, + ) + if multiStepFeedback.isCanceled(): + return + localBoundaries = algRunner.runAddAutoIncrementalField( + inputLyr=localBoundaries, + fieldName="local_featid", + context=context, + feedback=None + ) + if multiStepFeedback.isCanceled(): + return + segments = self.algRunner.runExplodeLines( + localBoundaries, context, feedback=None, is_child_algorithm=True + ) + if multiStepFeedback.isCanceled(): + return + segments = algRunner.runAddAutoIncrementalField( + inputLyr=segments, + fieldName="seg_featid", + context=context, + feedback=None + ) + if multiStepFeedback.isCanceled(): + return + flags = SpatialRelationsHandler().checkDE9IM( + layerA=segments, + layerB=polygonLyr, + mask="*1*******", + cardinality="1..*", + feedback=None, + ctx=context, + ) + if multiStepFeedback.isCanceled(): + return + featidList = list( + set(i["seg_featid"] for i in segments.getFeatures(list(flags.keys()))) + ) + if multiStepFeedback.isCanceled(): + return + if len(featidList) == 0: + return + expressionStr = f"seg_featid in {tuple(featidList)}" + if ",)" in expressionStr: + expressionStr = expressionStr.replace(",)", ")") + segmentedFlags = algRunner.runFilterExpression( + segments, + expression=expressionStr, + context=context, + feedback=None, + ) + if multiStepFeedback.isCanceled(): + return + mergedSegments = processing.run( + "native:dissolve", + {"INPUT": segmentedFlags, "OUTPUT": "memory:"}, + context=context, + feedback=None, + )["OUTPUT"] + if multiStepFeedback.isCanceled(): + return + flagLyr = algRunner.runMultipartToSingleParts( + mergedSegments, context, feedback=None + ) + return flagLyr + + + multiStepFeedback.setCurrentStep(currentStep) + + stepSize = 100 / nRegions + futures = set() + pool = concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count() - 1) + multiStepFeedback.pushInfo( + self.tr("Submitting verifying unused boundaries tasks to thread...") + ) + for current, localGeographicBoundsLyr in enumerate( + geographicBoundaryLayerList, start=0 + ): + if multiStepFeedback.isCanceled(): + pool.shutdown(cancel_futures=True) + break + futures.add(pool.submit(compute, localGeographicBoundsLyr)) + multiStepFeedback.setProgress(current * stepSize) + + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + + multiStepFeedback.pushInfo(self.tr("Evaluating results...")) + for current, future in enumerate(concurrent.futures.as_completed(futures)): + if multiStepFeedback.isCanceled(): + break + localFlagLyr = future.result() + multiStepFeedback.pushInfo( + self.tr( + f"Verifying unused boundaries from region {current+1}/{nRegions} is done." + ) + ) + multiStepFeedback.setProgress(current * stepSize) + if localFlagLyr is None or localFlagLyr.featureCount() == 0: + continue + unused_boundary_flag_sink.addFeatures( + localFlagLyr.getFeatures(), QgsFeatureSink.FastInsert + ) + + def extractFeaturesUsingGeographicBounds( + self, inputLyr, geographicBounds, context, onlySelected=False, feedback=None + ): + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(2, feedback) + if feedback is not None + else None + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(0) + extractedLyr = self.algRunner.runExtractByLocation( + inputLyr=inputLyr + if not onlySelected + else QgsProcessingFeatureSourceDefinition(inputLyr.id(), True), + intersectLyr=geographicBounds, + context=context, feedback=multiStepFeedback, - ctx=context, ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(1) + self.algRunner.runCreateSpatialIndex( + inputLyr=extractedLyr, context=context, feedback=multiStepFeedback + ) + return extractedLyr def name(self): """ diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/LayerManagementAlgs/detectNullGeometriesAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/detectNullGeometriesAlgorithm.py similarity index 97% rename from DsgTools/core/DSGToolsProcessingAlgs/Algs/LayerManagementAlgs/detectNullGeometriesAlgorithm.py rename to DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/detectNullGeometriesAlgorithm.py index e8824c539..74406244e 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/LayerManagementAlgs/detectNullGeometriesAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/detectNullGeometriesAlgorithm.py @@ -116,7 +116,7 @@ def group(self): Returns the name of the group this algorithm belongs to. This string should be localised. """ - return self.tr("Layer Management Algorithms") + return self.tr("Quality Assurance Tools (Identification Processes)") def groupId(self): """ @@ -126,7 +126,7 @@ def groupId(self): contain lowercase alphanumeric characters only and no spaces or other formatting characters. """ - return "DSGTools: Layer Management Algorithms" + return "DSGTools: Quality Assurance Tools (Identification Processes)" def tr(self, string): return QCoreApplication.translate("DetectNullGeometriesAlgorithm", string) diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/dissolvePolygonsWithSameAttributesAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/dissolvePolygonsWithSameAttributesAlgorithm.py index e644d9b41..d675d6a12 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/dissolvePolygonsWithSameAttributesAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/dissolvePolygonsWithSameAttributesAlgorithm.py @@ -22,28 +22,16 @@ """ from PyQt5.QtCore import QCoreApplication -import processing from DsgTools.core.GeometricTools.layerHandler import LayerHandler from qgis.core import ( - QgsDataSourceUri, - QgsFeature, - QgsFeatureSink, - QgsGeometry, QgsProcessing, - QgsProcessingAlgorithm, QgsProcessingMultiStepFeedback, QgsProcessingOutputVectorLayer, QgsProcessingParameterBoolean, - QgsProcessingParameterEnum, - QgsProcessingParameterFeatureSink, - QgsProcessingParameterFeatureSource, QgsProcessingParameterField, - QgsProcessingParameterMultipleLayers, - QgsProcessingParameterNumber, QgsProcessingParameterVectorLayer, - QgsProcessingUtils, - QgsSpatialIndex, QgsWkbTypes, + QgsProcessingParameterDistance, ) from ...algRunner import AlgRunner @@ -73,11 +61,17 @@ def initAlgorithm(self, config): self.SELECTED, self.tr("Process only selected features") ) ) - self.addParameter( - QgsProcessingParameterNumber( - self.MIN_AREA, self.tr("Max dissolve area"), minValue=0, optional=True - ) + param = QgsProcessingParameterDistance( + self.MIN_AREA, + self.tr("Max dissolve area"), + parentParameterName=self.INPUT, + minValue=0, + optional=True, ) + param.setMetadata( {'widget_wrapper': + { 'decimals': 10 } + }) + self.addParameter(param) self.addParameter( QgsProcessingParameterField( self.ATTRIBUTE_BLACK_LIST, @@ -138,6 +132,7 @@ def processAlgorithm(self, parameters, context, feedback): attributeBlackList=attributeBlackList, onlySelected=onlySelected, feedback=multiStepFeedback, + attributeTupple=False if tol == -1 else True, ) currentStep += 1 diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/enforceSpatialRulesAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/enforceSpatialRulesAlgorithm.py index c120d24ce..6084225bf 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/enforceSpatialRulesAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/enforceSpatialRulesAlgorithm.py @@ -148,7 +148,7 @@ def setFlags(self, flagDict, ptLayer, lLayer, polLayer): newFeature = QgsFeature(fields) newFeature["reason"] = flagText.format(text=flag["text"]) newFeature.setGeometry(g) - layerMap[geom.type()].addFeature( + layerMap[g.type()].addFeature( newFeature, QgsFeatureSink.FastInsert ) return (ptLayer, lLayer, polLayer) diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyAndFixInvalidGeometriesAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyAndFixInvalidGeometriesAlgorithm.py index 7663dbf0a..086a18f36 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyAndFixInvalidGeometriesAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyAndFixInvalidGeometriesAlgorithm.py @@ -107,7 +107,9 @@ def processAlgorithm(self, parameters, context, feedback): onlySelected = self.parameterAsBool(parameters, self.SELECTED, context) ignoreClosed = self.parameterAsBool(parameters, self.IGNORE_CLOSED, context) fixInput = self.parameterAsBool(parameters, self.TYPE, context) - self.prepareFlagSink(parameters, inputLyr, QgsWkbTypes.Point, context) + self.prepareFlagSink( + parameters, inputLyr, QgsWkbTypes.Point, context, addFeatId=True + ) multiStepFeedback = QgsProcessingMultiStepFeedback(2, feedback) currentStep = 0 @@ -128,7 +130,11 @@ def processAlgorithm(self, parameters, context, feedback): for current, (key, outDict) in enumerate(flagDict.items()): if multiStepFeedback.isCanceled(): break - self.flagFeature(flagGeom=outDict["geom"], flagText=outDict["reason"]) + self.flagFeature( + flagGeom=outDict["geom"], + flagText=f"""Reason: {outDict["reason"]}""", + featid=outDict["featid"], + ) multiStepFeedback.setProgress(current * progressSize) return {self.FLAGS: self.flag_id, self.OUTPUT: inputLyr} diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyGapsAndOverlapsInCoverageAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyGapsAndOverlapsInCoverageAlgorithm.py index 9ae7abc3d..0f2dfe783 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyGapsAndOverlapsInCoverageAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyGapsAndOverlapsInCoverageAlgorithm.py @@ -21,6 +21,7 @@ ***************************************************************************/ """ +from collections import defaultdict from PyQt5.QtCore import QCoreApplication import processing @@ -42,6 +43,7 @@ QgsProcessingUtils, QgsProject, QgsWkbTypes, + QgsProcessingMultiStepFeedback ) from .validationAlgorithm import ValidationAlgorithm @@ -109,22 +111,37 @@ def processAlgorithm(self, parameters, context, feedback): self.prepareFlagSink(parameters, inputLyrList[0], QgsWkbTypes.Polygon, context) # Compute the number of steps to display within the progress bar and # get features from source - + nSteps = 4 if not frameLyr else 5 + multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Creating unified layer")) coverage = layerHandler.createAndPopulateUnifiedVectorLayer( - inputLyrList, QgsWkbTypes.Polygon, onlySelected=onlySelected + inputLyrList, QgsWkbTypes.Polygon, onlySelected=onlySelected, feedback=multiStepFeedback ) - lyr = self.overlayCoverage(coverage, context) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Overlaying coverage")) + lyr = self.overlayCoverage(coverage, context, feedback=multiStepFeedback) + currentStep += 1 if frameLyr: + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Getting gaps with geographic bounds")) self.getGapsOfCoverageWithFrame(lyr, frameLyr, context) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) featureList, total = self.getIteratorAndFeatureCount( lyr ) # only selected is not applied because we are using an inner layer, not the original ones - geomDict = self.getGeomDict(featureList, isMulti, feedback, total) - self.raiseFlags(geomDict, feedback) + multiStepFeedback.setProgressText(self.tr("Raising flags")) + geomDict = self.getGeomDict(featureList, isMulti, multiStepFeedback, total) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + self.raiseFlags(geomDict, multiStepFeedback) QgsProject.instance().removeMapLayer(lyr) return {self.FLAGS: self.flag_id} - def overlayCoverage(self, coverage, context): + def overlayCoverage(self, coverage, context, feedback): output = QgsProcessingUtils.generateTempFilename("output.shp") parameters = { "ainput": coverage, @@ -142,7 +159,7 @@ def overlayCoverage(self, coverage, context): "GRASS_VECTOR_DSCO": "", "GRASS_VECTOR_LCO": "", } - x = processing.run("grass7:v.overlay", parameters, context=context) + x = processing.run("grass7:v.overlay", parameters, context=context, feedback=feedback) lyr = QgsProcessingUtils.mapLayerFromString(x["output"], context) lyr.setCrs(coverage.crs()) return lyr @@ -196,7 +213,7 @@ def getGapsOfCoverageWithFrame( self.flagFeature(geom, self.tr("Gap in coverage with frame")) def getGeomDict(self, featureList, isMulti, feedback, total): - geomDict = dict() + geomDict = defaultdict(list) for current, feat in enumerate(featureList): # Stop the algorithm if cancel button has been clicked if feedback.isCanceled(): @@ -205,8 +222,6 @@ def getGeomDict(self, featureList, isMulti, feedback, total): if isMulti and not geom.isMultipart(): geom.convertToMultiType() geomKey = geom.asWkb() - if geomKey not in geomDict: - geomDict[geomKey] = [] geomDict[geomKey].append(feat) # # Update the progress bar attrList = feat.attributes() diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyTerrainModelErrorsAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyTerrainModelErrorsAlgorithm.py index 2f0306e21..022cd3b03 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyTerrainModelErrorsAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/identifyTerrainModelErrorsAlgorithm.py @@ -20,6 +20,8 @@ * * ***************************************************************************/ """ +import concurrent.futures +import os from PyQt5.QtCore import QCoreApplication from qgis.core import ( QgsGeometry, @@ -31,6 +33,9 @@ QgsProcessingParameterNumber, QgsProcessingParameterVectorLayer, QgsWkbTypes, + QgsProcessingMultiStepFeedback, + QgsProcessingFeatureSourceDefinition, + QgsProcessingContext, ) from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner @@ -46,6 +51,7 @@ class IdentifyTerrainModelErrorsAlgorithm(ValidationAlgorithm): CONTOUR_INTERVAL = "CONTOUR_INTERVAL" GEOGRAPHIC_BOUNDS = "GEOGRAPHIC_BOUNDS" CONTOUR_ATTR = "CONTOUR_ATTR" + GROUP_BY_SPATIAL_PARTITION = "GROUP_BY_SPATIAL_PARTITION" POINT_FLAGS = "POINT_FLAGS" LINE_FLAGS = "LINE_FLAGS" @@ -87,7 +93,12 @@ def initAlgorithm(self, config): optional=False, ) ) - + self.addParameter( + QgsProcessingParameterBoolean( + self.GROUP_BY_SPATIAL_PARTITION, + self.tr("Run algothimn grouping by spatial partition"), + ) + ) self.addParameter( QgsProcessingParameterFeatureSink( self.POINT_FLAGS, self.tr("{0} Point Flags").format(self.displayName()) @@ -103,7 +114,9 @@ def processAlgorithm(self, parameters, context, feedback): """ Here is where the processing itself takes place. """ - spatialRealtionsHandler = SpatialRelationsHandler() + self.spatialRealtionsHandler = SpatialRelationsHandler() + self.algRunner = AlgRunner() + self.layerHandler = LayerHandler() inputLyr = self.parameterAsVectorLayer(parameters, self.INPUT, context) if inputLyr is None: raise QgsProcessingException( @@ -117,6 +130,9 @@ def processAlgorithm(self, parameters, context, feedback): geoBoundsLyr = self.parameterAsVectorLayer( parameters, self.GEOGRAPHIC_BOUNDS, context ) + groupBySpatialPartition = self.parameterAsBool( + parameters, self.GROUP_BY_SPATIAL_PARTITION, context + ) point_flagSink, point_flag_id = self.prepareAndReturnFlagSink( parameters, inputLyr, QgsWkbTypes.Point, context, self.POINT_FLAGS ) @@ -124,16 +140,30 @@ def processAlgorithm(self, parameters, context, feedback): parameters, inputLyr, QgsWkbTypes.LineString, context, self.LINE_FLAGS ) - invalidDict = spatialRealtionsHandler.validateTerrainModel( - contourLyr=inputLyr, - onlySelected=onlySelected, - heightFieldName=heightFieldName, - threshold=threshold, - geoBoundsLyr=geoBoundsLyr, - feedback=feedback, + invalidDict = ( + self.spatialRealtionsHandler.validateTerrainModel( + contourLyr=inputLyr, + onlySelected=onlySelected, + heightFieldName=heightFieldName, + threshold=threshold, + geoBoundsLyr=geoBoundsLyr, + feedback=feedback, + ) + if not groupBySpatialPartition + else self.validateTerrainModelInParalel( + contourLyr=inputLyr, + onlySelected=onlySelected, + heightFieldName=heightFieldName, + threshold=threshold, + geoBoundsLyr=geoBoundsLyr, + context=context, + feedback=feedback, + ) ) for flagGeom, text in invalidDict.items(): + if feedback.isCanceled(): + break geom = QgsGeometry() geom.fromWkb(flagGeom) flagSink = ( @@ -145,6 +175,123 @@ def processAlgorithm(self, parameters, context, feedback): return {self.POINT_FLAGS: point_flag_id, self.LINE_FLAGS: line_flag_id} + def validateTerrainModelInParalel( + self, + contourLyr, + onlySelected, + heightFieldName, + threshold, + geoBoundsLyr, + context, + feedback, + ): + flagDict = dict() + nSteps = 3 + multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Splitting geographic bounds")) + geographicBoundaryLayerList = self.layerHandler.createMemoryLayerForEachFeature( + layer=geoBoundsLyr, context=context, feedback=multiStepFeedback + ) + currentStep += 1 + + def compute(localGeographicBoundsLyr): + localContext = QgsProcessingContext() + if multiStepFeedback.isCanceled(): + return {} + bufferedBounds = self.algRunner.runBuffer( + inputLayer=localGeographicBoundsLyr, + distance=1e-6, + context=localContext, + feedback=None, + ) + if multiStepFeedback.isCanceled(): + return {} + clippedContours = self.algRunner.runClip( + inputLayer=contourLyr + if not onlySelected + else QgsProcessingFeatureSourceDefinition(contourLyr.id(), True), + overlayLayer=bufferedBounds, + context=localContext, + feedback=None, + ) + if multiStepFeedback.isCanceled(): + return {} + singlePartContours = self.algRunner.runMultipartToSingleParts( + inputLayer=clippedContours, context=localContext, feedback=None + ) + if multiStepFeedback.isCanceled(): + return {} + return self.spatialRealtionsHandler.validateTerrainModel( + contourLyr=singlePartContours, + onlySelected=False, + heightFieldName=heightFieldName, + threshold=threshold, + geoBoundsLyr=localGeographicBoundsLyr, + feedback=None, + ) + + multiStepFeedback.setCurrentStep(currentStep) + nRegions = len(geographicBoundaryLayerList) + if nRegions == 0: + return flagDict + stepSize = 100 / nRegions + futures = set() + pool = concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count() - 1) + multiStepFeedback.pushInfo( + self.tr( + "Submitting terrain model problem identification by region tasks to thread..." + ) + ) + for current, localGeographicBoundsLyr in enumerate( + geographicBoundaryLayerList, start=0 + ): + if multiStepFeedback.isCanceled(): + break + futures.add(pool.submit(compute, localGeographicBoundsLyr)) + multiStepFeedback.setProgress(current * stepSize) + + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + + multiStepFeedback.pushInfo(self.tr("Evaluating results...")) + for current, future in enumerate(concurrent.futures.as_completed(futures)): + if multiStepFeedback.isCanceled(): + break + localFlagDict = future.result() + multiStepFeedback.pushInfo( + self.tr(f"Identification of region {current+1}/{nRegions} is done.") + ) + multiStepFeedback.setProgress(current * stepSize) + flagDict.update(localFlagDict) + return flagDict + + def extractFeaturesUsingBufferedGeographicBounds( + self, inputLyr, geographicBounds, context, onlySelected=False, feedback=None + ): + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(2, feedback) + if feedback is not None + else None + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(0) + extractedLyr = self.algRunner.runExtractByLocation( + inputLyr=inputLyr + if not onlySelected + else QgsProcessingFeatureSourceDefinition(inputLyr.id(), True), + intersectLyr=geographicBounds, + context=context, + feedback=multiStepFeedback, + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(1) + self.algRunner.runCreateSpatialIndex( + inputLyr=extractedLyr, context=context, feedback=multiStepFeedback + ) + return extractedLyr + def name(self): """ Returns the algorithm name, used for identifying the algorithm. This diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/overlayElementsWithAreasAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/overlayElementsWithAreasAlgorithm.py index efff4a487..3b3e3725d 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/overlayElementsWithAreasAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/overlayElementsWithAreasAlgorithm.py @@ -151,6 +151,7 @@ def processAlgorithm(self, parameters, context, feedback): overlayLyr.renameAttribute(0, "fid") overlayLyr.renameAttribute(1, "cl") overlayLyr.commitChanges() + self.algRunner.runCreateSpatialIndex(overlayLyr, context=context) # 1- check method # 2- if overlay and keep, use clip and symetric difference # 3- if remove outside, use clip @@ -193,10 +194,9 @@ def runOverlay(self, lyr, overlayLyr, behavior, context, feedback): if behavior == OverlayElementsWithAreasAlgorithm.RemoveInside: return outputDiffLyr if behavior == OverlayElementsWithAreasAlgorithm.OverlayAndKeep: - outsideFeats = [i for i in outputDiffLyr.getFeatures()] outputLyr.startEditing() outputLyr.beginEditCommand("") - outputLyr.addFeatures(outsideFeats) + outputLyr.addFeatures(outputDiffLyr.getFeatures()) outputLyr.endEditCommand() outputLyr.commitChanges() return outputLyr diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapLayerOnLayerAndUpdateAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapLayerOnLayerAndUpdateAlgorithm.py index bc7c371e0..1752a69ae 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapLayerOnLayerAndUpdateAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapLayerOnLayerAndUpdateAlgorithm.py @@ -44,6 +44,7 @@ QgsProcessingParameterMultipleLayers, QgsProcessingParameterNumber, QgsProcessingParameterVectorLayer, + QgsProcessingParameterFeatureSource, QgsProcessingUtils, QgsSpatialIndex, QgsWkbTypes, @@ -81,7 +82,7 @@ def initAlgorithm(self, config): ) self.addParameter( - QgsProcessingParameterVectorLayer( + QgsProcessingParameterFeatureSource( self.REFERENCE_LAYER, self.tr("Reference layer"), [QgsProcessing.TypeVectorAnyGeometry], @@ -133,7 +134,7 @@ def processAlgorithm(self, parameters, context, feedback): self.invalidSourceError(parameters, self.INPUT) ) onlySelected = self.parameterAsBool(parameters, self.SELECTED, context) - refLyr = self.parameterAsVectorLayer(parameters, self.REFERENCE_LAYER, context) + refLyr = self.parameterAsSource(parameters, self.REFERENCE_LAYER, context) if refLyr is None: raise QgsProcessingException( self.invalidSourceError(parameters, self.REFERENCE_LAYER) @@ -162,19 +163,19 @@ def processAlgorithm(self, parameters, context, feedback): multiStepFeedback.setProgressText(self.tr("Building local cache...")) multiStepFeedback.setCurrentStep(currentStep) refLyr = algRunner.runAddAutoIncrementalField( - refLyr, context, multiStepFeedback + refLyr, context, multiStepFeedback, is_child_algorithm=True ) currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) - algRunner.runCreateSpatialIndex(refLyr, context, multiStepFeedback) + algRunner.runCreateSpatialIndex(refLyr, context, multiStepFeedback, is_child_algorithm=True) currentStep += 1 multiStepFeedback.setCurrentStep(currentStep) multiStepFeedback.setProgressText(self.tr("Running snap...")) snapped = algRunner.runSnapGeometriesToLayer( inputLayer=auxLyr, - referenceLayer=refLyr, + referenceLayer=refLyr if buildLocalCache else parameters[self.REFERENCE_LAYER], tol=tol, behavior=behavior, context=context, diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapToGridAndUpdateAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapToGridAndUpdateAlgorithm.py index 46870da80..76db5a7e3 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapToGridAndUpdateAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/snapToGridAndUpdateAlgorithm.py @@ -74,16 +74,15 @@ def initAlgorithm(self, config): self.SELECTED, self.tr("Process only selected features") ) ) - - self.addParameter( - QgsProcessingParameterDistance( - self.TOLERANCE, - self.tr("Tolerance"), - parentParameterName=self.INPUT, - minValue=0, - defaultValue=0.001, - ) + param = QgsProcessingParameterDistance( + self.TOLERANCE, + self.tr("Tolerance"), + parentParameterName=self.INPUT, + minValue=0, + defaultValue=1e-10, ) + param.setMetadata({'widget_wrapper':{'decimals': 12}}) + self.addParameter(param) self.addOutput( QgsProcessingOutputVectorLayer( self.OUTPUT, self.tr("Original layer with features snapped to grid") diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/LayerManagementAlgs/spellCheckerAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/spellCheckerAlgorithm.py similarity index 88% rename from DsgTools/core/DSGToolsProcessingAlgs/Algs/LayerManagementAlgs/spellCheckerAlgorithm.py rename to DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/spellCheckerAlgorithm.py index 8ea159149..2f3e2540a 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/LayerManagementAlgs/spellCheckerAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/spellCheckerAlgorithm.py @@ -16,7 +16,7 @@ ) from qgis.PyQt.Qt import QVariant -from .spellChecker.spellCheckerCtrl import SpellCheckerCtrl +from ..LayerManagementAlgs.spellChecker.spellCheckerCtrl import SpellCheckerCtrl class SpellCheckerAlgorithm(QgsProcessingAlgorithm): @@ -153,10 +153,21 @@ def displayName(self): return self.tr("Spell check") def group(self): - return self.tr("Layer Management Algorithms") + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Quality Assurance Tools (Identification Processes)") def groupId(self): - return "DSGTools: Layer Management Algorithms" + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Quality Assurance Tools (Identification Processes)" def tr(self, string): return QCoreApplication.translate("SpellCheckerAlgorithm", string) diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/streamOrder.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/streamOrder.py new file mode 100644 index 000000000..ebc574412 --- /dev/null +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/streamOrder.py @@ -0,0 +1,177 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-03-29 + git sha : $Format:%H$ + copyright : (C) 2023 by Philipe Borba - Cartographic Engineer @ Brazilian Army + email : borba.philipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +from DsgTools.core.DSGToolsProcessingAlgs.algRunner import AlgRunner +from qgis.PyQt.QtCore import (QCoreApplication, QVariant) +from qgis.core import (QgsProcessing, + QgsFeatureSink, + QgsProcessingAlgorithm, + QgsProcessingParameterFeatureSink, + QgsProcessingParameterVectorLayer, + QgsFeature, + QgsField, + QgsGeometry, + QgsPointXY, + QgsProcessingException, + QgsProcessingMultiStepFeedback, + QgsProcessingParameterFeatureSource, + QgsVectorLayerUtils, + ) +from DsgTools.core.GeometricTools import graphHandler + +class StreamOrder(QgsProcessingAlgorithm): + + INPUT = 'INPUT' + OUTPUT = 'OUTPUT' + + def initAlgorithm(self, config=None): + self.addParameter( + QgsProcessingParameterFeatureSource( + self.INPUT, + self.tr("Input network"), + [QgsProcessing.TypeVectorLine], + optional=False, + ) + ) + + self.addParameter( + QgsProcessingParameterFeatureSink( + self.OUTPUT, + self.tr('Output') + ) + ) + + def processAlgorithm(self, parameters, context, feedback): + + try: + import networkx as nx + except ImportError: + raise QgsProcessingException( + self.tr( + "This algorithm requires the Python networkx library. Please install this library and try again." + ) + ) + algRunner = AlgRunner() + networkLayer = self.parameterAsSource(parameters, self.INPUT, context) + fields = networkLayer.fields() + fields.append(QgsField('stream_order', QVariant.Int)) + (sink, sink_id) = self.parameterAsSink( + parameters, + self.OUTPUT, + context, + fields, + networkLayer.wkbType(), + networkLayer.sourceCrs(), + ) + multiStepFeedback = QgsProcessingMultiStepFeedback(6, feedback) + currentStep = 0 + multiStepFeedback.setCurrentStep(currentStep) + localCache = algRunner.runCreateFieldWithExpression( + inputLyr=parameters[self.INPUT], + expression="$id", + fieldName="featid", + fieldType=1, + context=context, + feedback=multiStepFeedback, + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + algRunner.runCreateSpatialIndex( + inputLyr=localCache, context=context, feedback=multiStepFeedback + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + nodesLayer = algRunner.runExtractSpecificVertices( + inputLyr=localCache, + vertices="0,-1", + context=context, + feedback=multiStepFeedback, + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + ( + nodeDict, + nodeIdDict, + edgeDict, + hashDict, + networkBidirectionalGraph, + ) = graphHandler.buildAuxStructures( + nx, nodesLayer=nodesLayer, edgesLayer=localCache, feedback=multiStepFeedback, directed=True + ) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + G_copy = graphHandler.evaluateStreamOrder(networkBidirectionalGraph, feedback=multiStepFeedback) + currentStep += 1 + multiStepFeedback.setCurrentStep(currentStep) + if len(G_copy.edges) == 0: + return {self.OUTPUT: sink_id} + stepSize = 100 / len(G_copy.edges) + for current, (n0, n1) in enumerate(G_copy.edges): + if multiStepFeedback.isCanceled(): + break + newFeat = QgsFeature(fields) + oldFeat = edgeDict[G_copy[n0][n1]["featid"]] + newFeat.setGeometry(oldFeat.geometry()) + for idx, attrValue in enumerate(oldFeat.attributes()): + newFeat.setAttribute(idx, attrValue) + newFeat["stream_order"] = G_copy[n0][n1]["stream_order"] + + sink.addFeature(newFeat) + multiStepFeedback.setProgress(current * stepSize) + return {self.OUTPUT: sink_id} + + def tr(self, string): + return QCoreApplication.translate('Processing', string) + + def createInstance(self): + return StreamOrder() + + def name(self): + return 'streamorder' + + def displayName(self): + """ + Returns the translated algorithm name, which should be used for any + user-visible display of the algorithm name. + """ + return self.tr("Stream Order") + + def group(self): + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Quality Assurance Tools (Network Processes)") + + def groupId(self): + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Quality Assurance Tools (Network Processes)" + + def shortHelpString(self): + return self.tr("O algoritmo orderna ou direciona fluxo, como linhas de drenagem ") + diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/OtherAlgs/unicodeFilterAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/unicodeFilterAlgorithm.py similarity index 88% rename from DsgTools/core/DSGToolsProcessingAlgs/Algs/OtherAlgs/unicodeFilterAlgorithm.py rename to DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/unicodeFilterAlgorithm.py index f49be31da..f73cd09be 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/OtherAlgs/unicodeFilterAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/unicodeFilterAlgorithm.py @@ -151,13 +151,24 @@ def name(self): return "unicodefilter" def displayName(self): - return self.tr("Identifica Feições que contém unicode não permitido") + return self.tr("Identify features with invalid unicode") def group(self): - return self.tr("Other Algorithms") + """ + Returns the name of the group this algorithm belongs to. This string + should be localised. + """ + return self.tr("Quality Assurance Tools (Identification Processes)") def groupId(self): - return "DSGTools: Other Algorithms" + """ + Returns the unique ID of the group this algorithm belongs to. This + string should be fixed for the algorithm, and must not be localised. + The group id should be unique within each provider. Group id should + contain lowercase alphanumeric characters only and no spaces or other + formatting characters. + """ + return "DSGTools: Quality Assurance Tools (Identification Processes)" def shortHelpString(self): return self.tr( diff --git a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/validationAlgorithm.py b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/validationAlgorithm.py index 56d98bd85..ddbb326f9 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/validationAlgorithm.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/Algs/ValidationAlgs/validationAlgorithm.py @@ -58,13 +58,15 @@ def getIteratorAndFeatureCount(self, lyr, onlySelected=False): except: return [], 0 - def prepareFlagSink(self, parameters, source, wkbType, context): + def prepareFlagSink(self, parameters, source, wkbType, context, addFeatId=False): (self.flagSink, self.flag_id) = self.prepareAndReturnFlagSink( - parameters, source, wkbType, context, self.FLAGS + parameters, source, wkbType, context, self.FLAGS, addFeatId=addFeatId ) - def prepareAndReturnFlagSink(self, parameters, source, wkbType, context, UI_FIELD): - flagFields = self.getFlagFields() + def prepareAndReturnFlagSink( + self, parameters, source, wkbType, context, UI_FIELD, addFeatId=False + ): + flagFields = self.getFlagFields(addFeatId=addFeatId) (flagSink, flag_id) = self.parameterAsSink( parameters, UI_FIELD, @@ -77,20 +79,24 @@ def prepareAndReturnFlagSink(self, parameters, source, wkbType, context, UI_FIEL raise QgsProcessingException(self.invalidSinkError(parameters, UI_FIELD)) return (flagSink, flag_id) - def getFlagFields(self): + def getFlagFields(self, addFeatId=False): fields = QgsFields() fields.append(QgsField("reason", QVariant.String)) + if addFeatId: + fields.append(QgsField("featid", QVariant.String)) return fields - def flagFeature(self, flagGeom, flagText, fromWkb=False, sink=None): + def flagFeature(self, flagGeom, flagText, featid=None, fromWkb=False, sink=None): """ Creates and adds to flagSink a new flag with the reason. :param flagGeom: (QgsGeometry) geometry of the flag; :param flagText: (string) Text of the flag """ flagSink = self.flagSink if sink is None else sink - newFeat = QgsFeature(self.getFlagFields()) + newFeat = QgsFeature(self.getFlagFields(addFeatId=featid is not None)) newFeat["reason"] = flagText + if featid is not None: + newFeat["featid"] = featid if fromWkb: geom = QgsGeometry() geom.fromWkb(flagGeom) diff --git a/DsgTools/core/DSGToolsProcessingAlgs/algRunner.py b/DsgTools/core/DSGToolsProcessingAlgs/algRunner.py index 25321f5d2..99c43782a 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/algRunner.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/algRunner.py @@ -63,12 +63,24 @@ def getGrassReturn(self, outputDict, context, returnError=False): else: return lyr - def runDissolve(self, inputLyr, context, feedback=None, outputLyr=None, field=None, is_child_algorithm=False): + def runDissolve( + self, + inputLyr, + context, + feedback=None, + outputLyr=None, + field=None, + is_child_algorithm=False, + ): outputLyr = "memory:" if outputLyr is None else outputLyr field = [] if field is None else field parameters = {"INPUT": inputLyr, "FIELD": field, "OUTPUT": outputLyr} output = processing.run( - "native:dissolve", parameters, context=context, feedback=feedback, is_child_algorithm=is_child_algorithm + "native:dissolve", + parameters, + context=context, + feedback=feedback, + is_child_algorithm=is_child_algorithm, ) return output["OUTPUT"] @@ -504,11 +516,23 @@ def runRemoveNull(self, inputLayer, context, feedback=None, outputLyr=None): ) return output["OUTPUT"] - def runClip(self, inputLayer, overlayLayer, context, feedback=None, outputLyr=None): + def runClip( + self, + inputLayer, + overlayLayer, + context, + feedback=None, + outputLyr=None, + is_child_algorithm=False, + ): outputLyr = "memory:" if outputLyr is None else outputLyr parameters = {"INPUT": inputLayer, "OVERLAY": overlayLayer, "OUTPUT": outputLyr} output = processing.run( - "native:clip", parameters, context=context, feedback=feedback + "native:clip", + parameters, + context=context, + feedback=feedback, + is_child_algorithm=is_child_algorithm, ) return output["OUTPUT"] @@ -564,6 +588,7 @@ def runBuffer( mitterLimit=None, feedback=None, outputLyr=None, + is_child_algorithm=False, ): endCapStyle = 0 if endCapStyle is None else endCapStyle joinStyle = 0 if joinStyle is None else joinStyle @@ -581,7 +606,11 @@ def runBuffer( "OUTPUT": outputLyr, } output = processing.run( - "native:buffer", parameters, context=context, feedback=feedback + "native:buffer", + parameters, + context=context, + feedback=feedback, + is_child_algorithm=is_child_algorithm, ) return output["OUTPUT"] @@ -767,7 +796,7 @@ def runMergeVectorLayers( def runSaveSelectedFeatures(self, inputLyr, context, feedback=None, outputLyr=None): outputLyr = "memory:" if outputLyr is None else outputLyr - parameters = {"LAYERS": inputLyr, "OUTPUT": outputLyr} + parameters = {"INPUT": inputLyr, "OUTPUT": outputLyr} output = processing.run( "native:saveselectedfeatures", parameters, @@ -1005,6 +1034,7 @@ def runCreateFieldWithExpression( fieldPrecision=0, feedback=None, outputLyr=None, + is_child_algorithm=False, ): outputLyr = "memory:" if outputLyr is None else outputLyr output = processing.run( @@ -1019,6 +1049,7 @@ def runCreateFieldWithExpression( }, context=context, feedback=feedback, + is_child_algorithm=is_child_algorithm, ) return output["OUTPUT"] @@ -1272,6 +1303,7 @@ def runCreateGrid( outputLyr=None, hOverlay=0, vOverlay=0, + is_child_algorithm=False, ): outputLyr = "memory:" if outputLyr is None else outputLyr output = processing.run( @@ -1288,6 +1320,7 @@ def runCreateGrid( }, context=context, feedback=feedback, + is_child_algorithm=is_child_algorithm, ) return output["OUTPUT"] @@ -1367,3 +1400,101 @@ def runIdentifyUnsharedVertexOnSharedEdgesAlgorithm( is_child_algorithm=is_child_algorithm, ) return output["FLAGS"] + + def runShortestLine( + self, + sourceLayer, + destinationLayer, + context, + method=0, + neighbors=1, + maxDistance=None, + feedback=None, + outputLyr=None, + is_child_algorithm=False, + ): + outputLyr = "memory:" if outputLyr is None else outputLyr + output = processing.run( + "native:shortestline", + { + "SOURCE": sourceLayer, + "DESTINATION": destinationLayer, + "METHOD": method, + "NEIGHBORS": neighbors, + "DISTANCE": maxDistance, + "OUTPUT": outputLyr, + }, + context=context, + feedback=feedback, + is_child_algorithm=is_child_algorithm, + ) + return output["OUTPUT"] + + def runRetainFields( + self, + inputLayer, + fields, + context, + feedback=None, + outputLyr=None, + is_child_algorithm=False, + ): + outputLyr = "memory:" if outputLyr is None else outputLyr + output = processing.run( + "native:retainfields", + {"INPUT": inputLayer, "FIELDS": fields, "OUTPUT": "TEMPORARY_OUTPUT"}, + context=context, + feedback=feedback, + is_child_algorithm=is_child_algorithm, + ) + return output["OUTPUT"] + + def runExtractByExtent( + self, + inputLayer, + extent, + context, + clip=True, + feedback=None, + outputLyr=None, + is_child_algorithm=False, + ): + outputLyr = "memory:" if outputLyr is None else outputLyr + output = processing.run( + "native:extractbyextent", + { + "INPUT": inputLayer, + "EXTENT": extent, + "CLIP": clip, + "OUTPUT": outputLyr, + }, + context=context, + feedback=feedback, + is_child_algorithm=is_child_algorithm, + ) + return output["OUTPUT"] + + def runSelectByLocation( + self, + inputLyr, + intersectLyr, + context, + predicate=None, + method=None, + feedback=None, + is_child_algorithm=False, + ): + predicate = [0] if predicate is None else predicate + method = [0] if method is None else method + processing.run( + "native:selectbylocation", + { + "INPUT": inputLyr, + "INTERSECT": intersectLyr, + "PREDICATE": predicate, + "METHOD": method, + }, + context=context, + feedback=feedback, + is_child_algorithm=is_child_algorithm, + ) diff --git a/DsgTools/core/DSGToolsProcessingAlgs/dsgtoolsProcessingAlgorithmProvider.py b/DsgTools/core/DSGToolsProcessingAlgs/dsgtoolsProcessingAlgorithmProvider.py index 7f79a438f..d9bdeffc9 100644 --- a/DsgTools/core/DSGToolsProcessingAlgs/dsgtoolsProcessingAlgorithmProvider.py +++ b/DsgTools/core/DSGToolsProcessingAlgs/dsgtoolsProcessingAlgorithmProvider.py @@ -20,15 +20,36 @@ * * ***************************************************************************/ """ + +from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.extractByDE9IM import ( + ExtractByDE9IMAlgorithm, +) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.line2Multiline import ( + Line2Multiline, +) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.pointsInPolygonGridAlgorithm import ( + PointsInPolygonGridAlgorithm, +) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.selectByDE9IM import ( + SelectByDE9IMAlgorithm, +) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.splitPolygonsAlgorithm import ( + SplitPolygons, +) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.splitPolygonsByGrid import ( + SplitPolygonsByGrid, +) +from processing.core.ProcessingConfig import ProcessingConfig, Setting +from PyQt5.QtCore import QCoreApplication +from qgis.core import QgsApplication, QgsProcessingProvider +from qgis.PyQt.QtGui import QIcon + from DsgTools.core.DSGToolsProcessingAlgs.Algs.EditingAlgs.createEditingGridAlgorithm import ( CreateEditingGridAlgorithm, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.EnvironmentSetterAlgs.setFreeHandToolParametersAlgorithm import ( SetFreeHandToolParametersAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.buildTerrainSlicingFromContoursAlgorihtm import ( - BuildTerrainSlicingFromContoursAlgorihtm, -) from DsgTools.core.DSGToolsProcessingAlgs.Algs.GeometricAlgs.donutHoleExtractorAlgorithm import ( DonutHoleExtractorAlgorithm, ) @@ -68,15 +89,9 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.assignFormatRulesToLayersAlgorithm import ( AssignFormatRulesToLayersAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.detectNullGeometriesAlgorithm import ( - DetectNullGeometriesAlgorithm, -) from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.assignMeasureColumnToLayersAlgorithm import ( AssignMeasureColumnToLayersAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.lockAttributeEditingAlgorithm import ( - LockAttributeEditingAlgorithm, -) from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.assignValueMapToLayersAlgorithm import ( AssignValueMapToLayersAlgorithm, ) @@ -95,14 +110,17 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.loadShapefileAlgorithm import ( LoadShapefileAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.lockAttributeEditingAlgorithm import ( + LockAttributeEditingAlgorithm, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.matchAndApplyQmlStylesToLayersAlgorithm import ( MatchAndApplyQmlStylesToLayersAlgorithm, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.removeEmptyLayers import ( RemoveEmptyLayers, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.spellCheckerAlgorithm import ( - SpellCheckerAlgorithm, +from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.setRemoveDuplicateNodePropertyOnLayers import ( + SetRemoveDuplicateNodePropertyOnLayers, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.batchRunAlgorithm import ( BatchRunAlgorithm, @@ -116,28 +134,15 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.createFramesWithConstraintAlgorithm import ( CreateFramesWithConstraintAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.filterLayerListByGeometryType import ( - FilterLayerListByGeometryType, -) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.selectFeaturesOnCurrentCanvas import ( - SelectFeaturesOnCurrentCanvas, -) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.addUnsharedVertexOnIntersectionsAlgorithm import ( - AddUnsharedVertexOnIntersectionsAlgorithm, -) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.addUnsharedVertexOnSharedEdgesAlgorithm import ( - AddUnsharedVertexOnSharedEdgesAlgorithm, -) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.extendLinesToGeographicBoundsAlgorithm import ( - ExtendLinesToGeographicBoundsAlgorithm, -) -from .Algs.OtherAlgs.createReviewGridAlgorithm import CreateReviewGridAlgorithm from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.exportToMemoryLayer import ( ExportToMemoryLayer, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.fileInventoryAlgorithm import ( FileInventoryAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.filterLayerListByGeometryType import ( + FilterLayerListByGeometryType, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.pecCalculatorAlgorithm import ( PecCalculatorAlgorithm, ) @@ -154,18 +159,24 @@ ParameterFMEManagerType, RunRemoteFMEAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.selectFeaturesOnCurrentCanvas import ( + SelectFeaturesOnCurrentCanvas, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.stringCsvToFirstLayerWithElementsAlgorithm import ( StringCsvToFirstLayerWithElementsAlgorithm, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.stringCsvToLayerListAlgorithm import ( StringCsvToLayerListAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.unicodeFilterAlgorithm import ( - UnicodeFilterAlgorithm, -) from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.updateOriginalLayerAlgorithm import ( UpdateOriginalLayerAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.addUnsharedVertexOnIntersectionsAlgorithm import ( + AddUnsharedVertexOnIntersectionsAlgorithm, +) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.addUnsharedVertexOnSharedEdgesAlgorithm import ( + AddUnsharedVertexOnSharedEdgesAlgorithm, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.adjustNetworkConnectivityAlgorithm import ( AdjustNetworkConnectivityAlgorithm, ) @@ -181,6 +192,9 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.deaggregateGeometriesAlgorithm import ( DeaggregatorAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.detectNullGeometriesAlgorithm import ( + DetectNullGeometriesAlgorithm, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.dissolvePolygonsWithSameAttributesAlgorithm import ( DissolvePolygonsWithSameAttributesAlgorithm, ) @@ -191,6 +205,9 @@ EnforceSpatialRulesAlgorithm, ParameterSpatialRulesSetType, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.extendLinesToGeographicBoundsAlgorithm import ( + ExtendLinesToGeographicBoundsAlgorithm, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.hierarchicalSnapLayerOnLayerAndUpdateAlgorithm import ( HierarchicalSnapLayerOnLayerAndUpdateAlgorithm, ParameterSnapHierarchyType, @@ -207,15 +224,18 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDanglesAlgorithm import ( IdentifyDanglesAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDrainageAngleIssues import ( + IdentifyDrainageAngleIssues, +) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDrainageFlowIssues import ( + IdentifyDrainageFlowIssues, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDrainageFlowIssuesWithOtherHydrographicClassesAlgorithm import ( IdentifyDrainageFlowIssuesWithHydrographyElementsAlgorithm, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDrainageLoops import ( IdentifyDrainageLoops, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyNetworkConstructionIssuesAlgorithm import ( - IdentifyNetworkConstructionIssuesAlgorithm, -) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDuplicatedFeaturesAlgorithm import ( IdentifyDuplicatedFeaturesAlgorithm, ) @@ -249,8 +269,8 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyMultiPartGeometriesAlgorithm import ( IdentifyMultiPartGeometriesAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyUnmergedLinesWithSameAttributeSetAlgorithm import ( - IdentifyUnmergedLinesWithSameAttributeSetAlgorithm, +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyNetworkConstructionIssuesAlgorithm import ( + IdentifyNetworkConstructionIssuesAlgorithm, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyOutOfBoundsAnglesAlgorithm import ( IdentifyOutOfBoundsAnglesAlgorithm, @@ -285,6 +305,9 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyTerrainModelErrorsAlgorithm import ( IdentifyTerrainModelErrorsAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyUnmergedLinesWithSameAttributeSetAlgorithm import ( + IdentifyUnmergedLinesWithSameAttributeSetAlgorithm, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyUnsharedVertexOnIntersectionsAlgorithm import ( IdentifyUnsharedVertexOnIntersectionsAlgorithm, ) @@ -332,6 +355,9 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.snapToGridAndUpdateAlgorithm import ( SnapToGridAndUpdateAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.spellCheckerAlgorithm import ( + SpellCheckerAlgorithm, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.topologicalCleanAlgorithm import ( TopologicalCleanAlgorithm, ) @@ -350,25 +376,21 @@ from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.unbuildPolygonsAlgorithm import ( UnbuildPolygonsAlgorithm, ) +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.unicodeFilterAlgorithm import ( + UnicodeFilterAlgorithm, +) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.verifyCountourStackingAlgorithm import ( VerifyCountourStackingAlgorihtm, ) from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.verifyNetworkDirectioningAlgorithm import ( VerifyNetworkDirectioningAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDrainageFlowIssues import ( - IdentifyDrainageFlowIssues, +from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.streamOrder import ( + StreamOrder, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.ValidationAlgs.identifyDrainageAngleIssues import ( - IdentifyDrainageAngleIssues, +from DsgTools.core.DSGToolsProcessingAlgs.Algs.OtherAlgs.createReviewGridAlgorithm import ( + CreateReviewGridAlgorithm, ) -from DsgTools.core.DSGToolsProcessingAlgs.Algs.LayerManagementAlgs.setRemoveDuplicateNodePropertyOnLayers import ( - SetRemoveDuplicateNodePropertyOnLayers, -) -from processing.core.ProcessingConfig import ProcessingConfig, Setting -from PyQt5.QtCore import QCoreApplication -from qgis.core import QgsApplication, QgsProcessingProvider -from qgis.PyQt.QtGui import QIcon class DSGToolsProcessingAlgorithmProvider(QgsProcessingProvider): @@ -491,7 +513,6 @@ def getAlgList(self): StringCsvToFirstLayerWithElementsAlgorithm(), IdentifyDrainageFlowIssues(), IdentifyDrainageAngleIssues(), - BuildTerrainSlicingFromContoursAlgorihtm(), SetRemoveDuplicateNodePropertyOnLayers(), IdentifyDrainageLoops(), IdentifyDrainageFlowIssuesWithHydrographyElementsAlgorithm(), @@ -503,6 +524,13 @@ def getAlgList(self): FilterLayerListByGeometryType(), SmallHoleRemoverAlgorithm(), ReclassifyAdjacentPolygonsAlgorithm(), + StreamOrder(), + PointsInPolygonGridAlgorithm(), + SplitPolygons(), + SplitPolygonsByGrid(), + SelectByDE9IMAlgorithm(), + ExtractByDE9IMAlgorithm(), + Line2Multiline(), ] return algList diff --git a/DsgTools/core/GeometricTools/graphHandler.py b/DsgTools/core/GeometricTools/graphHandler.py new file mode 100644 index 000000000..47612656d --- /dev/null +++ b/DsgTools/core/GeometricTools/graphHandler.py @@ -0,0 +1,166 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-03-29 + git sha : $Format:%H$ + copyright : (C) 2023 by Philipe Borba - Cartographic Engineer @ Brazilian Army + email : borba.philipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +from collections import defaultdict +from itertools import tee +from typing import Iterable +from itertools import chain +from itertools import product +from itertools import starmap +from functools import partial + +from qgis.core import QgsGeometry, QgsFeature, QgsProcessingMultiStepFeedback + +def fetch_connected_nodes(G, node, max_degree, seen=None, feedback=None): + if seen == None: + seen = [node] + for neighbor in G.neighbors(node): + if feedback is not None and feedback.isCanceled(): + break + if G.degree(neighbor) > max_degree: + continue + if neighbor not in seen: + seen.append(neighbor) + fetch_connected_nodes(G, neighbor, max_degree, seen) + return seen + + +def pairwise(iterable: Iterable) -> Iterable: + "s -> (s0,s1), (s1,s2), (s2, s3), ..." + a, b = tee(iterable) + next(b, None) + return zip(a, b) + +def flipLine(edgeDict: dict, edgeId: int) -> QgsFeature: + edgeFeat = edgeDict[edgeId] + edgeGeomAsQgsLine = edgeFeat.geometry().constGet() + reversedGeom = QgsGeometry(edgeGeomAsQgsLine.reversed()) + newFeat = QgsFeature(edgeFeat) + newFeat.setGeometry(reversedGeom) + return newFeat + +def buildGraph(nx, hashDict, nodeDict, feedback=None, directed=False): + G = nx.Graph() if not directed else nx.DiGraph() + progressStep = 100 / len(hashDict) + for current, (edgeId, (wkb_1, wkb_2)) in enumerate(hashDict.items()): + if feedback is not None and feedback.isCanceled(): + break + G.add_edge(nodeDict[wkb_1], nodeDict[wkb_2]) + G[nodeDict[wkb_1]][nodeDict[wkb_2]]["featid"] = edgeId + if feedback is not None: + feedback.setProgress(current * progressStep) + return G + +def buildAuxStructures(nx, nodesLayer, edgesLayer, feedback=None, directed=False): + multiStepFeedback = QgsProcessingMultiStepFeedback(3, feedback) if feedback is not None else None + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(0) + edgeDict = {feat["featid"]: feat for feat in edgesLayer.getFeatures()} + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(1) + nodeDict = defaultdict(list) + nodeIdDict = defaultdict(list) + nodeCount = nodesLayer.featureCount() + stepSize = 100 / nodeCount + auxId = 0 + hashDict = defaultdict(lambda: [[], []]) + for current, nodeFeat in enumerate(nodesLayer.getFeatures()): + if multiStepFeedback is not None and multiStepFeedback.isCanceled(): + break + geom = nodeFeat.geometry() + geomWkb = geom.asWkb() + if geomWkb not in nodeDict: + nodeDict[geomWkb] = auxId + nodeIdDict[auxId] = geomWkb + auxId += 1 + hashDict[nodeFeat["featid"]][nodeFeat["vertex_pos"]] = geomWkb + if multiStepFeedback is not None: + multiStepFeedback.setProgress(current * stepSize) + + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(2) + networkBidirectionalGraph = buildGraph( + nx, hashDict, nodeDict, feedback=multiStepFeedback, directed=directed + ) + return nodeDict, nodeIdDict, edgeDict, hashDict, networkBidirectionalGraph + +def evaluateStreamOrder(G, feedback=None): + G = G.copy() + firstOrderNodes = set(node for node in G.nodes if G.degree(node) == 1 and len(list(G.successors(node))) > 0) + stepSize = 100 / len(G.edges) + current = 0 + G_copy = G.copy() + visitedNodes = set() + for n0, n1 in G_copy.edges: + G_copy[n0][n1]["stream_order"] = 0 + while len(firstOrderNodes) > 0: + if feedback is not None and feedback.isCanceled(): + return G_copy + for node in firstOrderNodes: + if node in visitedNodes: + continue + connectedNodes = fetch_connected_nodes(G, node, 2) + pairs = [(a,b) for i in connectedNodes for a,b in G.out_edges(i)] + predOrderValueList = [G_copy[n0][node]["stream_order"] for n0 in G_copy.predecessors(node)] + startIdx = max(predOrderValueList) + 1 if len(predOrderValueList) > 0 else 1 + for idx, (n0, n1) in enumerate(pairs, start=startIdx): + current += 1 + if feedback is not None and feedback.isCanceled(): + return G_copy + G_copy[n0][n1]["stream_order"] = idx if G_copy.degree(n0) <= 2 else max( + [idx] + [G_copy[i][n0]["stream_order"] + 1 for i in G_copy.predecessors(n0)] + ) + G.remove_edge(n0, n1) + succList = list(G_copy.successors(n1)) + if len(succList) > 1: + for i in G_copy.successors(n1): + G_copy[n1][i]["stream_order"] = idx + 1 + G.remove_edge(n1, succList[0]) + if feedback is not None: + feedback.setProgress(current * stepSize) + for n in connectedNodes: + G.remove_node(n) + visitedNodes.add(n) + # visitedNodes.add(node) + firstOrderNodes = set(node for node in G.nodes if G.degree(node) == 1 and len(list(G.successors(node))) > 0) - visitedNodes + # firstOrderNodes = [node for node in G.nodes if G.degree(node) == 1 and node not in visitedNodes] + return G_copy + + +# def evaluateStreamOrder(nx, G, feedback=None): +# G = G.copy() +# firstOrderNodes = set(node for node in G.nodes if G.degree(node) == 1 and len(list(G.successors(node))) > 0) +# stepSize = 100 / len(G.edges) +# current = 0 +# G_copy = G.copy() +# visitedNodes = set() +# for n0, n1 in G_copy.edges: +# G_copy[n0][n1]["stream_order"] = 0 +# roots = (v for v, d in G.in_degree() if d == 0) +# leaves = (v for v, d in G.out_degree() if d == 0) +# all_paths = partial(nx.all_simple_paths, G) +# pathDict = defaultdict(list) +# for path in sorted(chain.from_iterable(starmap(all_paths, product(roots, leaves))), key=lambda x: len(x), reverse=True): +# pathDict[path[0]].append(path) + + + diff --git a/DsgTools/core/GeometricTools/layerHandler.py b/DsgTools/core/GeometricTools/layerHandler.py index 7c2744be4..99c3311f2 100644 --- a/DsgTools/core/GeometricTools/layerHandler.py +++ b/DsgTools/core/GeometricTools/layerHandler.py @@ -28,6 +28,7 @@ from itertools import combinations import os from typing import List +from uuid import uuid4 from processing.tools import dataobjects @@ -475,6 +476,9 @@ def updateOriginalLayerFeatures( idsToRemove, featuresToAdd = set(), set() lyr.startEditing() lyr.beginEditCommand("Updating layer {0}".format(lyr.name())) + nSteps = len(inputDict) + if nSteps == 0 or feedback.isCanceled(): + return localTotal = 100 / len(inputDict) if inputDict else 0 futures = set() pool = concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count() - 1) @@ -524,6 +528,10 @@ def evaluate(id_, featDict): list(map(changeGeometryLambda, geometriesToChange)) featuresToAdd = featuresToAdd.union(addedFeatures) idsToRemove = idsToRemove.union(deletedIds) + if current % 1000 == 0: + multiStepFeedback.pushInfo( + self.tr(f"Evaluated {current}/{nSteps} results.") + ) if feedback is not None: feedback.setProgress(localTotal * current) lyr.addFeatures(list(featuresToAdd)) @@ -1366,7 +1374,7 @@ def evaluate(feat): self.fixGeometryFromInput( inputLyr, parameterDict, geometryType, _newFeatSet, feat, geom, id ) - return flagDict, _newFeatSet + return flagDict, _newFeatSet, feat for current, feat in enumerate(iterator): if feedback is not None and feedback.isCanceled(): @@ -1377,16 +1385,29 @@ def evaluate(feat): if multiStepFeedback is not None: multiStepFeedback.setCurrentStep(1) multiStepFeedback.pushInfo(self.tr("Evaluating results...")) + pkFields = inputLyr.primaryKeyAttributes() + pkFieldNames = [ + field.name() + for idx, field in enumerate(inputLyr.fields()) + if idx in pkFields + ] for current, future in enumerate(concurrent.futures.as_completed(futures)): if feedback is not None and feedback.isCanceled(): break - output, _newFeatSet = future.result() + output, _newFeatSet, feat = future.result() if output: + featIdText = ( + f"{feat.id()}" + if pkFields == [] + else f"{','.join(feat.attribute(i) for i in pkFields)}" + ) + featIdText = featIdText.replace(",)", "").replace("(", "") for point, errorDict in output.items(): if point in flagDict: flagDict[point]["reason"] += errorDict["reason"] else: flagDict[point] = errorDict + flagDict[point]["featid"] = featIdText if _newFeatSet: newFeatSet = newFeatSet.union(_newFeatSet) if feedback is not None: @@ -1998,7 +2019,7 @@ def getMergedLayer( lyr if not onlySelected else algRunner.runSaveSelectedFeatures( - lineLyr, context, feedback=multiStepFeedback + lyr, context, feedback=multiStepFeedback ) ) currentStep += 1 @@ -2031,7 +2052,7 @@ def getCentroidsAndBoundariesFromPolygons( context = ( dataobjects.createContext(feedback=feedback) if context is None else context ) - multiStepFeedback = QgsProcessingMultiStepFeedback(7, feedback) + multiStepFeedback = QgsProcessingMultiStepFeedback(8, feedback) multiStepFeedback.setCurrentStep(0) multiStepFeedback.pushInfo(self.tr("Getting constraint lines")) linesLyr = self.getLinesLayerFromPolygonsAndLinesLayers( @@ -2052,14 +2073,16 @@ def getCentroidsAndBoundariesFromPolygons( inputLyr, context, feedback=multiStepFeedback ) multiStepFeedback.setCurrentStep(3) + algRunner.runCreateSpatialIndex(edgeLyr, context, feedback=multiStepFeedback) + multiStepFeedback.setCurrentStep(4) explodedEdges = algRunner.runExplodeLines( edgeLyr, context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(4) + multiStepFeedback.setCurrentStep(5) explodedWithoutDuplicates = algRunner.runRemoveDuplicatedGeometries( explodedEdges, context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(5) + multiStepFeedback.setCurrentStep(6) self.buildCenterPoints( inputLyr, outputCenterPointSink, @@ -2069,7 +2092,7 @@ def getCentroidsAndBoundariesFromPolygons( context=context, algRunner=algRunner, ) - multiStepFeedback.setCurrentStep(6) + multiStepFeedback.setCurrentStep(7) self.filterEdges( explodedWithoutDuplicates, constraintSpatialIdx, @@ -2083,8 +2106,6 @@ def buildCenterPoints( self, inputLyr, outputCenterPointSink, - polygonBoundaryLyr, - constraintLineLyr=None, context=None, feedback=None, algRunner=None, @@ -2098,34 +2119,16 @@ def buildCenterPoints( context = ( dataobjects.createContext(feedback=feedback) if context is None else context ) - multiStepFeedback = QgsProcessingMultiStepFeedback(6, feedback) + multiStepFeedback = QgsProcessingMultiStepFeedback(3, feedback) multiStepFeedback.setCurrentStep(0) - mergedLineLyr = ( - polygonBoundaryLyr - if constraintLineLyr is None - else algRunner.runMergeVectorLayers( - [polygonBoundaryLyr, constraintLineLyr], - context, - feedback=multiStepFeedback, - ) - ) - multiStepFeedback.setCurrentStep(1) - splitSegmentsLyr = algRunner.runExplodeLines( - mergedLineLyr, context, feedback=multiStepFeedback - ) - multiStepFeedback.setCurrentStep(2) - segmentsWithoutDuplicates = algRunner.runRemoveDuplicatedGeometries( - splitSegmentsLyr, context, feedback=multiStepFeedback - ) - multiStepFeedback.setCurrentStep(3) outputPolygonLyr = algRunner.runPolygonize( - segmentsWithoutDuplicates, context, feedback=multiStepFeedback + inputLyr, context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(4) + multiStepFeedback.setCurrentStep(1) centroidLyr = algRunner.runPointOnSurface( outputPolygonLyr, context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(5) + multiStepFeedback.setCurrentStep(2) centroidsWithAttributes = algRunner.runJoinAttributesByLocation( centroidLyr, inputLyr, context, feedback=multiStepFeedback ) @@ -2144,22 +2147,160 @@ def filterEdges( ): """ """ notBoundarySet = set() - stepSize = 100 / inputLyr.featureCount() - featList = [i for i in inputLyr.getFeatures()] - for current, feat in enumerate(featList): - if feedback is not None and feedback.isCanceled(): - break + nFeats = inputLyr.featureCount() + if nFeats == 0: + return + stepSize = 100 / nFeats + pool = concurrent.futures.ThreadPoolExecutor(max_workers=os.cpu_count() - 1) + futures = set() + + def evaluate(feat): + outputSet = set() featGeom = feat.geometry() featBB = featGeom.boundingBox() for candidateId in constraintSpatialIdx.intersects(featBB): if featGeom.within(constraintIdDict[candidateId].geometry()): - notBoundarySet.add(feat) - break - if feedback is not None: - feedback.setProgress(current * stepSize) - for feat in featList: - if feat not in notBoundarySet: + outputSet.add(feat) + return outputSet + return outputSet + + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(2, feedback) + if feedback is not None + else None + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(0) + multiStepFeedback.setProgressText(self.tr("Submitting tasks to thread")) + + for current, feat in enumerate(inputLyr.getFeatures()): + if multiStepFeedback is not None and multiStepFeedback.isCanceled(): + break + futures.add(pool.submit(evaluate, feat)) + if multiStepFeedback is not None: + multiStepFeedback.setProgress(current * stepSize) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(1) + multiStepFeedback.setProgressText(self.tr("Evaluating results")) + for current, future in enumerate(concurrent.futures.as_completed(futures)): + outputSet = future.result() + if outputSet == {}: + continue + for feat in outputSet: + if feat in notBoundarySet: + continue outputBoundarySink.addFeature(feat, QgsFeatureSink.FastInsert) + if multiStepFeedback is not None: + multiStepFeedback.setProgress(current * stepSize) + + def getPolygonsFromCenterPointsAndBoundariesAlt( + self, + inputCenterPointLyr, + constraintLineLyrList=None, + constraintPolygonLyrList=None, + attributeBlackList=None, + geographicBoundaryLyr=None, + onlySelected=False, + suppressPolygonWithoutCenterPointFlag=False, + context=None, + feedback=None, + algRunner=None, + ): + algRunner = AlgRunner() if algRunner is None else algRunner + constraintLineLyrList = ( + [] if constraintLineLyrList is None else constraintLineLyrList + ) + constraintPolygonList = ( + [] if constraintPolygonLyrList is None else constraintPolygonLyrList + ) + attributeBlackList = [] if attributeBlackList is None else attributeBlackList + constraintPolygonListWithGeoBounds = ( + constraintPolygonList + [geographicBoundaryLyr] + if geographicBoundaryLyr is not None + else constraintPolygonList + ) + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(8, feedback) + if feedback is not None + else None + ) + currentStep = 0 + if multiStepFeedback is not None: + multiStepFeedback.setProgressText(self.tr("Merging all into one layer...")) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) + constraintPolygonsAsPolygonsLyr = ( + algRunner.runMergeVectorLayers( + inputList=constraintPolygonListWithGeoBounds, + context=context, + feedback=multiStepFeedback, + ) + if len(constraintPolygonListWithGeoBounds) > 0 + else None + ) + currentStep += 1 + constraintPolygonsAsLinesLyr = ( + algRunner.runPolygonsToLines( + inputLyr=constraintPolygonsAsPolygonsLyr, + context=context, + feedback=multiStepFeedback, + ) + if constraintPolygonsAsPolygonsLyr is not None + else None + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) + allLinesLyr = algRunner.runMergeVectorLayers( + inputList=constraintLineLyrList + [constraintPolygonsAsLinesLyr] + if constraintPolygonsAsLinesLyr is not None + else constraintLineLyrList, + context=context, + feedback=multiStepFeedback, + ) + currentStep += 1 + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Exploding lines...")) + explodedLines = algRunner.runExplodeLines( + inputLyr=allLinesLyr, context=context, feedback=multiStepFeedback + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) + algRunner.runCreateSpatialIndex(explodedLines, context, multiStepFeedback) + currentStep += 1 + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText(self.tr("Splitting lines...")) + splitLines = algRunner.runSplitLinesWithLines( + inputLyr=explodedLines, + linesLyr=explodedLines, + context=context, + feedback=multiStepFeedback, + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText( + self.tr("Starting the process of building polygons...") + ) + builtPolygonLyr = algRunner.runPolygonize( + splitLines, context, feedback=multiStepFeedback + ) + currentStep += 1 + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) + multiStepFeedback.setProgressText( + self.tr("Relating center points with built polygons...") + ) + return self.relateCenterPointsWithPolygons( + inputCenterPointLyr, + builtPolygonLyr, + constraintPolygonList=constraintPolygonList, + geomBoundary=geographicBoundaryLyr, + attributeBlackList=attributeBlackList, + suppressPolygonWithoutCenterPointFlag=suppressPolygonWithoutCenterPointFlag, + context=context, + feedback=multiStepFeedback, + ) def getPolygonsFromCenterPointsAndBoundaries( self, @@ -2216,7 +2357,7 @@ def getPolygonsFromCenterPointsAndBoundaries( if geographicBoundaryLyr is not None else constraintPolygonList ) - multiStepFeedback = QgsProcessingMultiStepFeedback(7, feedback) + multiStepFeedback = QgsProcessingMultiStepFeedback(8, feedback) # 1. Merge Polygon lyrs into one currentStep = 0 multiStepFeedback.setCurrentStep(currentStep) @@ -2306,18 +2447,25 @@ def relateCenterPointsWithPolygons( :return polygonList, flagList: list of polygons (QgsFeature) """ nSteps = 4 if constraintPolygonList else 2 - multiStepFeedback = QgsProcessingMultiStepFeedback(nSteps, feedback) + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(nSteps, feedback) + if feedback is not None + else None + ) currentStep = 0 - multiStepFeedback.setCurrentStep(currentStep) + algRunner = AlgRunner() + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) # If there were no polygon list, it was breaking the method. # tried to solve it with this if else if constraintPolygonList: - constraintPolygonLyr = self.algRunner.runMergeVectorLayers( + constraintPolygonLyr = algRunner.runMergeVectorLayers( constraintPolygonList, context, feedback=multiStepFeedback ) currentStep += 1 - multiStepFeedback.setCurrentStep(currentStep) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) ( constraintPolygonLyrSpatialIdx, constraintPolygonLyrIdDict, @@ -2330,7 +2478,8 @@ def relateCenterPointsWithPolygons( QgsSpatialIndex(), {}, ) - multiStepFeedback.setCurrentStep(currentStep) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) builtPolygonToCenterPointDict = self.buildCenterPolygonToCenterPointDict( inputCenterPointLyr, builtPolygonLyr, @@ -2338,7 +2487,8 @@ def relateCenterPointsWithPolygons( feedback=multiStepFeedback, ) currentStep += 1 - multiStepFeedback.setCurrentStep(currentStep) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(currentStep) ( polygonList, flagList, @@ -2707,3 +2857,42 @@ def evaluateAddVertex(feat): layer.beginEditCommand(self.tr("DsgTools adding missing vertexes")) list(map(changeGeometryLambda, updateSet)) layer.endEditCommand() + + def createMemoryLayerForEachFeature( + self, layer, context, returnFeature=False, feedback=None + ): + layerList = [] + nFeats = layer.featureCount() + if nFeats == 0: + return layerList + stepSize = 100 / nFeats + for current, feat in enumerate(layer.getFeatures()): + if feedback is not None and feedback.isCanceled(): + return layerList + temp = self.createMemoryLayerWithFeature(layer, feat, context) + item = (feat, temp) if returnFeature else temp + layerList.append(item) + if feedback is not None: + feedback.setProgress(current * stepSize) + return layerList + + def createMemoryLayerWithFeature(self, layer, feat, context=None, isSource=False): + context = QgsProcessingContext() if context is None else context + crs = layer.crs() if not isSource else layer.sourceCrs() + temp_name = ( + f"{layer.name()}-{str(uuid4())}" if not isSource else f"{str(uuid4())}" + ) + temp = QgsVectorLayer( + f"{QgsWkbTypes.displayString(layer.wkbType())}?crs={crs.authid()}", + temp_name, + "memory", + ) + temp_data = temp.dataProvider() + fields = layer.dataProvider().fields() if not isSource else layer.fields() + temp_data.addAttributes(fields.toList()) + temp.updateFields() + temp_data.addFeature(feat) + self.algRunner.runCreateSpatialIndex( + inputLyr=temp, context=context, is_child_algorithm=True + ) + return temp diff --git a/DsgTools/core/GeometricTools/spatialRelationsHandler.py b/DsgTools/core/GeometricTools/spatialRelationsHandler.py index d5491e5d1..537b1b9da 100644 --- a/DsgTools/core/GeometricTools/spatialRelationsHandler.py +++ b/DsgTools/core/GeometricTools/spatialRelationsHandler.py @@ -20,25 +20,24 @@ * * ***************************************************************************/ """ -from __future__ import absolute_import + +import concurrent.futures from itertools import tee, combinations from collections import defaultdict, OrderedDict +import os from qgis.core import ( - Qgis, - QgsFeature, QgsProject, QgsGeometry, QgsExpression, QgsVectorLayer, QgsSpatialIndex, - QgsFeatureRequest, QgsProcessingContext, QgsProcessingFeedback, QgsProcessingMultiStepFeedback, + QgsFeatureRequest, ) -from qgis.analysis import QgsGeometrySnapper, QgsInternalGeometrySnapper from qgis.PyQt.Qt import QObject from qgis.PyQt.QtCore import QRegExp, QCoreApplication from qgis.PyQt.QtGui import QRegExpValidator @@ -68,6 +67,7 @@ class SpatialRelationsHandler(QObject): QCoreApplication.translate("EnforceSpatialRulesAlgorithm", "does not overlap"), QCoreApplication.translate("EnforceSpatialRulesAlgorithm", "contains"), QCoreApplication.translate("EnforceSpatialRulesAlgorithm", "does not contain"), + QCoreApplication.translate("EnforceSpatialRulesAlgorithm", "de9im"), ) ( EQUALS, @@ -85,6 +85,7 @@ class SpatialRelationsHandler(QObject): NOTOVERLAPS, CONTAINS, NOTCONTAINS, + DE9IM, ) = range(len(__predicates)) def __init__(self, iface=None, parent=None): @@ -112,16 +113,20 @@ def validateTerrainModel( Does several validation procedures with terrain elements. """ invalidDict = OrderedDict() - multiStepFeedback = QgsProcessingMultiStepFeedback( - 7, feedback + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(7, feedback) + if feedback is not None + else None ) # ajustar depois - multiStepFeedback.setCurrentStep(0) - multiStepFeedback.setProgressText(self.tr("Splitting lines...")) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(0) + multiStepFeedback.setProgressText(self.tr("Splitting lines...")) splitLinesLyr = self.algRunner.runSplitLinesWithLines( contourLyr, contourLyr, context=context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(1) - multiStepFeedback.setProgressText(self.tr("Building aux structure...")) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(1) + multiStepFeedback.setProgressText(self.tr("Building aux structure...")) ( contourSpatialIdx, contourIdDict, @@ -132,7 +137,8 @@ def validateTerrainModel( attributeName=heightFieldName, feedback=multiStepFeedback, ) - multiStepFeedback.setCurrentStep(2) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(2) geoBoundsGeomEngine, geoBoundsPolygonEngine = ( (None, None) if geoBoundsLyr is None @@ -140,8 +146,11 @@ def validateTerrainModel( geoBoundsLyr, context=context, feedback=multiStepFeedback ) ) - multiStepFeedback.setCurrentStep(3) - multiStepFeedback.setProgressText(self.tr("Validating contour relations...")) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(3) + multiStepFeedback.setProgressText( + self.tr("Validating contour relations...") + ) contourFlags = self.validateContourRelations( contourNodeDict, heightFieldName, @@ -149,18 +158,20 @@ def validateTerrainModel( geoBoundsPolygonEngine=geoBoundsPolygonEngine, ) invalidDict.update(contourFlags) - multiStepFeedback.setCurrentStep(4) - multiStepFeedback.setProgressText( - self.tr("Finding contour out of threshold...") - ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(4) + multiStepFeedback.setProgressText( + self.tr("Finding contour out of threshold...") + ) contourOutOfThresholdDict = self.findContourOutOfThreshold( heightsDict, threshold, feedback=multiStepFeedback ) invalidDict.update(contourOutOfThresholdDict) if len(invalidDict) > 0: return invalidDict - multiStepFeedback.setCurrentStep(5) - multiStepFeedback.setProgressText(self.tr("Building contour area dict..")) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(5) + multiStepFeedback.setProgressText(self.tr("Building contour area dict..")) contourAreaDict = self.buildContourAreaDict( inputLyr=splitLinesLyr, geoBoundsLyr=geoBoundsLyr, @@ -171,8 +182,9 @@ def validateTerrainModel( context=context, feedback=multiStepFeedback, ) - multiStepFeedback.setCurrentStep(6) - multiStepFeedback.setProgressText(self.tr("Finding missing contours...")) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(6) + multiStepFeedback.setProgressText(self.tr("Finding missing contours...")) misingContourDict = self.findMissingContours( contourAreaDict, threshold, context=context, feedback=multiStepFeedback ) @@ -185,12 +197,18 @@ def getGeoBoundsGeomEngine(self, geoBoundsLyr, context=None, feedback=None): """ if geoBoundsLyr is None: return None, None - multiStepFeedback = QgsProcessingMultiStepFeedback(2, feedback) - multiStepFeedback.setCurrentStep(0) + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(2, feedback) + if feedback is not None + else None + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(0) mergedPolygonLyr = self.algRunner.runAggregate( geoBoundsLyr, context=context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(1) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(1) mergedPolygonGeom = ( [i for i in mergedPolygonLyr.getFeatures()][0].geometry() if mergedPolygonLyr.featureCount() != 0 @@ -238,8 +256,13 @@ def buildContourAreaDict( "areaIdDict": {}, "areaContourRelations": {}, } - multiStepFeedback = QgsProcessingMultiStepFeedback(4, feedback) - multiStepFeedback.setCurrentStep(0) + multiStepFeedback = ( + QgsProcessingMultiStepFeedback(4, feedback) + if feedback is not None + else None + ) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(0) boundsLineLyr = ( self.algRunner.runPolygonsToLines( geoBoundsLyr, context, feedback=multiStepFeedback @@ -248,15 +271,18 @@ def buildContourAreaDict( else None ) lineLyrList = [inputLyr] if boundsLineLyr is None else [inputLyr, boundsLineLyr] - multiStepFeedback.setCurrentStep(1) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(1) linesLyr = self.algRunner.runMergeVectorLayers( lineLyrList, context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(2) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(2) polygonLyr = self.algRunner.runPolygonize( linesLyr, context, feedback=multiStepFeedback ) - multiStepFeedback.setCurrentStep(3) + if multiStepFeedback is not None: + multiStepFeedback.setCurrentStep(3) self.populateContourAreaDict( polygonLyr, geoBoundsLyr, diff --git a/DsgTools/core/NetworkTools/BDGExRequestHandler.py b/DsgTools/core/NetworkTools/BDGExRequestHandler.py index 699dda7c4..63b227d8f 100644 --- a/DsgTools/core/NetworkTools/BDGExRequestHandler.py +++ b/DsgTools/core/NetworkTools/BDGExRequestHandler.py @@ -45,15 +45,15 @@ def __init__(self, parent=None): super(BDGExRequestHandler, self).__init__() self.availableServicesDict = { "mapcache": { - "url": "https://bdgex.eb.mil.br/mapcache", + "url": "http://bdgex.eb.mil.br/mapcache", "services": {"WMS": dict()}, }, "mapindex": { - "url": "https://bdgex.eb.mil.br/cgi-bin/mapaindice", + "url": "http://bdgex.eb.mil.br/cgi-bin/mapaindice", "services": {"WMS": dict(), "WFS": dict()}, }, "auxlayers": { - "url": "https://bdgex.eb.mil.br/cgi-bin/geoportal", + "url": "http://bdgex.eb.mil.br/cgi-bin/geoportal", "services": {"WMS": dict(), "WFS": dict()}, }, } diff --git a/DsgTools/core/Utils/threadingTools.py b/DsgTools/core/Utils/threadingTools.py new file mode 100644 index 000000000..d60c4c57d --- /dev/null +++ b/DsgTools/core/Utils/threadingTools.py @@ -0,0 +1,63 @@ +# -*- coding: utf-8 -*- +""" +/*************************************************************************** + DsgTools + A QGIS plugin + Brazilian Army Cartographic Production Tools + ------------------- + begin : 2023-04-17 + git sha : $Format:%H$ + copyright : (C) 2023 by Philipe Borba - Cartographic Engineer @ Brazilian Army + email : borba.philipe@eb.mil.br + ***************************************************************************/ +/*************************************************************************** + * * + * This program is free software; you can redistribute it and/or modify * + * it under the terms of the GNU General Public License as published by * + * the Free Software Foundation; either version 2 of the License, or * + * (at your option) any later version. * + * * + ***************************************************************************/ +""" + +import itertools + +import concurrent.futures +import itertools + + +def concurrently(handler, inputs, *, max_concurrency=5): + """ + Calls the function ``handler`` on the values ``inputs``. + + ``handler`` should be a function that takes a single input, which is the + individual values in the iterable ``inputs``. + + Generates (input, output) tuples as the calls to ``handler`` complete. + + See https://alexwlchan.net/2019/10/adventures-with-concurrent-futures/ for an explanation + of how this function works. + + """ + # Make sure we get a consistent iterator throughout, rather than + # getting the first element repeatedly. + handler_inputs = iter(inputs) + + with concurrent.futures.ThreadPoolExecutor() as executor: + futures = { + executor.submit(handler, input): input + for input in itertools.islice(handler_inputs, max_concurrency) + } + + while futures: + done, _ = concurrent.futures.wait( + futures, return_when=concurrent.futures.FIRST_COMPLETED + ) + + for fut in done: + original_input = futures.pop(fut) + yield fut.result() + + for input in itertools.islice(handler_inputs, len(done)): + fut = executor.submit(handler, input) + futures[fut] = input diff --git a/DsgTools/gui/ProcessingUI/enforceSpatialRuleWrapper.py b/DsgTools/gui/ProcessingUI/enforceSpatialRuleWrapper.py index ee989e23a..f20c269e1 100644 --- a/DsgTools/gui/ProcessingUI/enforceSpatialRuleWrapper.py +++ b/DsgTools/gui/ProcessingUI/enforceSpatialRuleWrapper.py @@ -29,7 +29,6 @@ from qgis.PyQt.QtGui import QRegExpValidator from qgis.PyQt.QtWidgets import ( QWidget, - QCheckBox, QComboBox, QLineEdit, QVBoxLayout, @@ -153,13 +152,15 @@ def cardinalityWidget(self): le.setPlaceholderText("1..*") return le - def useDE9IM(self): - """ - Identifies whether user chose to input predicate as a DE-9IM mask. - :return: (bool) whether GUI should handle the DE-9IM mask widget over - the combo box selection. - """ - return self.panel.cb.isChecked() + def _check_de9im_is_available(self, row): + otw = self.panel.otw + predicate = otw.getValue(row, 3) + handler = SpatialRelationsHandler() + enableDE9IM = predicate == handler.DE9IM + otw.itemAt(row, 4).setEnabled(enableDE9IM) + if not enableDE9IM: + otw.setValue(row, 4, "") + return enableDE9IM def _checkCardinalityAvailability(self, row): """ @@ -170,11 +171,6 @@ def _checkCardinalityAvailability(self, row): :return: (bool) whether cardinality is available """ otw = self.panel.otw - if self.useDE9IM(): - # if user is using the DE-9IM input, cardinality won't be - # managed - otw.itemAt(row, 7).setEnabled(True) - return True predicate = otw.getValue(row, 3) handler = SpatialRelationsHandler() noCardinality = predicate in ( @@ -187,7 +183,7 @@ def _checkCardinalityAvailability(self, row): handler.NOTOVERLAPS, handler.NOTCONTAINS, ) - otw.itemAt(row, 7).setEnabled(not noCardinality) + otw.itemAt(row, 7).setEnabled(noCardinality) if noCardinality: otw.setValue(row, 7, "") return not noCardinality @@ -218,6 +214,10 @@ def postAddRowStandard(self, row): ) # also triggers the action for the first time it is open self._checkCardinalityAvailability(row) + predicateWidget.currentIndexChanged.connect( + partial(self._check_de9im_is_available, row) + ) + self._check_de9im_is_available(row) def postAddRowModeler(self, row): """ @@ -245,6 +245,10 @@ def checkLayerBeforeConnect(le, filterExp): partial(self._checkCardinalityAvailability, row) ) self._checkCardinalityAvailability(row) + predicateWidget.currentIndexChanged.connect( + partial(self._check_de9im_is_available, row) + ) + self._check_de9im_is_available(row) def standardPanel(self): """ @@ -254,9 +258,6 @@ def standardPanel(self): widget = QWidget() layout = QVBoxLayout() # added as an attribute in order to make it easier to be read - widget.cb = QCheckBox() - widget.cb.setText(self.tr("Use DE-9IM inputs")) - layout.addWidget(widget.cb) widget.otw = OrderedTableWidget( headerMap={ 0: { @@ -318,18 +319,6 @@ def standardPanel(self): } ) - def handlePredicateColumns(checked): - """ - Predicate input widgets are mutually exclusively: the user may only - input data through either of them. This method manages hiding and - showing correct columns in accord to the user selection. - :param checked: (bool) whether the DE-9IM usage checkbox is ticked. - """ - widget.otw.tableWidget.hideColumn(3 if checked else 4) - widget.otw.tableWidget.showColumn(4 if checked else 3) - - widget.cb.toggled.connect(handlePredicateColumns) - widget.cb.toggled.emit(widget.cb.isChecked()) widget.otw.setHeaderDoubleClickBehaviour("replicate") widget.otw.rowAdded.connect(self.postAddRowStandard) layout.addWidget(widget.otw) @@ -351,9 +340,6 @@ def modelerPanel(self): widget = QWidget() layout = QVBoxLayout() # added as an attribute in order to make it easier to be read - widget.cb = QCheckBox() - widget.cb.setText(self.tr("Use DE-9IM inputs")) - layout.addWidget(widget.cb) widget.otw = OrderedTableWidget( headerMap={ 0: { @@ -415,18 +401,6 @@ def modelerPanel(self): } ) - def handlePredicateColumns(checked): - """ - Predicate input widgets are mutually exclusively: the user may only - input data through either of them. This method manages hiding and - showing correct columns in accord to the user selection. - :param checked: (bool) whether the DE-9IM usage checkbox is ticked. - """ - widget.otw.tableWidget.hideColumn(3 if checked else 4) - widget.otw.tableWidget.showColumn(4 if checked else 3) - - widget.cb.toggled.connect(handlePredicateColumns) - widget.cb.toggled.emit(widget.cb.isChecked()) widget.otw.setHeaderDoubleClickBehaviour("replicate") widget.otw.rowAdded.connect(self.postAddRowModeler) layout.addWidget(widget.otw) @@ -492,10 +466,6 @@ def setValue(self, value): if not value: return otw = self.panel.otw - useDE9IM = value[0].get("useDE9IM", False) - self.panel.cb.setChecked(useDE9IM) - # signal must be triggered to adjust the correct column display - self.panel.cb.toggled.emit(useDE9IM) isNotModeler = self.dialogType != DIALOG_MODELER invalids = list() for rule in value: @@ -530,8 +500,9 @@ def readStandardPanel(self): """ ruleList = list() otw = self.panel.otw - useDe9im = self.useDE9IM() + handler = SpatialRelationsHandler() for row in range(otw.rowCount()): + useDE9IM = otw.getValue(row, 3) == handler.DE9IM ruleList.append( SpatialRule( name=otw.getValue(row, 0).strip(), # or \ @@ -543,7 +514,7 @@ def readStandardPanel(self): layer_b=otw.getValue(row, 5), filter_b=otw.getValue(row, 6), cardinality=otw.getValue(row, 7) or "1..*", - useDE9IM=useDe9im, + useDE9IM=useDE9IM, checkLoadedLayer=False, ).asDict() ) diff --git a/DsgTools/gui/ProductionTools/MapTools/GenericSelectionTool/genericSelectionTool.py b/DsgTools/gui/ProductionTools/MapTools/GenericSelectionTool/genericSelectionTool.py index 65f840a12..92ba5132d 100644 --- a/DsgTools/gui/ProductionTools/MapTools/GenericSelectionTool/genericSelectionTool.py +++ b/DsgTools/gui/ProductionTools/MapTools/GenericSelectionTool/genericSelectionTool.py @@ -69,7 +69,6 @@ def __init__(self, iface): self.reset() self.blackList = self.getBlackList() self.cursorChanged = False - self.cursorChangingHotkey = Qt.Key_Alt self.menuHovered = False # indicates hovering actions over context menu self.geometryHandler = GeometryHandler(iface=self.iface) @@ -91,17 +90,6 @@ def addTool(self, manager, callback, parentMenu, iconBasePath): ) self.setAction(action) - def keyPressEvent(self, e): - """ - Reimplemetation of keyPressEvent() in order to handle cursor changing hotkey (Alt). - """ - if e.key() == self.cursorChangingHotkey and not self.cursorChanged: - self.cursorChanged = True - QApplication.setOverrideCursor(QCursor(Qt.PointingHandCursor)) - else: - self.cursorChanged = False - QApplication.restoreOverrideCursor() - def getBlackList(self): settings = QSettings() settings.beginGroup("PythonPlugins/DsgTools/Options") @@ -120,17 +108,6 @@ def reset(self): self.isEmittingPoint = False self.rubberBand.reset(QgsWkbTypes.PolygonGeometry) - def keyPressEvent(self, e): - """ - Reimplemetation of keyPressEvent() in order to handle cursor changing hotkey (F2). - """ - if e.key() == self.cursorChangingHotkey and not self.cursorChanged: - self.cursorChanged = True - QApplication.setOverrideCursor(QCursor(Qt.PointingHandCursor)) - else: - self.cursorChanged = False - QApplication.restoreOverrideCursor() - def canvasMoveEvent(self, e): """ Used only on rectangle select. @@ -235,7 +212,7 @@ def layerHasPartInBlackList(self, lyrName): return True return False - def getPrimitiveDict(self, e, hasControlModifier=False): + def getPrimitiveDict(self, e, hasControlModifier=False, hasAltModifier=False): """ Builds a dict with keys as geometryTypes of layer, which are Qgis.Point (value 0), Qgis.Line (value 1) or Qgis.Polygon (value 2), and values as layers from self.iface.mapCanvas().layers(). When self.iface.mapCanvas().layers() is called, a list of @@ -245,7 +222,12 @@ def getPrimitiveDict(self, e, hasControlModifier=False): primitiveDict = dict() firstGeom = self.checkSelectedLayers() visibleLayers = QgsProject.instance().layerTreeRoot().checkedLayers() - for lyr in self.iface.mapCanvas().layers(): # ordered layers + iterator = ( + self.iface.mapCanvas().layers() + if not hasAltModifier + else [self.iface.activeLayer()] + ) + for lyr in iterator: # ordered layers # layer types other than VectorLayer are ignored, as well as layers in black list and layers that are not visible if ( not isinstance(lyr, QgsVectorLayer) @@ -738,19 +720,6 @@ def checkSelectedFeaturesOnDict(self, menuDict): notSelectedFeaturesDict[cl].append(feat) return selectedFeaturesDict, notSelectedFeaturesDict - def getSelectedRasters(self, e): - rasters = [] - rect = self.getCursorRect(e) - layers = self.iface.mapCanvas().layers() - for layer in self.iface.mapCanvas().layers(): - if not isinstance(layer, QgsRasterLayer): - continue - bbRect = self.canvas.mapSettings().mapToLayerCoordinates(layer, rect) - if not layer.extent().intersects(bbRect): - continue - rasters.append(layer) - return rasters - def addRasterMenu(self, menu, rasters): rasterMenu = QMenu(title="Rasters", parent=menu) for raster in rasters: @@ -767,84 +736,82 @@ def createContextMenu(self, e): if selected: firstGeom = self.checkSelectedLayers() # setting a list of features to iterate over - layerList = self.getPrimitiveDict(e, hasControlModifier=selected) + layerList = self.getPrimitiveDict( + e, + hasControlModifier=selected, + hasAltModifier=QApplication.keyboardModifiers() == Qt.AltModifier, + ) layers = [] for key in layerList: layers += layerList[key] - if layers: - rect = self.getCursorRect(e) - lyrFeatDict = dict() - for layer in layers: - if not isinstance(layer, QgsVectorLayer): + if not layers: + return + rect = self.getCursorRect(e) + lyrFeatDict = dict() + for layer in layers: + if not isinstance(layer, QgsVectorLayer): + continue + geomType = layer.geometryType() + # iterate over features inside the mouse bounding box + bbRect = self.canvas.mapSettings().mapToLayerCoordinates(layer, rect) + for feature in layer.getFeatures(QgsFeatureRequest(bbRect)): + geom = feature.geometry() + if not geom: continue - geomType = layer.geometryType() - # iterate over features inside the mouse bounding box - bbRect = self.canvas.mapSettings().mapToLayerCoordinates(layer, rect) - for feature in layer.getFeatures(QgsFeatureRequest(bbRect)): - geom = feature.geometry() - if geom: - searchRect = self.geometryHandler.reprojectSearchArea( - layer, rect - ) - if selected: - # if Control was held, appending behaviour is different - if not firstGeom: - firstGeom = geomType - elif firstGeom > geomType: - firstGeom = geomType - if geomType == firstGeom and geom.intersects(searchRect): - # only appends features if it has the same geometry as first selected feature - if layer in lyrFeatDict: - lyrFeatDict[layer].append(feature) - else: - lyrFeatDict[layer] = [feature] + searchRect = self.geometryHandler.reprojectSearchArea(layer, rect) + if selected: + # if Control was held, appending behaviour is different + if not firstGeom: + firstGeom = geomType + elif firstGeom > geomType: + firstGeom = geomType + if geomType == firstGeom and geom.intersects(searchRect): + # only appends features if it has the same geometry as first selected feature + if layer in lyrFeatDict: + lyrFeatDict[layer].append(feature) else: - if geom.intersects(searchRect): - if layer in lyrFeatDict: - lyrFeatDict[layer].append(feature) - else: - lyrFeatDict[layer] = [feature] - lyrFeatDict = self.filterStrongestGeometry(lyrFeatDict) - # rasters = self.getSelectedRasters(e) - if lyrFeatDict: - moreThanOneFeat = ( - len(list(lyrFeatDict.values())) > 1 - or len(list(lyrFeatDict.values())[0]) > 1 - ) - if moreThanOneFeat: - # if there are overlapping features (valid candidates only) - ( - selectedFeaturesDict, - notSelectedFeaturesDict, - ) = self.checkSelectedFeaturesOnDict(menuDict=lyrFeatDict) - self.setContextMenuStyle( - e=e, - dictMenuSelected=selectedFeaturesDict, - dictMenuNotSelected=notSelectedFeaturesDict, - ) + lyrFeatDict[layer] = [feature] else: - layer = list(lyrFeatDict.keys())[0] - feature = lyrFeatDict[layer][0] - selected = QApplication.keyboardModifiers() == Qt.ControlModifier - if e.button() == Qt.LeftButton: - # if feature is selected, we want it to be de-selected - self.setSelectionFeature( - layer=layer, - feature=feature, - selectAll=False, - setActiveLayer=True, - ) - elif selected: - self.iface.setActiveLayer(layer) + if not geom.intersects(searchRect): + continue + if layer in lyrFeatDict: + lyrFeatDict[layer].append(feature) else: - self.iface.openFeatureForm(layer, feature, showModal=False) - # elif rasters and e.button() == Qt.LeftButton: - # self.openRastersMenu(e, rasters) - - def openRastersMenu(self, e, rasters): - menu = QMenu() - self.addRasterMenu(menu, rasters) - menu.exec_(self.canvas.viewport().mapToGlobal(e.pos())) + lyrFeatDict[layer] = [feature] + lyrFeatDict = self.filterStrongestGeometry(lyrFeatDict) + if not lyrFeatDict: + return + moreThanOneFeat = ( + len(list(lyrFeatDict.values())) > 1 + or len(list(lyrFeatDict.values())[0]) > 1 + ) + if moreThanOneFeat: + # if there are overlapping features (valid candidates only) + ( + selectedFeaturesDict, + notSelectedFeaturesDict, + ) = self.checkSelectedFeaturesOnDict(menuDict=lyrFeatDict) + self.setContextMenuStyle( + e=e, + dictMenuSelected=selectedFeaturesDict, + dictMenuNotSelected=notSelectedFeaturesDict, + ) + else: + layer = list(lyrFeatDict.keys())[0] + feature = lyrFeatDict[layer][0] + selected = QApplication.keyboardModifiers() == Qt.ControlModifier + if e.button() == Qt.LeftButton: + # if feature is selected, we want it to be de-selected + self.setSelectionFeature( + layer=layer, + feature=feature, + selectAll=False, + setActiveLayer=True, + ) + elif selected: + self.iface.setActiveLayer(layer) + else: + self.iface.openFeatureForm(layer, feature, showModal=False) def unload(self): self.deactivate() diff --git a/DsgTools/gui/ProductionTools/MapTools/SelectRasterTool/selectRaster.py b/DsgTools/gui/ProductionTools/MapTools/SelectRasterTool/selectRaster.py index 46e7c55c6..877773b33 100644 --- a/DsgTools/gui/ProductionTools/MapTools/SelectRasterTool/selectRaster.py +++ b/DsgTools/gui/ProductionTools/MapTools/SelectRasterTool/selectRaster.py @@ -53,26 +53,6 @@ def setAction(self, action): self.toolAction = action self.toolAction.setCheckable(True) - # def activate(self): - # """ - # Activate tool. - # """ - # if self.toolAction: - # self.toolAction.setChecked(True) - # QgsMapTool.activate(self) - - # def deactivate(self): - # """ - # Deactivate tool. - # """ - # try: - # if self.toolAction: - # self.toolAction.setChecked(False) - # if self is not None: - # QgsMapTool.deactivate(self) - # except: - # pass - def canvasPressEvent(self, e): self.run() @@ -113,7 +93,10 @@ def addRasterMenu(self, menu, rasters): for raster in rasters: action = rasterMenu.addAction(raster.name()) action.triggered.connect(lambda b, raster=raster: self.selectOnly(raster)) - # menu.addMenu(rasterMenu) + dummyAction = rasterMenu.addAction("") + dummyAction.setSeparator(True) + action = rasterMenu.addAction(self.tr("Deselect all rasters")) + action.triggered.connect(lambda x: self.selectAll(visible=False)) def selectOnly(self, raster): for otherRaster in self.rasters: @@ -121,6 +104,13 @@ def selectOnly(self, raster): otherRaster, otherRaster.id() == raster.id() ) self.toolAction.setChecked(False) + + def selectAll(self, visible=True): + for otherRaster in self.rasters: + self.iface.layerTreeView().setLayerVisible( + otherRaster, visible + ) + self.toolAction.setChecked(False) def unload(self): self.deactivate() diff --git a/DsgTools/metadata.txt b/DsgTools/metadata.txt index 260b1673f..e75fe667c 100644 --- a/DsgTools/metadata.txt +++ b/DsgTools/metadata.txt @@ -10,7 +10,7 @@ name=DSG Tools qgisMinimumVersion=3.22 description=Brazilian Army Cartographic Production Tools -version=4.7.0 +version=4.8.0 author=Brazilian Army Geographic Service email=suporte.dsgtools@dsg.eb.mil.br about= @@ -44,6 +44,42 @@ about= # Uncomment the following line and add your changelog: changelog= + 4.7.1: + + 4.7.0: + Novas funcionalidades: + - Novo processo de selecionar feições no canvas de camadas selecionadas; + - Novo processo de filtrar lista de camadas no processing por tipo geométrico; + - Novo processo de remover holes pequenos de camadas de cobertura; + - Novo processo de dissolver polígonos para vizinhos (heurística pelo maior comprimento da intersecção); + - Novo processo de construir grid de pontos dentro de polígonos; + - Novo processo de dividir polígonos; + - Novo processo de dividir polígonos por grid; + - Novo processo de selecionar por DE9IM; + - Novo processo de extrair feições por DE9IM; + - Processo de converter linha para multilinha portado do ferramentas experimentais; + + Melhorias: + - Adicionada a opção de dar pan na barra de ferramentas de revisão; + - Adicionada mudanca de ferramenta atual nos icones das ferramentas de filtro; + - Processing de construção do diagrama de elevação portado para o Ferramentas de Edição; + - Adicionado o comportamento no seletor genérico de selecionar somente na camada ativa quando a tecla Alt estiver selecionada; + - Adicionada a opção de rodar a construção de polígonos por polígono de área geográfica (por MI); + - Melhoria de desempenho na construção de polígonos (adicionado paralelismo em thread); + - Melhoria de desempenho na verificação de delimitadores não utilizados no processo de construção de polígonos; + - Adicionada a opção de verificar ou não delimitadores não utilizados no processo de construção de polígonos; + - Melhoria de desempenho na identificação de erros de construção do terreno (roda em thread por área geográfica); + - A ferramenta de verificação de erros de relacionamentos espaciais agora permite regras com de9im e relacionamentos espaciais simultaneamente; + - Adicionada a opção de desligar todas as imagens ativas na ferramenta de seleção de raster; + - Adicionado o id da geometria na flag do identificar geometrias inválidas; + - O menu de aquisição agora permite reclassificação de polígono para ponto (particularmente útil quando se está corrigindo flags de áreas sem centroide na construção de polígonos utilizando linha e centroide); + + + Correção de bug: + - Corrigido o bug de sempre apontar flags quando a geometria tem buraco do processo de identificar geometrias com densidade incorreta de vértices; + - Correção de bug no processo de adicionar vértice em segmento compartilhado; + - Correção de bug no processo de dissolver polígonos com mesmo conjunto de atributos quando é passada uma área mínima para o dissolve; + - Correção de bug no acesso ao BDGEx (a url do serviço mudou e o código teve de ser atualizado, mudando a url do serviço de https para http); 4.6.0: Novas funcionalidades: - Novo processo de estender linhas próximas da moldura;