Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report hiearchical organization of vector layers #3762

Merged
merged 2 commits into from May 4, 2021

Conversation

rouault
Copy link
Member

@rouault rouault commented Apr 29, 2021

  • Add C++, C, SWIG API to explore hiearchical organization of vector layers
  • Implement it in FileGDB and OpenFileGDB drivers
  • Make "ogrinfo my.gdb" use that
$ ogrinfo ../autotest/ogr/data/filegdb/featuredataset.gdb
INFO: Open of `../autotest/ogr/data/filegdb/featuredataset.gdb'
      using driver `OpenFileGDB' successful.
Layer: standalone (Point)
Group fd1:
  Layer: fd1_lyr1 (Point)
  Layer: fd1_lyr2 (Point)
Group fd2:
  Layer: fd2_lyr (Point)

@rouault rouault added this to the 3.4.0 milestone Apr 29, 2021
@rouault
Copy link
Member Author

rouault commented Apr 29, 2021

CC @nyalldawson

@nyalldawson
Copy link
Collaborator

Make "ogrinfo my.gdb" use that

Nice enhancement!

@nyalldawson
Copy link
Collaborator

One question -- if we add support in future for raster layers with a hierarchy, would we then add GDALGroupGetRasterLayerNames/ etc ?

…and implement it in FileGDB/OpenFileGDB drivers

Add GDALGroup::GetVectorLayerNames() and OpenVectorLayer(), and
corresponding C and SWIG API

Fixes https://trac.osgeo.org/gdal/ticket/5752
```
$ ogrinfo ../autotest/ogr/data/filegdb/featuredataset.gdb
INFO: Open of `../autotest/ogr/data/filegdb/featuredataset.gdb'
      using driver `OpenFileGDB' successful.
Layer: standalone (Point)
Group fd1:
  Layer: fd1_lyr1 (Point)
  Layer: fd1_lyr2 (Point)
Group fd2:
  Layer: fd2_lyr (Point)
```
@rouault
Copy link
Member Author

rouault commented Apr 30, 2021

if we add support in future for raster layers with a hierarchy, would we then add GDALGroupGetRasterLayerNames/ etc ?

good question. For the very few drivers that support the multidimensional raster API, there is GDALGroupOpenMDArray() which returns a GDALMDArray which is a sort-of generalization of a GDALDataset to multidimensional (most things have an equivalent, but not always in a straightforward way). This is used for netCDF, HDF4, HDF5 and GRIB (and Zarr at some point). Hiearchical netCDF and HDF5 are represented like that. I can't really think of formats that would be 2D only and have a hierarchical organization, except some formulations of KML. Or possibly GeoPackage by adding an extension that would describe hiearchy.

@Saijin-Naib
Copy link
Sponsor

Even, this is massive! Thanks so much for this!

@nyalldawson
Copy link
Collaborator

@Saijin-Naib
you may be able to assist here -- I was experimenting in ArcMap to see if it's possible to put a raster layer inside a specific dataset in a gdb, and I couldn't find anyway to do this. Do you know if this is supported at all?

@nyalldawson
Copy link
Collaborator

@rouault

I can't really think of formats that would be 2D only and have a hierarchical organization, except some formulations of KML. Or possibly GeoPackage by adding an extension that would describe hiearchy.

Fair enough. I tried experimenting to see if it's supported in gdb, but couldn't find any way to do this (see comment above), so I suspect it's not supported in geodatabases either.

@Saijin-Naib
Copy link
Sponsor

Saijin-Naib commented May 2, 2021

@Saijin-Naib
you may be able to assist here -- I was experimenting in ArcMap to see if it's possible to put a raster layer inside a specific dataset in a gdb, and I couldn't find anyway to do this. Do you know if this is supported at all?

So, in FileGeoDataBase land, you want a Raster stored in a Feature Class? I think Feature Class can only hold vector, but there is the concept of Raster Catalogs, which may be a similar hierarchical structure in FileGeoDataBase.

Don't let me go more than Wednesday without giving you data and answers.

@nyalldawson
Copy link
Collaborator

the concept of Raster Catalogs

I believe they are closer in concept to a multidimensional raster array (mentioned earlier in this thread), but I've never used one personally to know for sure

@Saijin-Naib
Copy link
Sponsor

GitHub_Test.gdb.zip
I'm not sure if this helps test/verify any.

I created a Raster Catalog, which doesn't allow for any hierarchical organization within itself, but it does present itself as a branch in the hierarchy of the FileGeoDataBase.
image

@rouault
Copy link
Member Author

rouault commented May 4, 2021

Hand-opening the .gdbtable file corresponding to the raster catalog, one can see a flat list of a few raster in it:

$ ogrinfo GitHub_Test.gdb/a0000001c.gdbtable -al -ro -q

Layer name: Raster_Catalog_Test
OGRFeature(Raster_Catalog_Test):8
  Name (String) = raster.tif
  Shape_Length (Real) = 0.00656165927648544
  Shape_Area (Real) = 2.5833213698756e-06
  MULTIPOLYGON (((-5.69537067040801 40.2483222410083,-5.69537067040801 40.2496345713735,-5.69340217113495 40.2496345713735,-5.69340217113495 40.2483222410083,-5.69537067040801 40.2483222410083)))

OGRFeature(Raster_Catalog_Test):9
  Name (String) = raster_1class.tif
  Shape_Length (Real) = 0.56370166584281
  Shape_Area (Real) = 0.0180237197418431
  MULTIPOLYGON (((-1.6943944208324 47.1502113193274,-1.70292458683252 47.2483082413673,-1.51952600479126 47.255526073277,-1.51099583879113 47.1577572375536,-1.6943944208324 47.1502113193274)))

OGRFeature(Raster_Catalog_Test):10
  Name (String) = raster_4class.tif
  Shape_Length (Real) = 0.56370166584281
  Shape_Area (Real) = 0.0180237197418431
  MULTIPOLYGON (((-1.6943944208324 47.1502113193274,-1.70292458683252 47.2483082413673,-1.51952600479126 47.255526073277,-1.51099583879113 47.1577572375536,-1.6943944208324 47.1502113193274)))

OGRFeature(Raster_Catalog_Test):11
  Name (String) = raster_5class.tif
  Shape_Length (Real) = 0.56370166584281
  Shape_Area (Real) = 0.0180237197418431
  MULTIPOLYGON (((-1.6943944208324 47.1502113193274,-1.70292458683252 47.2483082413673,-1.51952600479126 47.255526073277,-1.51099583879113 47.1577572375536,-1.6943944208324 47.1502113193274)))

OGRFeature(Raster_Catalog_Test):12
  Name (String) = raster_6class.tif
  Shape_Length (Real) = 0.56370166584281
  Shape_Area (Real) = 0.0180237197418431
  MULTIPOLYGON (((-1.6943944208324 47.1502113193274,-1.70292458683252 47.2483082413673,-1.51952600479126 47.255526073277,-1.51099583879113 47.1577572375536,-1.6943944208324 47.1502113193274)))

Anyway, raster-in-gdb is something for later...

@rouault rouault merged commit 2a6e127 into OSGeo:master May 4, 2021
@nyalldawson
Copy link
Collaborator

Hey @Saijin-Naib what is "parcel_fabric_test"? Is that some special kind of layer?

@Saijin-Naib
Copy link
Sponsor

Saijin-Naib commented May 4, 2021

Hey @Saijin-Naib what is "parcel_fabric_test"? Is that some special kind of layer?

Yeah, it's one of the number of special data types that can be stored only in Feature Dataset. If wanted, I'd like to enumerate a test dataset full of those special types.

I included it here since it also presents itself as a branch in the hierarchical organization of the FileGeoDataBase

@rouault
Copy link
Member Author

rouault commented May 4, 2021

what is "parcel_fabric_test"? Is that some special kind of layer?

A "DECadastralFabric":

OGRFeature(GDB_Items):10
  UUID (String) = {A59D3CEA-9925-4D49-BBC8-401012D361C3}
  Type (String) = {A3803369-5FC2-4963-BAE0-13EFFC09DD73}
  Name (String) = Parcel_Fabric_Test
  PhysicalName (String) = PARCEL_FABRIC_TEST
  Path (String) = \Feature_Dataset_Test\Parcel_Fabric_Test
  DatasetSubtype1 (Integer) = (null)
  DatasetSubtype2 (Integer) = (null)
  DatasetInfo1 (String) = (null)
  DatasetInfo2 (String) = (null)
  URL (String) = 
  Definition (String) = <DECadastralFabric xsi:type='typens:DECadastralFabric' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:xs='http://www.w3.org/2001/XMLSchema' xmlns:typens='http://www.esri.com/schemas/ArcGIS/10.7'><CatalogPath>\Feature_Dataset_Test\Parcel_Fabric_Test</CatalogPath><Name>Parcel_Fabric_Test</Name><ChildrenExpanded>false</ChildrenExpanded><DatasetType>esriDTCadastralFabric</DatasetType><DSID>10</DSID><Versioned>false</Versioned><CanVersion>false</CanVersion><ConfigurationKeyword>DEFAULTS</ConfigurationKeyword><RequiredGeodatabaseClientVersion>10.1</RequiredGeodatabaseClientVersion><Extent xsi:type='typens:EnvelopeN'><XMin>NaN</XMin><YMin>NaN</YMin><XMax>NaN</XMax><YMax>NaN</YMax><SpatialReference xsi:type='typens:ProjectedCoordinateSystem'><WKT>PROJCS[&quot;NAD_1983_2011_StatePlane_New_York_West_FIPS_3103_Ft_US&quot;,GEOGCS[&quot;GCS_NAD_1983_2011&quot;,DATUM[&quot;D_NAD_1983_2011&quot;,SPHEROID[&quot;GRS_1980&quot;,6378137.0,298.257222101]],PRIMEM[&quot;Greenwich&quot;,0.0],UNIT[&quot;Degree&quot;,0.0174532925199433]],PROJECTION[&quot;Transverse_Mercator&quot;],PARAMETER[&quot;False_Easting&quot;,1148291.666666667],PARAMETER[&quot;False_Northing&quot;,0.0],PARAMETER[&quot;Central_Meridian&quot;,-78.58333333333333],PARAMETER[&quot;Scale_Factor&quot;,0.9999375],PARAMETER[&quot;Latitude_Of_Origin&quot;,40.0],UNIT[&quot;Foot_US&quot;,0.3048006096012192],AUTHORITY[&quot;EPSG&quot;,6541]]</WKT><XOrigin>-17299100</XOrigin><YOrigin>-47344700</YOrigin><XYScale>3048.0060960121928</XYScale><ZOrigin>-100000</ZOrigin><ZScale>10000</ZScale><MOrigin>-100000</MOrigin><MScale>10000</MScale><XYTolerance>0.0032808333333333331</XYTolerance><ZTolerance>0.001</ZTolerance><MTolerance>0.001</MTolerance><HighPrecision>true</HighPrecision><WKID>103119</WKID><LatestWKID>6541</LatestWKID></SpatialReference></Extent><SpatialReference xsi:type='typens:ProjectedCoordinateSystem'><WKT>PROJCS[&quot;NAD_1983_2011_StatePlane_New_York_West_FIPS_3103_Ft_US&quot;,GEOGCS[&quot;GCS_NAD_1983_2011&quot;,DATUM[&quot;D_NAD_1983_2011&quot;,SPHEROID[&quot;GRS_1980&quot;,6378137.0,298.257222101]],PRIMEM[&quot;Greenwich&quot;,0.0],UNIT[&quot;Degree&quot;,0.0174532925199433]],PROJECTION[&quot;Transverse_Mercator&quot;],PARAMETER[&quot;False_Easting&quot;,1148291.666666667],PARAMETER[&quot;False_Northing&quot;,0.0],PARAMETER[&quot;Central_Meridian&quot;,-78.58333333333333],PARAMETER[&quot;Scale_Factor&quot;,0.9999375],PARAMETER[&quot;Latitude_Of_Origin&quot;,40.0],UNIT[&quot;Foot_US&quot;,0.3048006096012192],AUTHORITY[&quot;EPSG&quot;,6541]]</WKT><XOrigin>-17299100</XOrigin><YOrigin>-47344700</YOrigin><XYScale>3048.0060960121928</XYScale><ZOrigin>-100000</ZOrigin><ZScale>10000</ZScale><MOrigin>-100000</MOrigin><MScale>10000</MScale><XYTolerance>0.0032808333333333331</XYTolerance><ZTolerance>0.001</ZTolerance><MTolerance>0.001</MTolerance><HighPrecision>true</HighPrecision><WKID>103119</WKID><LatestWKID>6541</LatestWKID></SpatialReference><ConfigurationKeyword>DEFAULTS</ConfigurationKeyword><DefaultAccuracy>4</DefaultAccuracy><CadastralFieldEditsCollection xsi:type='typens:ArrayOfCadastralFieldEdits'></CadastralFieldEditsCollection><UseDefaultAccuracies>true</UseDefaultAccuracies><CompiledAccuracy>3</CompiledAccuracy><BufferDistanceForAdjustment>50</BufferDistanceForAdjustment><CadastralFabricType>0</CadastralFabricType><SurrogateVersion></SurrogateVersion><Version>3</Version><MaximumShiftThreshold>0.0099974599949199904</MaximumShiftThreshold><MultiGenerationEditing>false</MultiGenerationEditing><MultiLevelReconcile>false</MultiLevelReconcile><PinAdjustmentBoundary>true</PinAdjustmentBoundary><PinPointsWithinBoundary>true</PinPointsWithinBoundary><MinorVersion>0</MinorVersion><WriteAdjustmentVectors>true</WriteAdjustmentVectors><SysTableFieldDomainMap xsi:type='typens:Names'></SysTableFieldDomainMap><ArrayOfPropertySet xsi:type='typens:ArrayOfPropertySet'></ArrayOfPropertySet><PropsUserDefined xsi:type='typens:PropertySet'><PropertyArray xsi:type='typens:ArrayOfPropertySetProperty'></PropertyArray></PropsUserDefined><PropsImporterLoading xsi:type='typens:PropertySet'><PropertyArray xsi:type='typens:ArrayOfPropertySetProperty'><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>esriStoreCOGOShape</Key><Value xsi:type='xs:boolean'>true</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>esriAssignAccCat7ToRadialLinesGreaterThan</Key><Value xsi:type='xs:double'>10</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>esriDataLoadCurveDetectOffsetDefault</Key><Value xsi:type='xs:double'>0.10000000000000001</Value></PropertySetProperty></PropertyArray></PropsImporterLoading><PropsCoordTolerances xsi:type='typens:PropertySet'><PropertyArray xsi:type='typens:ArrayOfPropertySetProperty'><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>esriLowerLineCrackingOffset</Key><Value xsi:type='xs:double'>0.5</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>esriMaximumAllowableLineCrackingOffset</Key><Value xsi:type='xs:double'>99.999989839979676</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>esriRadialCrackingTolerance</Key><Value xsi:type='xs:double'>0.10000508001016002</Value></PropertySetProperty></PropertyArray></PropsCoordTolerances><PropsEditSettings xsi:type='typens:PropertySet'><PropertyArray xsi:type='typens:ArrayOfPropertySetProperty'><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>esriEditingWithLocks</Key><Value xsi:type='xs:boolean'>true</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>Solution Enabled</Key><Value xsi:type='xs:int'>1</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>Solution Version</Key><Value xsi:type='xs:int'>101</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>Solution Name</Key><Value xsi:type='xs:string'>Local Government Information Model</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>Solution Lines Anno Class</Key><Value xsi:type='xs:int'>56</Value></PropertySetProperty><PropertySetProperty xsi:type='typens:PropertySetProperty'><Key>Solution Tax Parcel Condo Class</Key><Value xsi:type='xs:int'>50</Value></PropertySetProperty></PropertyArray></PropsEditSettings><PropsCatalogDataset xsi:type='typens:PropertySet'><PropertyArray xsi:type='typens:ArrayOfPropertySetProperty'></PropertyArray></PropsCatalogDataset></DECadastralFabric>
  Documentation (String) = (null)
  ItemInfo (String) = (null)
  Properties (Integer) = 1
  Defaults (Binary) = (null)
  MULTIPOLYGON EMPTY

Interestingly the layers that in the screenshop appear under it have a path like \Feature_Dataset_Test\Parcel_Fabric_Test_Control , which is at the same level as the Parcel_Fabric_Test (\Feature_Dataset_Test\Parcel_Fabric_Test). I guess that ArcMap has a logic based on the fact that \Feature_Dataset_Test\Parcel_Fabric_Test is a prefix of the path of those layers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants