Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add arc_meta() convenience function for accessing metadata #163

Closed
elipousson opened this issue Mar 6, 2024 · 2 comments
Closed

Add arc_meta() convenience function for accessing metadata #163

elipousson opened this issue Mar 6, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@elipousson
Copy link
Contributor

Is your feature request related to a problem? Please describe.

Add a function, potentially named arc_meta(), that wraps arcgisutils::fetch_layer_metadata() to allow easy access to service, layer, and folder metadata.

At present, folder URLs will encounter an error when supplied to arc_open(). See this reprex: https://gist.github.com/elipousson/bcc6420a086a33cdc87725a33d16d031

Describe the solution you'd like

Below is a reprex showing a basic example. A more complete function should include URL validation (or this could live in the fetch_layer_metadata function) and potentially could include support for ArcGIS item-level metadata which is accessible at a different API end-point (using the Content API, I think).

library(arcgisutils)

arc_meta <- function(url,
                     token = NULL,
                     ...) {
  meta <- arcgisutils::fetch_layer_metadata(
    url = url,
    token = token,
    ...
  )

  if (hasName(meta, "editingInfo")) {
    meta[["editingInfo"]] <- lapply(
      meta[["editingInfo"]],
      arcgisutils::from_esri_date
    )
  }

  meta
}

# Get metadata for FeatureServer
url <- "https://geodata.md.gov/imap/rest/services/Transportation/MD_Transit/FeatureServer/9"
arc_meta(url)
#> $currentVersion
#> [1] 10.71
#> 
#> $id
#> [1] 9
#> 
#> $name
#> [1] "MTA Bus Stops"
#> 
#> $type
#> [1] "Feature Layer"
#> 
#> $parentLayer
#> NULL
#> 
#> $defaultVisibility
#> [1] TRUE
#> 
#> $minScale
#> [1] 36000
#> 
#> $maxScale
#> [1] 0
#> 
#> $geometryType
#> [1] "esriGeometryPoint"
#> 
#> $description
#> [1] "Maryland Department of Transportation's Maryland Transit Administration Fall 2023 Bus Stops including CityLink, LocalLink, QuickLink, Express BusLink, and Commuter Bus.  Ridership data is based on Automatic Passenger Counting (APC) system average daily weekday bus stop ridership (boarding, alighting, and total) from the Summer 2023 period and does not exclude outliers.  For service change details please see: https://www.mta.maryland.gov/servicechanges/fall2023"
#> 
#> $copyrightText
#> [1] "MD iMAP, MDOT MTA"
#> 
#> $editFieldsInfo
#> NULL
#> 
#> $ownershipBasedAccessControlForFeatures
#> NULL
#> 
#> $syncCanReturnChanges
#> [1] FALSE
#> 
#> $relationships
#> NULL
#> 
#> $isDataVersioned
#> [1] FALSE
#> 
#> $isDataArchived
#> [1] FALSE
#> 
#> $isCoGoEnabled
#> [1] FALSE
#> 
#> $supportsRollbackOnFailureParameter
#> [1] TRUE
#> 
#> $archivingInfo
#> $archivingInfo$supportsQueryWithHistoricMoment
#> [1] FALSE
#> 
#> $archivingInfo$startArchivingMoment
#> [1] -1
#> 
#> 
#> $supportsStatistics
#> [1] TRUE
#> 
#> $supportsAdvancedQueries
#> [1] TRUE
#> 
#> $supportsValidateSQL
#> [1] TRUE
#> 
#> $supportsCoordinatesQuantization
#> [1] TRUE
#> 
#> $supportsCalculate
#> [1] TRUE
#> 
#> $advancedQueryCapabilities
#> $advancedQueryCapabilities$supportsPagination
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsTrueCurve
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsQueryWithDistance
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsReturningQueryExtent
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsStatistics
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsHavingClause
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsOrderBy
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsDistinct
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsCountDistinct
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsQueryWithResultType
#> [1] TRUE
#> 
#> $advancedQueryCapabilities$supportsReturningGeometryCentroid
#> [1] FALSE
#> 
#> $advancedQueryCapabilities$supportsSqlExpression
#> [1] TRUE
#> 
#> 
#> $extent
#> $extent$xmin
#> [1] -8805437
#> 
#> $extent$ymin
#> [1] 4623813
#> 
#> $extent$xmax
#> [1] -8360390
#> 
#> $extent$ymax
#> [1] 4827901
#> 
#> $extent$spatialReference
#> $extent$spatialReference$wkid
#> [1] 102100
#> 
#> $extent$spatialReference$latestWkid
#> [1] 3857
#> 
#> 
#> 
#> $sourceSpatialReference
#> $sourceSpatialReference$wkid
#> [1] 102100
#> 
#> $sourceSpatialReference$latestWkid
#> [1] 3857
#> 
#> 
#> $drawingInfo
#> $drawingInfo$renderer
#> $drawingInfo$renderer$type
#> [1] "uniqueValue"
#> 
#> $drawingInfo$renderer$field1
#> [1] "Mode"
#> 
#> $drawingInfo$renderer$field2
#> NULL
#> 
#> $drawingInfo$renderer$field3
#> NULL
#> 
#> $drawingInfo$renderer$defaultSymbol
#> NULL
#> 
#> $drawingInfo$renderer$defaultLabel
#> NULL
#> 
#> $drawingInfo$renderer$uniqueValueInfos
#>                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    symbol
#> 1         esriPMS, 03d879d22563401ffbe4b42a57515f98, iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0NAAAAAXNSR0IB2cksfwAAAAlwSFlzAAAOxAAADsQBlSsOGwAAAfdJREFUOI2t0kHLKVEYB/D/MOo1myEaJi5pbJREWYgsLJS6ShYk38FW3Y1SlC9gIRs2FEuvtY2lxGVFJNdCCqVJIrkr7uvOmIV7/6vTc8759TydQ+I/h7wvJpPJ52q1cryDEARx4ziuZjKZfjzAfr//PZFIvN1Zp9NJAvgD3m63x2Y2m0UkEsFwOMRutxNctlgsMBgMKBaLqFQq9/sE8GXke/x+P9LpND4+PtBsNpHP5wVgo9GA2+1GLpd7gPcIwFAohM1mg/1+D4/Hg0qlgsPhgHA4jHa7DZqmoVKpMBqNoNPp4HQ6MRgMXoNmsxmZTAbVavVRY1kW8XgcqVTq6WypVEIgEJAGGYZBq9USjCmW2WwGq9UqPTJBEKIPIZZerwefzycNMgyDRqPxVJPJZKAoSlAHALVaLQ1yHAeHQ/x/x2IxQW06nUqDl8sF3W4Xp9MJXq8XFEUJkPV6jfF4DJZln/6vKMjzPBiGwfl8hkKhEO2Uoijo9XrQNI3lcikN0jSN6/UKuVwuOPw1SqUSJElCq9VKg7PZDHa7HSRJYj6f43g8CjCNRgOWZcHzPMbjsTTocrmw3W5BURTK5TIKhYIArNVqiEajWCwWCAaD0uC9A6kkk8mXew/QZrP9qtfr3ySlFyEIAkaj8ecT6HQ6Te9gf0d05H/Jb7jBpZyr1ZWwAAAAAElFTkSuQmCC, image/png, 15, 15, 0, 0, 0
#> 2 esriPMS, c0b2ad80a8a7c297bd5ca21280288413, iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0NAAAAAXNSR0IB2cksfwAAAAlwSFlzAAAOxAAADsQBlSsOGwAAAf1JREFUOI2t0r/q2lAcBfATEgwBo5j8jIooDi6KNIg+QZwks0snHYL4AkLBZ3DwASRT0cml4OgiDrrVIQZUlEYFcRDxD6JiJ21tNBTbM4Xzzf3ce+FS+M+hbh/j8fjbYrH49KZzDYVCX/1+/5c72O/35Wq1+vbJisXiZwC/wOv1eh9KkoRUKgVN07DZbEyL/X4/vF4varUaNE271QTw25Vv4TgO+XweNE2j2Wyi3W6bQEVRIIoiBEFAoVB4mJnARCKB1WqF9XoNURQRDAax3+8hSRJarRbsdjtYloWmaXC73bDb7dhut69Bn8+HSqUCXdfvnc1mgyzLaDQaD/9mMhlEIhH0er3XIM/zGI1Gpms+i2EYCAQC1iBBEDifz38FDodDxOPxh84Efnx8QFEU0yYMw5h6AHA6ndZgKBRCNBp9eiJZlk3deDy2Bk+nE7rdLo7HI5LJJBiGMSHL5RK6rkMQhIf3+xTc7XbgeR7n8xkUZRoDABiGgSAIYFkW8/ncGnQ4HLhcLiBJErPZ7CkIADRNg6IocBxnDU4mE0QiEZAkiel0isPhYMJcLhc8Hg92ux0Gg4E1WCqVoKoqGIZBvV5Hp9MxgdlsFul0GoZhoFwuW4O3BVZRVRWqqj6d3cFwOPwjl8sFLKUXIQgCPp/v+wMYi8WC72B/5vm7+If8BDOWowt7xLReAAAAAElFTkSuQmCC, image/png, 15, 15, 0, 0, 0
#> 3                 esriPMS, 7bb1ae019939ceeea58429b0f479d658, iVBORw0KGgoAAAANSUhEUgAAABQAAAAUCAYAAACNiR0NAAAAAXNSR0IB2cksfwAAAAlwSFlzAAAOxAAADsQBlSsOGwAAAe9JREFUOI2t07vq2mAYBvBHIzESiQlJRNEWo6Lg0EFxEZTsRXDuTTj3Mty8gro61Fkxm4NDcfQQDwQVD4uiCQl20irJPxTbZwrPG375Pnjjw3+O7/6wXC5/7na7L286t1gs9iMcDn9/gJPJ5Gu/33/7ZNVq9RuAP+BzstksCoUCVFXF+Xy2zQVBgCAI6PV62Gw299oDPF35nkAggFqtBpIkMRgMMB6PbaAsy8hkMuA4Ds1m82VmAxOJBI7HI06nEyRJgiiK0HUd+Xwew+EQFEWBpmnMZjNwHAeSJGEYxscgz/Not9vYbrePjiAIlEolDIfDl3eLxSIikQgWi8XHIMMw2O/3tms6ZbfbQRRFd9Dj8cCyrL8Ct9stUqnUS2cDWZaFLMu2j/j9flsPAMFg0B2MRqNIJpOOJyqXy7ZutVq5g6ZpYjQawTAM5HI5UBRlQw6HA+bzOTiOw+12cwev1ytCoRBM04TP57j38Pv9YFkWNE2/bIMjSNM0LMuC1+t9/gtsIUkSBEGAYRh3UNM0SJIEgiCgaRp0XbdhDMOA53lcLheoquoOtlot1Ot1UBSFbreL6XRqA8vlMkqlEtbrNTqdjjsIAI1Gw6l+RFEUKIriOHuA8Xh8WalUPrlKLhEE4dcLmE6nP7+LPcd5L/4hvwGElaBil1RkFAAAAABJRU5ErkJggg==, image/png, 15, 15, 0, 0, 0
#>              value            label description
#> 1              Bus              Bus            
#> 2     Commuter Bus     Commuter Bus            
#> 3 Bus/Commuter Bus Bus/Commuter Bus            
#> 
#> $drawingInfo$renderer$fieldDelimiter
#> [1] ","
#> 
#> 
#> $drawingInfo$transparency
#> [1] 0
#> 
#> $drawingInfo$labelingInfo
#>                            labelPlacement where labelExpression useCodedValues
#> 1 esriServerPointLabelPlacementAboveRight    NA     [stop_name]           TRUE
#>                                                                                       symbol
#> 1 esriTS, 0, 0, 0, 255, baseline, left, FALSE, 0, 0, 0, TRUE, Arial, 8, normal, normal, none
#>   minScale maxScale
#> 1    10000        0
#> 
#> 
#> $hasM
#> [1] FALSE
#> 
#> $hasZ
#> [1] FALSE
#> 
#> $allowGeometryUpdates
#> [1] FALSE
#> 
#> $allowTrueCurvesUpdates
#> [1] FALSE
#> 
#> $onlyAllowTrueCurveUpdatesByTrueCurveClients
#> [1] FALSE
#> 
#> $hasAttachments
#> [1] FALSE
#> 
#> $supportsApplyEditsWithGlobalIds
#> [1] FALSE
#> 
#> $htmlPopupType
#> [1] "esriServerHTMLPopupTypeAsHTMLText"
#> 
#> $objectIdField
#> [1] "OBJECTID"
#> 
#> $globalIdField
#> [1] ""
#> 
#> $displayField
#> [1] "stop_name"
#> 
#> $typeIdField
#> [1] "Mode"
#> 
#> $subtypeField
#> [1] ""
#> 
#> $fields
#>                   name                      type                      alias
#> 1             OBJECTID          esriFieldTypeOID                        FID
#> 2            stop_name       esriFieldTypeString                  stop_name
#> 3             Rider_On esriFieldTypeSmallInteger  Number of Riders Boarding
#> 4            Rider_Off esriFieldTypeSmallInteger Number of Riders Alighting
#> 5          Rider_Total esriFieldTypeSmallInteger     Total Number of Riders
#> 6  Stop_Ridership_Rank esriFieldTypeSmallInteger        Stop Ridership Rank
#> 7        Routes_Served       esriFieldTypeString      Routes Served by Stop
#> 8  Distribution_Policy       esriFieldTypeString        Distribution Policy
#> 9                 Mode       esriFieldTypeString                       Mode
#> 10             Shelter       esriFieldTypeString                    Shelter
#> 11              County       esriFieldTypeString                     County
#> 12             stop_id       esriFieldTypeString                    Stop ID
#>    domain editable nullable defaultValue           modelName length
#> 1      NA    FALSE    FALSE           NA            OBJECTID     NA
#> 2      NA     TRUE     TRUE           NA           stop_name    254
#> 3      NA     TRUE     TRUE           NA            Rider_On     NA
#> 4      NA     TRUE     TRUE           NA           Rider_Off     NA
#> 5      NA     TRUE     TRUE           NA         Rider_Total     NA
#> 6      NA     TRUE     TRUE           NA Stop_Ridership_Rank     NA
#> 7      NA     TRUE     TRUE           NA       Routes_Served    254
#> 8      NA     TRUE     TRUE           NA Distribution_Policy    255
#> 9      NA     TRUE     TRUE           NA                Mode     50
#> 10     NA     TRUE     TRUE           NA             Shelter     10
#> 11     NA     TRUE     TRUE           NA              County     60
#> 12     NA     TRUE     TRUE           NA             stop_id     50
#> 
#> $geometryField
#> $geometryField$name
#> [1] "Shape"
#> 
#> $geometryField$type
#> [1] "esriFieldTypeGeometry"
#> 
#> $geometryField$alias
#> [1] "Shape"
#> 
#> $geometryField$domain
#> NULL
#> 
#> $geometryField$editable
#> [1] FALSE
#> 
#> $geometryField$nullable
#> [1] TRUE
#> 
#> $geometryField$defaultValue
#> NULL
#> 
#> $geometryField$modelName
#> [1] "Shape"
#> 
#> 
#> $indexes
#>        name   fields isAscending isUnique description
#> 1  R1224_pk OBJECTID        TRUE     TRUE            
#> 2 S1135_idx    Shape        TRUE     TRUE            
#> 
#> $dateFieldsTimeReference
#> $dateFieldsTimeReference$timeZone
#> [1] "UTC"
#> 
#> $dateFieldsTimeReference$respectsDaylightSaving
#> [1] FALSE
#> 
#> 
#> $types
#>                 id             name domains
#> 1              Bus              Bus    NULL
#> 2     Commuter Bus     Commuter Bus    NULL
#> 3 Bus/Commuter Bus Bus/Commuter Bus    NULL
#>                                                                   templates
#> 1                    Bus; Bus/Commuter Bus, , Bus, esriFeatureEditToolPoint
#> 2 Commuter Bus; Commuter Bus/MARC, , Commuter Bus, esriFeatureEditToolPoint
#> 3       Bus; Bus/Commuter Bus, , Bus/Commuter Bus, esriFeatureEditToolPoint
#> 
#> $templates
#> NULL
#> 
#> $maxRecordCount
#> [1] 12500
#> 
#> $supportedQueryFormats
#> [1] "JSON, geoJSON"
#> 
#> $capabilities
#> [1] "Query,Extract"
#> 
#> $useStandardizedQueries
#> [1] TRUE
#> 
#> $standardMaxRecordCount
#> [1] 32000
#> 
#> $tileMaxRecordCount
#> [1] 8000
#> 
#> $maxRecordCountFactor
#> [1] 1

# Gets metadata for folder
folder_url <- "https://geodata.md.gov/imap/rest/services/Transportation/"
arc_meta(folder_url)
#> $currentVersion
#> [1] 10.71
#> 
#> $folders
#> NULL
#> 
#> $services
#>                                                     name          type
#> 1                      Transportation/MD_AlternativeFuel FeatureServer
#> 2                      Transportation/MD_AlternativeFuel     MapServer
#> 3            Transportation/MD_AnnualAverageDailyTraffic FeatureServer
#> 4            Transportation/MD_AnnualAverageDailyTraffic     MapServer
#> 5                       Transportation/MD_FreightNetwork FeatureServer
#> 6                       Transportation/MD_FreightNetwork     MapServer
#> 7   Transportation/MD_HighwayPerformanceMonitoringSystem     MapServer
#> 8                         Transportation/MD_LocalTransit FeatureServer
#> 9                         Transportation/MD_LocalTransit     MapServer
#> 10           Transportation/MD_MDOTSHADistrictBoundaries FeatureServer
#> 11           Transportation/MD_MDOTSHADistrictBoundaries     MapServer
#> 12        Transportation/MD_MDOTSHAParkandRideFacilities FeatureServer
#> 13        Transportation/MD_MDOTSHAParkandRideFacilities     MapServer
#> 14                       Transportation/MD_MVAFacilities FeatureServer
#> 15                       Transportation/MD_MVAFacilities     MapServer
#> 16        Transportation/MD_RoadCenterlinesComprehensive     MapServer
#> 17                     Transportation/MD_RoadCenterlines     MapServer
#> 18 Transportation/MD_RoadwayAdministrativeClassification FeatureServer
#> 19 Transportation/MD_RoadwayAdministrativeClassification     MapServer
#> 20             Transportation/MD_RoadwayInterchangeExits FeatureServer
#> 21             Transportation/MD_RoadwayInterchangeExits     MapServer
#> 22                  Transportation/MD_RoadwayMileMarkers FeatureServer
#> 23                  Transportation/MD_RoadwayMileMarkers     MapServer
#> 24                          Transportation/MD_TollPlazas FeatureServer
#> 25                          Transportation/MD_TollPlazas     MapServer
#> 26                      Transportation/MD_TrafficCameras FeatureServer
#> 27                      Transportation/MD_TrafficCameras     MapServer
#> 28                             Transportation/MD_Transit FeatureServer
#> 29                             Transportation/MD_Transit     MapServer
#> 30            Transportation/MD_WelcomeAndVisitorCenters FeatureServer
#> 31            Transportation/MD_WelcomeAndVisitorCenters     MapServer

Created on 2024-03-06 with reprex v2.1.0

Describe alternatives you've considered

This is currently supported by the esri2sf::esrimeta() function: https://github.com/elipousson/esri2sf/blob/e1b1947e6e4c778ae9b4ef660dafe367aa67d80f/R/esri2sf.R#L352

Additional context

A more general version of this function could retrieve rendering information and field information that could be useful for users. This functionality could be built into arcgisutils per R-ArcGIS/arcgisutils#4 but this proposed function could potentially provide a common higher-level user interface for a variety of metadata-related resources.
@elipousson elipousson added the enhancement New feature or request label Mar 6, 2024
@elipousson elipousson mentioned this issue Mar 14, 2024
Closed
@JosiahParry
Copy link
Collaborator

@elipousson this is identical to unclass(arc_open()) I believe.

EDIT: this is wrong. The difference is that arc_open() will not support folders.

@JosiahParry JosiahParry mentioned this issue Jul 5, 2024
Merged
3 tasks
@JosiahParry
Copy link
Collaborator

Please see reprex below

arcgislayers::arc_open("https://geodata.md.gov/imap/rest/services/Transportation")
#> $currentVersion
#> [1] 10.71
#> 
#> $folders
#> NULL
#> 
#> $services
#>                                                     name          type
#> 1                      Transportation/MD_AlternativeFuel FeatureServer
#> 2                      Transportation/MD_AlternativeFuel     MapServer
#> 3            Transportation/MD_AnnualAverageDailyTraffic FeatureServer
#> 4            Transportation/MD_AnnualAverageDailyTraffic     MapServer
#> 5                       Transportation/MD_FreightNetwork FeatureServer
#> 6                       Transportation/MD_FreightNetwork     MapServer
#> 7   Transportation/MD_HighwayPerformanceMonitoringSystem     MapServer
#> 8                         Transportation/MD_LocalTransit FeatureServer
#> 9                         Transportation/MD_LocalTransit     MapServer
#> 10           Transportation/MD_MDOTSHADistrictBoundaries FeatureServer
#> 11           Transportation/MD_MDOTSHADistrictBoundaries     MapServer
#> 12        Transportation/MD_MDOTSHAParkandRideFacilities FeatureServer
#> 13        Transportation/MD_MDOTSHAParkandRideFacilities     MapServer
#> 14                       Transportation/MD_MVAFacilities FeatureServer
#> 15                       Transportation/MD_MVAFacilities     MapServer
#> 16        Transportation/MD_RoadCenterlinesComprehensive     MapServer
#> 17                     Transportation/MD_RoadCenterlines     MapServer
#> 18 Transportation/MD_RoadwayAdministrativeClassification FeatureServer
#> 19 Transportation/MD_RoadwayAdministrativeClassification     MapServer
#> 20             Transportation/MD_RoadwayInterchangeExits FeatureServer
#> 21             Transportation/MD_RoadwayInterchangeExits     MapServer
#> 22                  Transportation/MD_RoadwayMileMarkers FeatureServer
#> 23                  Transportation/MD_RoadwayMileMarkers     MapServer
#> 24                          Transportation/MD_TollPlazas FeatureServer
#> 25                          Transportation/MD_TollPlazas     MapServer
#> 26                      Transportation/MD_TrafficCameras FeatureServer
#> 27                      Transportation/MD_TrafficCameras     MapServer
#> 28                             Transportation/MD_Transit FeatureServer
#> 29                             Transportation/MD_Transit     MapServer
#> 30            Transportation/MD_WelcomeAndVisitorCenters FeatureServer
#> 31            Transportation/MD_WelcomeAndVisitorCenters     MapServer
#> 
#> $url
#> [1] "https://geodata.md.gov/imap/rest/services/Transportation"

Created on 2024-07-05 with reprex v2.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@elipousson @JosiahParry