Add TrailsAndPathways to SGID

[gregbunce]

Summary

• Proposed Dataset Name: TrailsAndPathways
• Proposed ISO Category: Recreation
• Data Description: Utah Trails and Pathways
• Source Agency/Entity: AGRC
• Contact: Greg Bunce
• Update Frequency: Monthly

Triage

• Data Team Triage Complete
• Dev Team Triage Complete (assigned to)
• Cadastre Team Triage Complete

We are introducing data

The data was or will be added on 2020/06/25 to the following areas

• Internal SGID
• [ ] SGID10
• [ ] ArcGIS Online

The data is high quality

• Sweeper checks have run and passed
• The minimum requirements for metadata are populated
• The data complies with our domain rules

The data should propagate automatically to

• ArcGIS Online
• SGID10
• Open SGID
• Open data
• SGID.META.ChangeDetection

References to the data need to be manually added to

• gis.utah.gov data pages
• Stewardship doc and sgid-index
• SGID.META.AGOLItems

We have verified the data is live

• ArcGIS Online (@ZachBeck on 7/2/2020)
• Internal SGID (@gregbunce on 6/25/2020)
• SGID10 (@gregbunce on 6/26/2020)
• Open SGID (@steveoh on 6/28/2020)
• Open data (@jacobdadams on 7/1/2020)
• gis.utah.gov data pages (@gregbunce on 6/26/2020)
• Stewardship doc and sgid-index (@gregbunce on 6/25/2020)
• SGID.META.AGOLItems (@gregbunce on 6/25/2020)
• SGID.META.ChangeDetection (@steveoh on 6/28/2020)
• agol_validator checks (@jacobdadams on 7/1/2020)

Notification

• Blog post (@gregbunce )
• Twitter (@steveoh)

[ZachBeck]

How set in stone is the name, how about TrailsAndPaths instead?

[gregbunce]

we could to that. I was thinking it will read on the data page and AGOL/OpenData as "Utah Trails and Pathways" or Utah Trails and Paths". Do you want to see the feature class named "TrailsAndPaths"?

[ZachBeck]

I like TrailsAndPathways as the feature class name and Utah Trails and Pathways for AGOL/OpenData

[steveoh]

Nitpick: you will not be adding data to all three places you have checked at the top. You will only add it to the internal sgid. Our tools will propagate those out. I think only internal sgid should be checked.

That might be confusing but we added those three boxes incase you wanted to add something directly to those areas skipping some of the automation.

[gregbunce]

thanks for the nitpicking. ok, that's great! I didn't know if the propagation tools were for data already in place. so, thanks for clarifying. that makes my job much easier.

[steveoh]

Yes so this should be helpful for everyone.

Data in the internal sgid will be added to the sgid10 with swapper

If there is a record in the agol items table it will be added to

• the open sgid with cloudb
• arcgis online with forklift
• open data with the agol validator

Did I miss anything?

[nathankota]

very helpful...a possible addition to the readme?

Yes so this should be helpful for everyone.

Data in the internal sgid will be added to the sgid10 with swapper

If there is a record in the agol items table it will be added to

• the open sgid with cloudb
• arcgis online with forklift
• open data with the agol validator

Did I miss anything?

[gregbunce]

thanks again. yeah, my missing link was for newly added items - and how they propagate, so thanks! when i added the item to the META table, should i leave the item id blank until it gets one assigned?

[steveoh]

when i add the item to the META table, should i leave the item id blank until it gets one assigned?

yes, and the same applies to the stewardship row

[gregbunce]

this layer hasn't propagated to AGOL/OpenData or Open SGID yet and I'm out of the office next week. @ZachBeck any chance you could monitor that and check the verified boxes (above) for the remaining sources? (if yes = thank you!! & if no = you're mean). Just kidding.

Also, you may have to add the AGOL Item ID to the META table once one is created (I added a row to the table, but there's no ItemID yet so it's blank) and the AGOL URL to the stewardship log (again I added a row, but left the AGOL URL field blank).

[gregbunce]

the related blog post is pr 1433

[ZachBeck]

It looks like this has made it's way to the Recreation folder but hasn't been shared out to the Recreation group yet.

[jacobdadams]

I'm waiting to get the Monday morning forklift email, then I'll run the validator that will share it out to the proper groups.

…

On Mon, Jun 29, 2020 at 8:39 AM Zach Beck ***@***.***> wrote: It looks like this has made it's way to the Recreation folder but hasn't been shared out to the Recreation group yet. Are there any plans to deprecate the old trails data? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AJDGLXQBQB3BT46I5IQS6Q3RZCRTRANCNFSM4OIQJXDQ> .

[gregbunce]

Yeah, there's an issue in to deprecate it.

…

On Mon, Jun 29, 2020, 8:39 AM Zach Beck ***@***.***> wrote: It looks like this has made it's way to the Recreation folder but hasn't been shared out to the Recreation group yet. Are there any plans to deprecate the old trails data? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEA7QTBA65ORAHAW6KBRXSLRZCRTRANCNFSM4OIQJXDQ> .

[steveoh]

notice: when you add the website url to the steward ship tab, it needs to contain all parts of the url. greg used the implied path data/recreation/trails when it needs to be data/recreation/trails/index.html

[gregbunce]

Thanks for the note!

…

On Mon, Jun 29, 2020, 4:04 PM steveoh ***@***.***> wrote: *notice*: when you add the website url to the steward ship tab, it needs to contain all parts of the url. greg used the implied path data/recreation/trails when it needs to be data/recreation/trails/index.html — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEA7QTGNKIYWH76PHCJMVZLRZEFVRANCNFSM4OIQJXDQ> .

[steveoh]

@jacobdadams was this item validated in agol and shared properly? We should be able to wrap up this today if that is the case.

[jacobdadams]

Sorry, I got pulled away with some Covid stuff. Yes, it was. I've verified it's shared to the proper groups and available in opendata.

I left the ArcGIS Online box unchecked because it doesn't have a summary, description, or terms of use. @ZachBeck, do we have that info for this dataset?

[ZachBeck]

It has metadata in sde. That doesn't come across when the feature layer is created? I can update it manually if you want.

[jacobdadams]

It should have... let me look into it. Do we know for certain the metadata was in the SDE before forklift added it to AGOL? (even still, agol_validator should have grabbed it too... I need to investigate)

[ZachBeck]

I'm pretty sure it was there on Monday.

[jacobdadams]

Ok, after tracking it down, I think the problem with the metadata is that the SDE metadata is far longer than the ArcGIS API supports uploading (167,418 chars actual, max 32,767 characters supported). It looks like the vast majority of this is in the "lineage" item, tracking all the Geoprocessing tasks that have happened back to 2017.

@gregbunce, do you have any problems blowing away all this old metadata? This will allow any updates to the description, etc to propagate from the SDE to AGOL.

[steveoh]

@gregbunce, do you have any problems blowing away all this old metadata? This will allow any updates to the description, etc to propagate from the SDE to AGOL.

This should be a scheduled event. this will continue to grow and be a reoccurring issue.

[gregbunce]

Yes, please blow it away. Thanks!

…

On Wed, Jul 1, 2020, 1:08 PM Jake Adams ***@***.***> wrote: Ok, after tracking it down, I think the problem with the metadata is that the SDE metadata is far longer than the ArcGIS API supports uploading (167,418 chars actual, max 32,767 characters supported). It looks like the vast majority of this is in the "lineage" item, tracking all the Geoprocessing tasks that have happened back to 2017. @gregbunce <https://github.com/gregbunce>, do you have any problems blowing away all this old metadata? This will allow any updates to the description, etc to propagate from the SDE to AGOL. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#42 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEA7QTDZWEANUVLGNMRRDLTRZOCTPANCNFSM4OIQJXDQ> .

[gregbunce]

thanks everyone for picking up my slack on this last week. it looks good. it like the only remaining item it getting the metadata from Internal DB to AGOL. right? @jacobdadams is there anything i need to do?

[jacobdadams]

No, I just need to trim up the metadata. I'm already running the validator for this week's run, but I can trim the metadata and transfer it to AGOL tomorrow.

[gregbunce]

great, thanks again! no rush.

[jacobdadams]

Ok, most of the metadata has come through. The use limitations may be a little rough, but the rest of it should be there (except for the GP history, which should have been blown away).

[gregbunce]

thanks, jake. it looks good. from my angle, i think we can close this issue. @steveoh what's the process to close this?

[steveoh]

i think hit the button when you are confident

[gregbunce]

i see the 'Close issue' button. but, i want to make sure we're all in agreeance.

[gregbunce]

i'm signing off on this one and closing it. thanks for everyone's help.