How to make authentication work with arcgis online

[jkerr5]

Is it possible to secure koop feature services and use them with ArcGIS online? We tried with the @koopjs/auth-direct-file auth plugin but it doesn't seem to work.

[rgwozdz]

I assume your trying to add a Koop FeatureServer to AGO. If so, @koopjs/auth-direct-file _should work. It was specifically developed and tested for AGO. Are you getting any errors that could help trouble shoot?

[jkerr5]

Yes, that is correct and that was the answer I was hoping for! I will get some more info on the behavior we are seeing and get back to you.

[mfgumban]

Currently, we get an error message "The layer, TestLayer, cannot be added to the map." after trying to add the layer.

These are the steps we took:

• Setup and configure @koopjs/auth-direct-file on our Node app, with addHttp: true in options and a user-store.json file with a single set of credentials. The app is serving on localhost:9000, and also uses the MarkLogic provider plug-in.
• Tested with Postman and seems to be working:
  POST http://localhost:9000/marklogic/tokens?username=user&password=pwd returns a token.
  POST http://localhost:9000/marklogic/TestLayer/FeatureServer/0/query?where=1=1&resultRecordCount=10 with the token in an Authorization header returns an expected JSON response.
• On a new map in AGO, we go to Add->Add Layer from Web, select An ArcGIS Server Web Service, set URI to http://localhost:9000/marklogic/TestLayer/FeatureServer/0, and then add the layer.
• A few moments and we get the above error message. It may be of note that there's nothing getting logged on Koop, which is unusual.

We also tested this w/o the auth plugin and AGO was able to add the layer and pull data.

Is there a way to get more error/log information on AGO side?

[rgwozdz]

Ok, I think the issue is that AGO cannot reach your localhost. When you try to add this layer, it's not just the browser code that uses your Koop URL - the browser takes that URL and sends it off to the AGO servers, where it gets used in the authentication dance among other things.

If you want to test this locally, you still can - I've done it by using ngrok (the ngrok page is down as of this writing) to generate a url that tunnels to my localhost, then I give the ngrok'd url to AGO.

Without something like ngrok, you'll have to deploy your koop instance and expose it so the AGO servers can reach it and make requests.

[jkerr5]

I have a service running that has auth turned on and is publicly accessible but I still can't get it to add to AGO. I tried adding the service as an item under "Content" but once I enter the URL, the dialog just hangs with the message Fetching service info....

The service is running here so maybe you can try it and let us know if it looks like it is behaving correctly:
https://coronavirus-marklogic-featureservice.azurewebsites.net/marklogic/GDeltExample/FeatureServer

[rgwozdz]

Thanks @jkerr5. Unfortunately, something appears to have changed. Possibly with the AGO auth pattern, but I am not sure. No requests are failing, but yes, it fails to proceed. When we developed this, this "fetching" stage was followed by a prompt for username and password.

It may be a few weeks before I can personally look into/repair this. If you are interested in debugging, my suggestion is to follow the network traffic when connecting AGO to a secured ArcGIS server service and see what we are missing in Koop.

[jkerr5]

When auth is on, koop returns the following:

{
  "error": {
    "code": 499,
    "message": "Token Required",
    "details": []
  }
}

And then the call to checkurl on ArcGIS server returns this:

{
  "secured": true,
  "httpStatusCode": 499,
  "httpStatusMessage": "Token Required",
  "httpResponse": "",
  "url": "http://ec2-34-198-222-6.compute-1.amazonaws.com:8080/marklogic/GDeltExample/FeatureServer?f=json"
}

So, it seems to recognize that it is secured but then nothing else happens. Is that 499 code the correct http response for this?

[rgwozdz]

It used to be, at least. Do you have access to a genuine ArcGIS Server service that must be authenticated? If so, I would try to add that ArcGIS Server service to AGO and follow the network traffic - see if their is any differences in the responses of the genuine ArcGIS server service compared to Koop service.

[jkerr5]

OK, some clues:
For the secure hosted service, the web app makes a call to "checkurl" and then a followup call to the rest/info route on the feature service. So, something like this:

http://esri-ops.demo.marklogic.com/arcgis/sharing/rest/portals/checkurl?url=https%3A%2F%2Fservices.arcgis.com%2Fjk49NOwcwcDWjl1P%2Farcgis%2Frest%2Fservices%2Ftest%2FFeatureServer%3Ff%3Djson&f=json&<a bunch of other token stuff>

This returns with

{
   "secured":true,
   "httpStatusCode":499,
   "httpStatusMessage":"Token Required",
   "httpResponse":"",
   "url":"https://services.arcgis.com/jk49NOwcwcDWjl1P/arcgis/rest/services/test/FeatureServer?f=json"
}

And then a call to

https://services.arcgis.com/jk49NOwcwcDWjl1P/arcgis/rest/info?f=json

returns

{
   "currentVersion":10.7,
   "fullVersion":"10.7",
   "owningSystemUrl":"http://www.arcgis.com",
   "owningTenant":"jk49NOwcwcDWjl1P",
   "authInfo":{
      "isTokenBasedSecurity":true,
      "tokenServicesUrl":"https://www.arcgis.com/sharing/generateToken"
 }

For my koop service with auth turned on, it does the first call but does not follow up with a call to the rest/info endpoint.

If I understand this correctly, if I have a service with this URL

https://coronavirus-marklogic-featureservice.azurewebsites.net/marklogic/GDeltExample/FeatureServer

After the call to "checkurl", it should then make a call to

https://coronavirus-marklogic-featureservice.azurewebsites.net/marklogic/rest/info

Instead, the dialog just hangs saying "Fetching Service Info...". There are no errors or anything an no further network calls are made. Maybe there's some issue parsing the URL? I'm at a loss at this point unless I can find some documentation on the authentication process.

[rgwozdz]

When you add your Koop service to AGO, always do so with the layer, and use the Feature server endpoints that include rest/services in the path:

https://coronavirus-marklogic-featureservice.azurewebsites.net/marklogic//rest/services/GDeltExample/FeatureServer/0

I think this may do the trick; many arcgis clients will fail if they don't see rest/services in url.

[jkerr5]

OK, I will try that but other secured services work without doing that so I really don't understand what it doesn't like about the koop services. I compared responses and response headers and I really can't see a difference.

…

On Wed, Apr 1, 2020 at 3:36 PM Rich Gwozdz ***@***.***> wrote: When you add your Koop service to AGO, always do so with the layer, and use the Feature server endpoints that include rest/services in the path: https://coronavirus-marklogic-featureservice.azurewebsites.net/marklogic//rest/services/GDeltExample/FeatureServer/0 I think this may do the trick; many arcgis clients will fail if they don't see rest/services in url. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#178 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAB7YASZCZF6X6CFXZF5YXTRKM7O5ANCNFSM4LU3QXAA> .

[rgwozdz]

Let me know. I do believe the content of the service URL is the issue. In your test of a working service notice the url contains rest/services:
https://services.arcgis.com/jk49NOwcwcDWjl1P/arcgis/rest/services/test/FeatureServer

[jkerr5]

It looks like that is it! I have two serviceURLs that work and they both contain rest/services. I changed mine and that works. I don't need to add it with a layer number though. So, it looks like this is solved but perhaps you can update the notes in the auth plugin to indicate that that form of the service URL must be used?

[rgwozdz]

Yes, sorry for the doc omission. I think it's only been lightly used, otherwise this probably would have come up. This is the reason those rest/services routes were added :).

[jkerr5]

No problem. Thanks for the help!