-
Notifications
You must be signed in to change notification settings - Fork 10
Requests Koop AGOL Makes to Server
This pages is guide to the several requests that koop-agol makes to ArcGIS Server to successfully access Feature Services from ArcGIS Online.
There are two primary types of requests structures that Koop builds. The first are requests to Servers for a "single page datasets". These are feature service whose total count of data is less than MaxFeatureCount of the service (a single page of data). The other type of request structures, the more complicated ones, are for when the count is greater than the MaxFeatureCount. Below is documentation on how these structures form requests.
Before Koop know what type of request it needs to build it needs to know the total number of features in the service. It relies on the query below to get the count.
itemJson.url + '/' + options.layer + '/query?where=1=1&returnIdsOnly=true&returnCountOnly=true&f=json';An actual request looks like this:
And the response:
{ count: 997 }If the count from the service is less than the maxfeaturecount on the service, Koop can request a single page data rather easily. All it needs to know is the service url which it can use to request every thing from the /query endpoint:
If the count is great than MaxFeatureCount things get more complicated. Koop needs to "page" over the data. Since koop needs to support many versions of Server it needs to support different ways of paging.
The first thing koop checks is whether or not the service support pagination via the advancedQueryCapabilities.supportsPagination property on the service. If its does then it computes number of pages its going to need to make via:
var pages = Math.ceil(count / maxCount);Then it needs simply builds a series of page request URL using an incrementing offset value like:
var resultOffset = i * maxFeatureCount;
var pageUrl = url + '/' + options.layer + '/query?outSR=4326&f=json&outFields=*&where=1=1';
pageUrl += '&resultOffset=' + resultOffset;
pageUrl += '&resultRecordCount=' + maxFeatureCount;It stores each request in an array of uses an async queue to request each page and currently via 4 or 16 at a time (non-host vs hosted).
In the case when a service does not support pagination via resultOffsets Koop uses a request to outStatistics to help it figure out the page requests it needs to make to get all the data.
A sample stats call looks like this and uses to the ObjectID from the service to get the min and max OID (from which it can build pages):
Which gives koop:
{
displayFieldName: "",
fieldAliases: {
MIN_OID: "MIN_OID",
MAX_OID: "MAX_OID"
},
fields: [
{
name: "MIN_OID",
type: "esriFieldTypeDouble",
alias: "MIN_OID"
},
{
name: "MAX_OID",
type: "esriFieldTypeDouble",
alias: "MAX_OID"
}
],
features: [
{
attributes: {
MIN_OID: 1,
MAX_OID: 43097
}
}
]
}The output above tells koop it will need to make 44 requests using objectIDs that start at 1 and end at 43097.
A sample page request from this looks like:
http://maps2.dcgis.dc.gov/dcgis/rest/services/DCGIS_DATA/Property_and_Land_WebMercator/MapServer//30/query?outSR=4326&where=OBJECTID<=34000+AND+OBJECTID>=33001&f=json&outFields=*&geometry=&returnGeometry=true
In the case when a call to outstatistics fails or returns an invalid response Koop attempts to requests pages for all the data up to total feature count based on objectids starting at 0. It does this as a safety net against failed stats calls and only breaks down when the ObjectIDs are either non-sequential and exceed the total feature count or dont start at 0 (and exceed the feature count).
In the case when stats are not support, ie old server versions, koop pages over the data via the ObjectIDs. First it requests all the ObjectID using returnIdsOnly
var page = url +'/'+ layer + '/query?where=1=1&returnIdsOnly=true&f=json'From that request we split the returned array of IDs into chunks of 1000 (or whatever the MaxFeatureCount value is) and request each page.