Shippensburg Land Projections: Investigate Drexel / ANS API #3323
Comments
|
For a HUC-12 (139 km²), it takes less than 2 seconds the first time, and then a little more than 1 seconds on subsequent calls: ❯ time http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Plymouth\ Creek-Schuylkill\ River,\ HUC-12\ Subwatershed\ \(ID\ 020402031007\).geojson > /dev/null
________________________________________________________
Executed in 1.85 secs fish external
usr time 367.23 millis 133.00 micros 367.09 millis
sys time 98.98 millis 962.00 micros 98.02 millis❯ time http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Plymouth\ Creek-Schuylkill\ River,\ HUC-12\ Subwatershed\ \(ID\ 020402031007\).geojson > /dev/null
________________________________________________________
Executed in 1.25 secs fish external
usr time 305.86 millis 122.00 micros 305.73 millis
sys time 84.14 millis 740.00 micros 83.41 millisFor a HUC-10 (698 km²), it takes about 24 seconds the first time, and about 4 once cached: ❯ time http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Lower\ Schuylkill\ River,\ HUC-10\ Watershed\ \(ID\ 0204020310\).geojson > /dev/null
________________________________________________________
Executed in 24.06 secs fish external
usr time 305.70 millis 140.00 micros 305.56 millis
sys time 82.03 millis 938.00 micros 81.09 millis❯ time http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Lower\ Schuylkill\ River,\ HUC-10\ Watershed\ \(ID\ 0204020310\).geojson > /dev/null
________________________________________________________
Executed in 3.89 secs fish external
usr time 308.76 millis 132.00 micros 308.63 millis
sys time 79.53 millis 882.00 micros 78.65 millisFor a HUC-8 (4,958 km²), it takes almost a minute the first time, and about 7 seconds on subsequent calls: ❯ time http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Schuylkill,\ HUC-8\ Subbasin\ \(ID\ 02040203\).geojson > /dev/null
________________________________________________________
Executed in 55.22 secs fish external
usr time 307.69 millis 166.00 micros 307.52 millis
sys time 83.98 millis 1213.00 micros 82.76 millis❯ time http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Schuylkill,\ HUC-8\ Subbasin\ \(ID\ 02040203\).geojson > /dev/null
________________________________________________________
Executed in 7.07 secs fish external
usr time 313.13 millis 160.00 micros 312.97 millis
sys time 87.21 millis 1027.00 micros 86.18 millisWe do not know the duration of caching. Here's the output for the Schuylkill HUC-8: ❯ http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Schuylkill,\ HUC-8\ Subbasin\ \(ID\ 02040203\).geojson
HTTP/1.1 200 OK
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 551
Content-Type: text/html; charset=utf-8
Date: Wed, 27 May 2020 21:52:09 GMT
Server: nginx/1.15.8
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
{
"centers": {
"1": 5288185800,
"10": 16001472600,
"11": 2426226300,
"12": 69529230000,
"13": 48443607000,
"14": 6173243100,
"15": 397682100,
"2": 55849214700,
"3": 74663539200,
"4": 19647468900,
"5": 16073052300,
"6": 2686145400,
"7": 167243444100,
"8": 4651010100,
"9": 6147698400
},
"centers_np": {
"1": 5271332400,
"10": 15039144900,
"11": 2283069600,
"12": 66226437000,
"13": 47603339100,
"14": 6048302400,
"15": 397372500,
"2": 56176504200,
"3": 85419781200,
"4": 19741608000,
"5": 16679997000,
"6": 2634624000,
"7": 161418491100,
"8": 4393466100,
"9": 5887750500
},
"centers_osi": {
"1": 5285897100,
"10": 16001513100,
"11": 2428606800,
"12": 69575749200,
"13": 48456860400,
"14": 6172389900,
"15": 398129400,
"2": 55849112100,
"3": 74625773400,
"4": 19651956300,
"5": 16046587800,
"6": 2686999500,
"7": 167245635600,
"8": 4649630400,
"9": 6146379000
},
"corridors": {
"1": 5271406200,
"10": 14424579900,
"11": 2172564900,
"12": 64172549700,
"13": 45888007500,
"14": 5973941700,
"15": 392391900,
"2": 56383422300,
"3": 89910351900,
"4": 19779095700,
"5": 17335650600,
"6": 2600760600,
"7": 160699795200,
"8": 4394399400,
"9": 5822302500
},
"corridors_np": {
"1": 5271332400,
"10": 15039144900,
"11": 2283069600,
"12": 66226437000,
"13": 47603339100,
"14": 6048302400,
"15": 397372500,
"2": 56176504200,
"3": 85419781200,
"4": 19741608000,
"5": 16679997000,
"6": 2634624000,
"7": 161418491100,
"8": 4393466100,
"9": 5887750500
},
"corridors_osi": {
"1": 5272549200,
"10": 14427398700,
"11": 2172262500,
"12": 64233306900,
"13": 45896356800,
"14": 5974942500,
"15": 392555700,
"2": 56379307500,
"3": 89808568200,
"4": 19776469500,
"5": 17323920000,
"6": 2598774300,
"7": 160742040300,
"8": 4396157100,
"9": 5826610800
}
}Notably the response has an incorrect header: Content-Type: text/html; charset=utf-8It should be |
|
Turns out the time difference was not because of caching they implemented, but the difference between reading off disk and reading off memory in subsequent calls. They have since switched from HDDs to SSDs and now the times are: ~2s for HUC-12, ~4s for HUC-10, ~7s for HUC-8. |
|
Furthermore, the return values are now in NLCD: ❯ http http://watersheds.cci.drexel.edu/api/fzs_buildout/ < Schuylkill,\ HUC-8\ Subbasin\ \(ID\ 02040203\).geojson
HTTP/1.1 200 OK
Connection: keep-alive
Content-Encoding: gzip
Content-Length: 553
Content-Type: text/html; charset=utf-8
Date: Fri, 05 Jun 2020 16:29:09 GMT
Server: nginx/1.15.8
Vary: Accept-Encoding
X-Frame-Options: SAMEORIGIN
{
"centers": {
"11": 5288185800,
"21": 55849214700,
"22": 74663539200,
"23": 19647468900,
"24": 16073052300,
"31": 2686145400,
"41": 167243444100,
"42": 4651010100,
"43": 6147698400,
"52": 16001472600,
"71": 2426226300,
"81": 69529230000,
"82": 48443607000,
"90": 6173243100,
"95": 397682100
},
"centers_np": {
"11": 5271332400,
"21": 56176504200,
"22": 85419781200,
"23": 19741608000,
"24": 16679997000,
"31": 2634624000,
"41": 161418491100,
"42": 4393466100,
"43": 5887750500,
"52": 15039144900,
"71": 2283069600,
"81": 66226437000,
"82": 47603339100,
"90": 6048302400,
"95": 397372500
},
"centers_osi": {
"11": 5285897100,
"21": 55849112100,
"22": 74625773400,
"23": 19651956300,
"24": 16046587800,
"31": 2686999500,
"41": 167245635600,
"42": 4649630400,
"43": 6146379000,
"52": 16001513100,
"71": 2428606800,
"81": 69575749200,
"82": 48456860400,
"90": 6172389900,
"95": 398129400
},
"corridors": {
"11": 5271406200,
"21": 56383422300,
"22": 89910351900,
"23": 19779095700,
"24": 17335650600,
"31": 2600760600,
"41": 160699795200,
"42": 4394399400,
"43": 5822302500,
"52": 14424579900,
"71": 2172564900,
"81": 64172549700,
"82": 45888007500,
"90": 5973941700,
"95": 392391900
},
"corridors_np": {
"11": 5271332400,
"21": 56176504200,
"22": 85419781200,
"23": 19741608000,
"24": 16679997000,
"31": 2634624000,
"41": 161418491100,
"42": 4393466100,
"43": 5887750500,
"52": 15039144900,
"71": 2283069600,
"81": 66226437000,
"82": 47603339100,
"90": 6048302400,
"95": 397372500
},
"corridors_osi": {
"11": 5272549200,
"21": 56379307500,
"22": 89808568200,
"23": 19776469500,
"24": 17323920000,
"31": 2598774300,
"41": 160742040300,
"42": 4396157100,
"43": 5826610800,
"52": 14427398700,
"71": 2172262500,
"81": 64233306900,
"82": 45896356800,
"90": 5974942500,
"95": 392555700
}
} |
|
Drexel is going to make further changes to this API so that we can send in the keys of layers we want, and only return those layers, speeding this up by ~66%. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
From Mike Campagna at ANS:
They're working on converting the API to return actual NLCD codes.
They also said this:
Test the API with HUC-12, HUC-10, and HUC-8 GeoJSONs and report timing and DX issues.
The text was updated successfully, but these errors were encountered: