Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade numpy and install via pip #3145

Merged
merged 2 commits into from
Aug 15, 2019
Merged

Upgrade numpy and install via pip #3145

merged 2 commits into from
Aug 15, 2019

Conversation

caseycesari
Copy link
Member

Overview

Undos work done in #3090 to get around an issue with numpy, and installs an updated version of numpy that is compatible with ulmo and other dependencies.

Connects #3093

Testing Instructions

Checklist

  • All JavaScript tests pass ./scripts/testem.sh

Undos work done in #3090 to get around an issue with numpy,
and installs an updated version of numpy that is compatible
with ulmo and other dependencies.

Refs #3093
@rajadain
Copy link
Member

Using the Darby Creek-Mantua Creek, HUC-10 Watershed boundary, I'm seeing different Hydrology, Water Quality, and Summary Load numbers on Local (running on this branch) and Staging:

2019-08-14 14 50 15

diff -u staging/mapshed_hydrology.csv local/mapshed_hydrology.csv
--- staging/mapshed_hydrology.csv	2019-08-14 14:37:30.000000000 -0400
+++ local/mapshed_hydrology.csv	2019-08-14 14:37:33.000000000 -0400
@@ -1,14 +1,14 @@
 Month,"Stream Flow (cm)","Surface Runoff (cm)","Subsurface Flow (cm)","Point Src Flow (cm)","ET (cm)","Precip (cm)"
-Jan,8.58,1.88,3.46,3.24,0.59,7.92
-Feb,8.83,1.83,4.06,2.93,0.91,7.25
-Mar,10.03,1.73,5.06,3.24,2.63,8.75
-Apr,8.60,0.44,5.03,3.14,5.09,8.91
-May,7.46,0.49,3.73,3.24,9.07,9.63
-Jun,5.79,0.55,2.10,3.14,12.06,9.26
-Jul,5.22,0.92,1.06,3.24,11.88,10.81
-Aug,4.29,0.61,0.44,3.24,9.00,9.15
-Sep,4.14,0.69,0.31,3.14,5.63,8.69
-Oct,4.47,0.74,0.48,3.24,4.03,6.98
-Nov,5.01,1.21,0.66,3.14,2.25,8.40
-Dec,6.88,1.61,2.02,3.24,1.13,8.71
-Total,79.30,12.70,28.41,38.17,64.27,104.46
+Jan,9.15,1.88,4.04,3.24,0.59,7.92
+Feb,9.10,1.83,4.34,2.93,0.91,7.25
+Mar,10.16,1.73,5.20,3.24,2.63,8.75
+Apr,8.65,0.44,5.07,3.14,5.09,8.91
+May,7.48,0.49,3.75,3.24,9.07,9.63
+Jun,5.80,0.55,2.11,3.14,11.27,9.26
+Jul,5.22,0.92,1.06,3.24,10.70,10.81
+Aug,4.30,0.61,0.46,3.24,8.74,9.15
+Sep,4.22,0.69,0.40,3.14,5.46,8.69
+Oct,4.58,0.74,0.60,3.24,4.03,6.98
+Nov,5.39,1.21,1.05,3.14,2.25,8.40
+Dec,7.63,1.61,2.78,3.24,1.13,8.71
+Total,81.68,12.70,30.86,38.17,61.87,104.46
diff -u staging/mapshed_water_quality_loads.csv local/mapshed_water_quality_loads.csv
--- staging/mapshed_water_quality_loads.csv	2019-08-14 14:36:36.000000000 -0400
+++ local/mapshed_water_quality_loads.csv	2019-08-14 14:36:46.000000000 -0400
@@ -10,7 +10,7 @@
 "High-Density Mixed","242,440.8","5,323.9",546.6
 "Low-Density Open Space","205,543.5","5,469.9",581.6
 "Farm Animals",0.0,"6,833.7","1,663.0"
-"Stream Bank Erosion","253,144,182.0","136,021.0","46,096.0"
-"Subsurface Flow",0.0,"312,448.0","4,044.5"
+"Stream Bank Erosion","257,710,534.0","138,474.0","46,927.0"
+"Subsurface Flow",0.0,"338,819.4","4,385.9"
 "Point Sources",0.0,"1,752,465.0","77,164.0"
 "Septic Systems",0.0,"26,609.9",0.0
diff -u staging/mapshed_water_quality_summary_loads.csv local/mapshed_water_quality_summary_loads.csv
--- staging/mapshed_water_quality_summary_loads.csv	2019-08-14 14:36:32.000000000 -0400
+++ local/mapshed_water_quality_summary_loads.csv	2019-08-14 14:36:44.000000000 -0400
@@ -1,5 +1,5 @@
 Sources,Sediment,"Total Nitrogen","Total Phosphorus"
-"Total Loads (kg)","256,032,527.2","2,267,038.5","133,262.6"
-"Loading Rates (kg/ha)","6,137.73",54.35,3.19
-"Mean Annual Concentration (mg/L)",774.03,6.85,0.40
-"Mean Low-Flow Concentration (mg/L)",954.55,9.62,0.61
+"Total Loads (kg)","260,598,879.2","2,295,862.9","134,435.0"
+"Loading Rates (kg/ha)","6,247.20",55.04,3.22
+"Mean Annual Concentration (mg/L)",764.69,6.74,0.39
+"Mean Low-Flow Concentration (mg/L)",944.49,9.49,0.60
diff -u staging/project.gms local/project.gms
--- staging/project.gms	2019-08-14 14:56:31.000000000 -0400
+++ local/project.gms	2019-08-14 14:55:57.000000000 -0400
@@ -1,5 +1,5 @@
 10,6,0
-1.4.0,0.03228491379310344,0,10,0,0,0.06723457566537072,12.065378195901392,381204,3188.046345500479,378015.9536544995,619.7222697814921,0,2.9980752361369207,0.0045102962476675475,30,1961,1990,0.003255958295489718,59397.940628333665,0,0,0,0.18325275269809418
+1.4.0,0.03228491379310344,0,10,0,0,0.06723457566537072,9.418960164181,381204,3188.046345500479,378015.9536544995,619.7222697814921,0,2.9980752361369207,0.0045102962476675475,30,1961,1990,0.003255958295489718,59397.940628333665,0,0,0,0.18325275269809418
 0
 0
 0

Relevant files are in here: numpy-upgrade.zip.

I'll try and take a look to see if I can figure out the source of this discrepancy.

@rajadain
Copy link
Member

rajadain commented Aug 14, 2019

As can be seen in the above diff, there is for some reason a difference in their Avg AWC value (Line 2 Comma 8 in the GMS file), which is causing the difference in numbers. That value is sourced from GeoTrellis rasters, and does not go through NumPy. It may be cached from an earlier run, either on my local or on staging.

I tried other shapes and the numbers match up there.

Copy link
Member

@rajadain rajadain left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 tested, this is working correctly. Numbers from MapShed match up (except in the case when geoprocessing results are different, as expected).

@rajadain rajadain assigned caseycesari and unassigned rajadain Aug 15, 2019
@caseycesari
Copy link
Member Author

Thanks!

@caseycesari caseycesari merged commit dbf7d49 into develop Aug 15, 2019
@caseycesari caseycesari deleted the cpc/upgrade-numpy branch August 15, 2019 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants