Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade numpy and install via pip #3145

Merged
merged 2 commits into from Aug 15, 2019
Merged

Upgrade numpy and install via pip #3145

merged 2 commits into from Aug 15, 2019

Conversation

@caseycesari
Copy link
Member

caseycesari commented Aug 14, 2019

Overview

Undos work done in #3090 to get around an issue with numpy, and installs an updated version of numpy that is compatible with ulmo and other dependencies.

Connects #3093

Testing Instructions

  • Analyze and run models for an area, and confirm that things work as expected.
  • Follow the instructions in #3090 to test that the segfault issue has been addressed.

Checklist

  • All JavaScript tests pass ./scripts/testem.sh
caseycesari added 2 commits Aug 12, 2019
Undos work done in #3090 to get around an issue with numpy,
and installs an updated version of numpy that is compatible
with ulmo and other dependencies.

Refs #3093
@rajadain

This comment has been minimized.

Copy link
Member

rajadain commented Aug 14, 2019

Using the Darby Creek-Mantua Creek, HUC-10 Watershed boundary, I'm seeing different Hydrology, Water Quality, and Summary Load numbers on Local (running on this branch) and Staging:

2019-08-14 14 50 15

diff -u staging/mapshed_hydrology.csv local/mapshed_hydrology.csv
--- staging/mapshed_hydrology.csv	2019-08-14 14:37:30.000000000 -0400
+++ local/mapshed_hydrology.csv	2019-08-14 14:37:33.000000000 -0400
@@ -1,14 +1,14 @@
 Month,"Stream Flow (cm)","Surface Runoff (cm)","Subsurface Flow (cm)","Point Src Flow (cm)","ET (cm)","Precip (cm)"
-Jan,8.58,1.88,3.46,3.24,0.59,7.92
-Feb,8.83,1.83,4.06,2.93,0.91,7.25
-Mar,10.03,1.73,5.06,3.24,2.63,8.75
-Apr,8.60,0.44,5.03,3.14,5.09,8.91
-May,7.46,0.49,3.73,3.24,9.07,9.63
-Jun,5.79,0.55,2.10,3.14,12.06,9.26
-Jul,5.22,0.92,1.06,3.24,11.88,10.81
-Aug,4.29,0.61,0.44,3.24,9.00,9.15
-Sep,4.14,0.69,0.31,3.14,5.63,8.69
-Oct,4.47,0.74,0.48,3.24,4.03,6.98
-Nov,5.01,1.21,0.66,3.14,2.25,8.40
-Dec,6.88,1.61,2.02,3.24,1.13,8.71
-Total,79.30,12.70,28.41,38.17,64.27,104.46
+Jan,9.15,1.88,4.04,3.24,0.59,7.92
+Feb,9.10,1.83,4.34,2.93,0.91,7.25
+Mar,10.16,1.73,5.20,3.24,2.63,8.75
+Apr,8.65,0.44,5.07,3.14,5.09,8.91
+May,7.48,0.49,3.75,3.24,9.07,9.63
+Jun,5.80,0.55,2.11,3.14,11.27,9.26
+Jul,5.22,0.92,1.06,3.24,10.70,10.81
+Aug,4.30,0.61,0.46,3.24,8.74,9.15
+Sep,4.22,0.69,0.40,3.14,5.46,8.69
+Oct,4.58,0.74,0.60,3.24,4.03,6.98
+Nov,5.39,1.21,1.05,3.14,2.25,8.40
+Dec,7.63,1.61,2.78,3.24,1.13,8.71
+Total,81.68,12.70,30.86,38.17,61.87,104.46
diff -u staging/mapshed_water_quality_loads.csv local/mapshed_water_quality_loads.csv
--- staging/mapshed_water_quality_loads.csv	2019-08-14 14:36:36.000000000 -0400
+++ local/mapshed_water_quality_loads.csv	2019-08-14 14:36:46.000000000 -0400
@@ -10,7 +10,7 @@
 "High-Density Mixed","242,440.8","5,323.9",546.6
 "Low-Density Open Space","205,543.5","5,469.9",581.6
 "Farm Animals",0.0,"6,833.7","1,663.0"
-"Stream Bank Erosion","253,144,182.0","136,021.0","46,096.0"
-"Subsurface Flow",0.0,"312,448.0","4,044.5"
+"Stream Bank Erosion","257,710,534.0","138,474.0","46,927.0"
+"Subsurface Flow",0.0,"338,819.4","4,385.9"
 "Point Sources",0.0,"1,752,465.0","77,164.0"
 "Septic Systems",0.0,"26,609.9",0.0
diff -u staging/mapshed_water_quality_summary_loads.csv local/mapshed_water_quality_summary_loads.csv
--- staging/mapshed_water_quality_summary_loads.csv	2019-08-14 14:36:32.000000000 -0400
+++ local/mapshed_water_quality_summary_loads.csv	2019-08-14 14:36:44.000000000 -0400
@@ -1,5 +1,5 @@
 Sources,Sediment,"Total Nitrogen","Total Phosphorus"
-"Total Loads (kg)","256,032,527.2","2,267,038.5","133,262.6"
-"Loading Rates (kg/ha)","6,137.73",54.35,3.19
-"Mean Annual Concentration (mg/L)",774.03,6.85,0.40
-"Mean Low-Flow Concentration (mg/L)",954.55,9.62,0.61
+"Total Loads (kg)","260,598,879.2","2,295,862.9","134,435.0"
+"Loading Rates (kg/ha)","6,247.20",55.04,3.22
+"Mean Annual Concentration (mg/L)",764.69,6.74,0.39
+"Mean Low-Flow Concentration (mg/L)",944.49,9.49,0.60
diff -u staging/project.gms local/project.gms
--- staging/project.gms	2019-08-14 14:56:31.000000000 -0400
+++ local/project.gms	2019-08-14 14:55:57.000000000 -0400
@@ -1,5 +1,5 @@
 10,6,0
-1.4.0,0.03228491379310344,0,10,0,0,0.06723457566537072,12.065378195901392,381204,3188.046345500479,378015.9536544995,619.7222697814921,0,2.9980752361369207,0.0045102962476675475,30,1961,1990,0.003255958295489718,59397.940628333665,0,0,0,0.18325275269809418
+1.4.0,0.03228491379310344,0,10,0,0,0.06723457566537072,9.418960164181,381204,3188.046345500479,378015.9536544995,619.7222697814921,0,2.9980752361369207,0.0045102962476675475,30,1961,1990,0.003255958295489718,59397.940628333665,0,0,0,0.18325275269809418
 0
 0
 0

Relevant files are in here: numpy-upgrade.zip.

I'll try and take a look to see if I can figure out the source of this discrepancy.

@rajadain

This comment has been minimized.

Copy link
Member

rajadain commented Aug 14, 2019

As can be seen in the above diff, there is for some reason a difference in their Avg AWC value (Line 2 Comma 8 in the GMS file), which is causing the difference in numbers. That value is sourced from GeoTrellis rasters, and does not go through NumPy. It may be cached from an earlier run, either on my local or on staging.

I tried other shapes and the numbers match up there.

Copy link
Member

rajadain left a comment

+1 tested, this is working correctly. Numbers from MapShed match up (except in the case when geoprocessing results are different, as expected).

@rajadain rajadain assigned caseycesari and unassigned rajadain Aug 15, 2019
@caseycesari

This comment has been minimized.

Copy link
Member Author

caseycesari commented Aug 15, 2019

Thanks!

@caseycesari caseycesari merged commit dbf7d49 into develop Aug 15, 2019
2 checks passed
2 checks passed
default Build finished.
Details
model-my-watershed-pull-requests Build #4084 succeeded in 13 min
Details
@caseycesari caseycesari deleted the cpc/upgrade-numpy branch Aug 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.