From 2e13a307860145591f045c1389ea0d2195bc4145 Mon Sep 17 00:00:00 2001 From: fernando-aristizabal Date: Wed, 7 Jul 2021 19:41:13 +0000 Subject: [PATCH] Squashed commit of the following: commit 7befb3929d573de1ee6128b05a33a84c7f77a6da Author: Brad Date: Thu Jul 1 10:16:49 2021 -0500 Allowance of evaluation at IFC sites and updating of bad eval sites list Updates to evaluation scripts to allow for Alpha testing at Iowa Flood Center (IFC) sites. Also, BAD_SITES variable updates to omit sites not suitable for evaluation from metric calculations. - The BAD_SITES list in tools_shared_variables.py was updated and reasons for site omission are documented. - Refactored run_test_case.py, synthesize_test_cases.py, tools_shared_variables.py, and eval_plots.py to allow for IFC comparisons. This resolves #424, resolves #412, and resolves #428. commit ac4ae06febed568d15e349fd3206b5e534f89fd0 Author: Brian Avant Date: Mon Jun 21 07:46:35 2021 -0500 Thalweg Profile Tool and lateral Thalweg Adjustment Threshold Adding a thalweg profile tool to identify significant drops in thalweg elevation. Also setting lateral thalweg adjustment threshold in hydroconditioning. - thalweg_drop_check.py checks the elevation along the thalweg for each stream path downstream of MS headwaters within a HUC. - Removing dissolveLinks arg from clip_vectors_to_wbd.py. - Cleaned up code in split_flows.py to make it more readable. - Refactored reduce_nhd_stream_density.py and adjust_headwater_streams.py to limit MS headwater points in agg_nhd_headwaters_adj.gpkg. - Fixed a bug in adjust_thalweg_lateral.py lateral elevation replacement threshold; changed threshold to 3 meters. - Updated aggregate_vector_inputs.py to log intermediate processes. This resolves #408, resolves #18, and resolves #409. commit ca8cb7e8000befe1b05495277f0b2d362565a860 Author: Brad Date: Thu Jun 10 10:54:32 2021 -0500 Feature to evaluate alternative CatFIM technique performance Added eval_catfim_alt.py to evaluate performance of alternative CatFIM techniques. This resolves #414. commit 129f2d567f2932ad6f3f2b15f8cdc92d8c1d0645 Author: NickChadwick-NOAA Date: Wed Jun 9 10:38:19 2021 -0500 Fim Run Memory Profiling To help analyze the memory consumption of the Fim Run process, the python module memory-profiler has been added to give insights into where peak memory usage is with in the codebase. In addition, the Dockerfile was previously broken due to the Taudem dependency removing the version that was previously being used by FIM. To fix this, and allow new docker images to be built, the Taudem version has been updated to the newest version on the Github repo and thus needs to be thoroughly tested to determine if this new version has affected the overall FIM outputs. Resolves #392 and #403 Additions - Added memory-profiler to Pipfile and Pipfile.lock. - Added mprof (memory-profiler cli utility) call to the time_and_tee_run_by_unitsh to create overall memory usage graph location in the /logs/{HUC}_memory.png of the outputs directory. - Added @profile decorator to all functions within scripts used in the run_by_unit.sh script to allow for memory usage tracking, which is then recorded in the /logs/{HUC}.log file of the outputs directory. Changes - Changed the Taudem version in Dockerfile.dev to 98137bb6541a0d0077a9c95becfed4e56d0aa0ac. - Changed all calls of python scripts in run_by_unit.sh to be called with the -m memory-profiler argument to allow scripts to also track memory usage. commit a001d75fd7905f7232423c9f9ada0b148291e66c Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Fri Jun 4 15:25:56 2021 -0500 Correct headwater assignment nws_lid layer Bug fix to the generate_nws_lid.py script - Fixes incorrectly assigned attribute field "is_headwater" for some sites in the nws_lid.gpkg layer. - Updated agg_nhd_headwaters_adj.gpkg, agg_nhd_streams_adj.gpkg, nwm_flows.gpkg, and nwm_catchments.gpkg input layers using latest NWS LIDs. This resolves #383 and resolves #397. commit cca68e14a337714ac28727da5de944e3b7c615d0 Author: Brad Date: Fri Jun 4 14:03:05 2021 -0500 Update CHANGELOG.md commit b9167d52a8bb6260d6bae066e9a51e39ae93c11b Author: RyanSpies-NOAA Date: Fri Jun 4 14:00:06 2021 -0500 BARC updates to cap xsec area and allow user to choose input bankfull geometry BARC updates to cap the bathy calculated xsec area in bathy_rc_adjust.py and allow user to choose input bankfull geometry. - Added new env variable to control which input file is used for the bankfull geometry input to bathy estimation workflow. - Modified the bathymetry cross section area calculation to cap the additional area value so that it cannot exceed the bankfull cross section area value for each stream segment (bankfull value obtained from regression equation dataset). - Modified the rating_curve_comparison.py plot output to always put the FIM rating curve on top of the USGS rating curve (avoids USGS points covering FIM). - Created a new aggregate csv file (aggregates for all hucs) for all of the usgs_elev_table.csv files (one per huc). - Evaluate the FIM Bathymetry Adjusted Rating Curve (BARC) tool performance using the estimated bankfull geometry dataset derived for the NWM route link dataset. This resolves #342, resolves #343, resolves #313, resolves #401, and resolves #402. commit ac25186becf5438be9a1d6ecd4c9af99fef043aa Author: Brad Date: Fri May 21 13:31:07 2021 -0500 Alpha test bug fixes and improvements Enhancements and bug fixes to synthesize_test_cases.py. - Addresses a bug where AHPS sites without benchmark data were receiving a CSI of 0 in the master metrics CSV produced by synthesize_test_cases.py. - Includes a feature enhancement to synthesize_test_cases.py that allows for the inclusion of user-specified testing versions in the master metrics CSV. - Removes some of the print statements used by synthesize_test_cases.py. This resolves #354 and resolves #386. commit f2d65eed0166d3d5818091318e2b6487e93209de Author: Brad Date: Tue May 18 13:34:26 2021 -0500 Update CHANGELOG.md commit 8fbf4b666efdba9ab7706d6a93b77baf07dede6f Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Tue May 18 13:26:56 2021 -0500 Bug fixes in preprocessing and run_test_case.py and update eval_plots Modifications and fixes to run_test_case.py, eval_plots.py, and AHPS preprocessing scripts. - Comment out return statement causing run_test_case.py to skip over sites/hucs when calculating contingency rasters. - Move bad sites list and query statement used to filter out bad sites to the tools_shared_variables.py. - Add print statements in eval_plots.py detailing the bad sites used and the query used to filter out bad sites. - Update AHPS preprocessing scripts to produce a domain shapefile. - Change output filenames produced in ahps preprocessing scripts. - Update workarounds for some sites in ahps preprocessing scripts. This resolves #381, resolves #369, resolves #326, resolves #385, resolves #377. commit cbc360f1f27ea15e87510024ae340b5f53fe1638 Author: NickChadwick-NOAA Date: Tue May 11 10:42:27 2021 -0500 Updated Eventlet package to 0.31.0 to patch a vulnerability The current version of Eventlet used in the Connector module of the FIM API is outdated and vulnerable. This update bumps the version to the patched version. Using new nhd inputs. commit b5da4b4869ccff2d87b2326c29d378990daf1c3e Author: NickChadwick-NOAA Date: Fri May 7 14:45:39 2021 -0500 Added fuctionality for new "Release" feature in FIM API New "Release" feature added to the FIM API. This feature will allow for automated FIM, CatFIM, and relevant metrics to be generated when a new FIM Version is released. See #373 for more detailed steps that take place in this feature. Additions - Added new window to the UI in api/frontend/gui/templates/index.html. - Added new job type to api/node/connector/connector.py to allow these release jobs to run. - Added additional logic in api/node/updater/updater.py to run the new eval and CatFIM scripts used in the release feature. Changes - Updated api/frontend/output_handler/output_handler.py to allow for copying more broad ranges of file paths instead of only the /data/outputs directory. Resolves Issues #264 #278 #307 and #373 commit 22ba0df00e403681e12d8f247e35da124536a74a Author: Brian Avant Date: Fri May 7 12:51:19 2021 -0500 Remove Great Lakes from wbd buffer - The gl_water_polygons.gpkg layer is used to mask out Great Lakes boundaries and remove NHDPlus HR coastline segments. These segments are causing issues later in run_by_unit.sh and unnecessarily increasing total processing time. Resolves issue #374 commit 25c3ec155cb4439f527bcd6dcdfb1415056f0975 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Mon May 3 10:07:19 2021 -0500 Generate nws_lid.gpkg Generate nws_lid.gpkg. - Generate nws_lid.gpkg with attributes indicating if site is a headwater nws_lid as well as if it is co-located with another nws_lid which is referenced to the same nwm_feature_id segment. This resolves #368. commit 9518cbe20383ebbff4d2dbfbf7e658ed112835e3 Author: Brian Avant Date: Fri Apr 30 15:41:29 2021 -0500 Refactor NHDPlus HR preprocessing workflow Refactor NHDPlus HR preprocessing workflow. - Consolidate NHD streams, NWM catchments, and headwaters MS and FR layers with mainstem column. - HUC8 intersections are included in the input headwaters layer. - clip_vectors_to_wbd.py removes incoming stream segment from the selected layers. This resolves #238. commit 199ab6e787b1742f0558a69247e08d411add2505 Author: Brian Avant Date: Wed Apr 28 11:32:11 2021 -0500 Refactor synthesize_test_case.py to handle exceptions during multiprocessing Refactor synthesize_test_case.py to handle exceptions during multiprocessing. Resolves issue #351 Changes - refactored inundation.py and run_test_case.py to handle exceptions without using sys.exit(). This isn't the ideal way to handle exceptions when using multiprocessing because the child process is expecting a return. commit 99c197b5ec6cbddb69d58795b0103980e8075c18 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Tue Apr 27 12:42:59 2021 -0500 Implement CatFIM threshold flows to Sierra test and add AHPS benchmark preprocessing scripts. Implement CatFIM threshold flows to Sierra test and add AHPS benchmark preprocessing scripts. - Produce CatFIM flows file when running rating_curve_get_usgs_gages.py. - Several scripts to preprocess AHPS benchmark data. Requires numerous file dependencies not available through Cahaba. - Modify rating_curve_comparison.py to ingest CatFIM threshold flows in calculations. - Modify eval_plots.py to save all site specific bar plots in same parent directory instead of in subdirectories. - Add variables to env.template for AHPS benchmark preprocessing. This resolves #341, resolves #361, and resolves #357. commit c31131716224eccfd937343deb67e226e24dcef3 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Thu Apr 22 15:32:10 2021 -0500 Update plotting script to not error out if FR or MS are not analyzed when using spatial argument update eval_plots.py to correctly produce spatial datasets if version is not analyzed. For example, if no FR versions are analyzed, instead of throwing an error, it will check to make sure there exists a FR analyzed datasets, if one exists then create spatial data for FR datasets (ble). commit 6387384b62523b601790a04e1f5dd2ca09578c29 Author: Brian Avant Date: Tue Apr 20 09:54:37 2021 -0500 Closing multiprocessing pool objects Updating repo to close all multiprocessing Pool objects to fix memory issues commit a304cf386b6c196d58d6bf7d794672ccea2a500d Author: Brian Avant Date: Mon Apr 19 12:14:47 2021 -0500 Preprocess NHDPlus HR rasters for consistent projections, nodata values, and convert from cm to meters. Preprocess NHDPlus HR rasters for consistent projections, nodata values, and convert from cm to meters. - preprocess_rasters.py reprojects raster, converts to meters, and updates nodata value to -9999. - Cleaned up log messages from bathy_rc_adjust.py and usgs_gage_crosswalk.py. - Outputs paths updated in generate_categorical_fim_mapping.py and generate_categorical_fim.py. - update_raster_profile cleans up raster crs, blocksize, nodata values, and converts elevation grids from cm to meters. - reproject_dem.py imports gdal to reproject elevation rasters because an error was occurring when using rasterio. - burn_in_levees.py replaces the gdal_calc.py command to resolve inconsistent outputs with burned in levee values. This resolves #300. commit 1a15cd822d249744c74f8a2c43ac7ffb588dc346 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Fri Apr 16 14:09:21 2021 -0500 Fixes production list in output_cleanup.py. Fixes production list in output_cleanup.py. commit 5a8ebcd1e8c054c0a6e82a724142a78a469bffe2 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Wed Apr 14 08:47:58 2021 -0500 Add mainstems attribute for sierra test Sierra test considered all USGS gage locations to be mainstems even though many actually occurred with tributaries. This resulted in unrealistic comparisons as incorrect gages were assigned to mainstems segments. This feature branch identifies gages that are on mainstems via attribute field. - Modifies usgs_gage_crosswalk.py to filter out gages from the usgs_gages.gpkg layer such that for a "MS" run, only consider gages that contain rating curve information (via curve attribute) and are also mainstems gages (via mainstems attribute). - Modifies usgs_gage_crosswalk.py to filter out gages from the usgs_gages.gpkg layer such that for a "FR" run, only consider gages that contain rating curve information (via curve attribute) and are not mainstems gages (via mainstems attribute). - Modifies how mainstems segments are determined by using the nwm_flows_ms.gpkg as a lookup to determine if the NWM segment specified by WRDS for a gage site is a mainstems gage. - Adds a mainstem attribute field to usgs_gages.gpkg that indicates whether a gage is located on a mainstems river. - Adds NWM_FLOWS_MS variable to the .env and .env.template files. - Adds the extent argument specified by user when running fim_run.sh to usgs_gage_crosswalk.py. commit c9161b81bf124a16f714486cb36a0e8d44067cf0 Author: RyanSpies-NOAA Date: Thu Apr 8 11:11:56 2021 -0500 Implementing a prototype technique to estimate the missing bathymetric component in the HAND-derived synthetic rating curves. Implementing a prototype technique to estimate the missing bathymetric component in the HAND-derived synthetic rating curves. The new Bathymetric Adjusted Rating Curve (BARC) function is built within the fim_run.sh workflow and will ingest bankfull geometry estimates provided by the user to modify the cross section area used in the synthetic rating curve generation. - add_crosswalk.py outputs the stream order variables to src_full_crosswalked.csv and calls the new bathy_rc_adjust.py if bathy env variable set to True and extent=MS. - run_by_unit.sh includes a new csv outputs for reviewing BARC calculations. - params_template.env & params_calibrated.env contain new BARC function input variables and on/off toggle variable. eval_plots.py now includes additional AHPS eval sites in the list of "bad_sites" (flagged issues with MS flowlines). - Adds bathy_rc_adjust.py - Imports the existing synthetic rating curve table and the bankfull geometry input data (topwidth and cross section area per COMID). - Performs new synthetic rating curve calculations with bathymetry estimation modifications. - Flags issues with the thalweg-notch artifact. This resolves #306, resolves #328, resolves #309, resolves #311, and resolves #330. commit 40631f42e5f9ac97204c9a54d4bd9cb7204efffa Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Mon Apr 5 13:45:09 2021 -0500 Create tool to retrieve rating curves from USGS sites and convert to elevation (NAVD88). Intended to be used as part of the Sierra Test. - Modify usgs_gage_crosswalk.py to: - Look for location_id instead of site_no attribute field in usgs_gages.gpkg file. - Filter out gages that do not have rating curves included in the usgs_rating_curves.csv - Modify rating_curve_comparison.py to perform a check on the age of the user specified usgs_rating_curves.csv and alert user to the age of the file and recommend updating if file is older the 30 days. - Add rating_curve_get_usgs_curves.py. This script will generate the following files: - usgs_rating_curves.csv: A csv file that contains rating curves (including converted to NAVD88 elevation) for USGS gages in a format that is compatible with rating_curve_comparisons.py. As it is is currently configured, only gages within CONUS will have rating curve data. - log.csv: A log file that records status for each gage and includes error messages. - usgs_gages.gpkg: A geospatial layer (in FIM projection) of all active USGS gages that meet a predefined criteria. Additionally, the curve attribute indicates whether a rating curve is found in the usgs_rating_curves.csv. This spatial file is only generated if the all option is passed with the -l argument. This resolves #289. commit 44b633381d37eb7f0a636ba56914850a33acdd42 Author: brian.avant Date: Thu Apr 1 13:40:49 2021 -0500 [1pt] Tool to compare synthetic rating curve with benchmark rating curve (sierra test) (#332) * `usgs_gage_crosswalk.py`: generates `usgs_elev_table.csv` in run_by_unit.py with elevation at USGS gages * `rating_curve_comparison.py`: post-processing script to plot and calculate metrics between FIM and USGS rating curve data * updates `aggregate_fim_outputs.py` call argument in `fim_run.sh` from 4 jobs to 6 jobs (optimizing API performance) * reroutes median elevation data from `add_crosswalk.py` and `rem.py` to new file (depreciating `hand_ref_elev_table.csv`) *- adds new files to `viz_whitelist` in `output_cleanup.py` commit 4df6d1d7a5935310a873415f046da6c84ddc4f0c Author: Brad Date: Thu Apr 1 12:27:10 2021 -0500 Update CHANGELOG.md commit 385ae0f5e5e96c6f07dd9796c0056b276e03cb41 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Thu Apr 1 12:24:05 2021 -0500 Update spatial option when performing eval plots Removes file dependencies from spatial option. Does require the WBD layer which should be specified in .env file. - Produces outputs in a format consistent with requirements needed for publishing. - Preserves leading zeros in huc information for all outputs from eval_plots.py. - Creates fim_performance_points.shp: this layer consists of all evaluated ahps points (with metrics). Spatial data retrieved from WRDS on the fly. - Creates fim_performance_polys.shp: this layer consists of all evaluated huc8s (with metrics). Spatial data retrieved from WBD layer. This resolves #325. commit 391b547ab9bec87eb26671a73190474c9c6c0a5d Author: Brad Date: Wed Mar 31 10:00:34 2021 -0500 Update README.md commit a60e6eee97391416a3c8f44242f181cdf02abac1 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Tue Mar 30 16:17:02 2021 -0500 Adds detail and more information to plotting capabilities Add detail/information to plotting capabilities. - Merge plot_functions.py into eval_plots.py and move eval_plots.py into the tools directory. - Remove plots subdirectory. - Optional argument to create barplots of CSI for each individual site. - Create a csv containing the data used to create the scatterplots. This resolves #325. commit 2b260655cd018d6ffb44f2868f55efdd1d98585d Author: Brad Date: Thu Mar 25 09:08:06 2021 -0500 Update CHANGELOG.md commit 34123ddb37d6b6c44caf3572d8683d6b433ebca1 Author: TrevorGrout-NOAA <69653333+TrevorGrout-NOAA@users.noreply.github.com> Date: Thu Mar 25 08:51:04 2021 -0500 Improvements to CatFIM service source data generation Improvements to CatFIM service source data generation. - Renamed generate_categorical_fim.py to generate_categorical_fim_mapping.py. - Updated the status outputs of the nws_lid_sites layer and saved it in the same directory as the merged catfim_library layer. - Additional stability fixes (such as improved compatability with WRDS updates). - Added generate_categorical_fim.py to wrap generate_categorical_fim_flows.py and generate_categorical_fim_mapping.py. - Create new nws_lid_sites shapefile located in same directory as the catfim_library shapefile. This resolves #305, resolves #321, and resolves #322. commit d43bd8b6ab433f68729fb9337bcb7cb6132461e2 Author: Brad Date: Wed Mar 24 10:21:04 2021 -0500 Patch to synthesize_test_cases.py Bug fix to synthesize_test_cases.py to allow comparison between testing version and official versions. --- CHANGELOG.md | 230 ++++++++- Pipfile | 1 + Pipfile.lock | 373 +++++++------- README.md | 5 +- api/frontend/gui/templates/index.html | 158 ++++-- api/frontend/output_handler/output_handler.py | 6 +- api/node/connector/connector.py | 157 +++--- api/node/connector/requirements.txt | 2 +- api/node/updater/updater.py | 306 +++++++++-- config/params_calibrated.env | 5 +- config/params_template.env | 5 +- fim_run.sh | 17 +- src/acquire_and_preprocess_inputs.py | 23 +- src/add_crosswalk.py | 29 +- src/adjust_headwater_streams.py | 237 +++++---- src/adjust_thalweg_lateral.py | 88 ++-- src/aggregate_fim_outputs.py | 22 +- src/aggregate_vector_inputs.py | 451 +++++++++-------- src/agreedem.py | 4 +- src/bathy_rc_adjust.py | 9 +- src/burn_in_levees.py | 42 ++ src/clip_vectors_to_wbd.py | 113 +++-- src/output_cleanup.py | 9 +- src/preprocess_rasters.py | 56 +++ src/raster.py | 462 ----------------- src/reachID_grid_to_vector_points.py | 98 ++-- src/reduce_nhd_stream_density.py | 158 ++++-- src/rem.py | 2 +- src/run_by_unit.sh | 118 +---- src/split_flows.py | 18 +- src/time_and_tee_run_by_unit.sh | 3 +- src/unique_pixel_and_allocation.py | 1 + src/usgs_gage_crosswalk.py | 4 +- src/utils/reproject_dem.py | 50 ++ src/utils/shared_functions.py | 125 +++++ src/utils/shared_variables.py | 28 ++ tools/.env.template | 2 + tools/cache_metrics.py | 46 +- tools/comparing_src.py | 393 --------------- tools/eval_alt_catfim.py | 221 ++++++++ tools/eval_plots.py | 150 +++--- tools/generate_categorical_fim.py | 84 ++-- tools/generate_categorical_fim_mapping.py | 27 +- tools/generate_nws_lid.py | 182 +++++++ tools/inundation.py | 231 +++++---- tools/preprocess_ahps_nws.py | 366 ++++++++++++++ tools/preprocess_ahps_usgs.py | 473 ++++++++++++++++++ tools/preprocess_download_usgs_grids.py | 162 ++++++ tools/rating_curve_comparison.py | 50 +- tools/rating_curve_get_usgs_curves.py | 79 ++- tools/run_test_case.py | 22 +- tools/synthesize_test_cases.py | 89 ++-- tools/thalweg_drop_check.py | 385 ++++++++++++++ tools/tools_shared_functions.py | 321 +++++++++++- tools/tools_shared_variables.py | 37 ++ 55 files changed, 4522 insertions(+), 2213 deletions(-) create mode 100755 src/burn_in_levees.py create mode 100755 src/preprocess_rasters.py delete mode 100644 src/raster.py create mode 100755 src/utils/reproject_dem.py delete mode 100755 tools/comparing_src.py create mode 100644 tools/eval_alt_catfim.py create mode 100644 tools/generate_nws_lid.py create mode 100644 tools/preprocess_ahps_nws.py create mode 100644 tools/preprocess_ahps_usgs.py create mode 100644 tools/preprocess_download_usgs_grids.py create mode 100644 tools/thalweg_drop_check.py diff --git a/CHANGELOG.md b/CHANGELOG.md index baa3ba19f..ebe838c85 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,23 +1,227 @@ All notable changes to this project will be documented in this file. We follow the [Semantic Versioning 2.0.0](http://semver.org/) format. +

+ +## v3.0.19.2 - 2021-07-01 - [PR #429](https://github.com/NOAA-OWP/cahaba/pull/429) + +Updates to evaluation scripts to allow for Alpha testing at Iowa Flood Center (IFC) sites. Also, `BAD_SITES` variable updates to omit sites not suitable for evaluation from metric calculations. + +## Changes +- The `BAD_SITES` list in `tools_shared_variables.py` was updated and reasons for site omission are documented. +- Refactored `run_test_case.py`, `synthesize_test_cases.py`, `tools_shared_variables.py`, and `eval_plots.py` to allow for IFC comparisons. + +

+ +## v3.0.19.1 - 2021-06-17 - [PR #417](https://github.com/NOAA-OWP/cahaba/pull/417) + +Adding a thalweg profile tool to identify significant drops in thalweg elevation. Also setting lateral thalweg adjustment threshold in hydroconditioning. + +## Additions +- `thalweg_drop_check.py` checks the elevation along the thalweg for each stream path downstream of MS headwaters within a HUC. + +## Removals +- Removing `dissolveLinks` arg from `clip_vectors_to_wbd.py`. + + +## Changes +- Cleaned up code in `split_flows.py` to make it more readable. +- Refactored `reduce_nhd_stream_density.py` and `adjust_headwater_streams.py` to limit MS headwater points in `agg_nhd_headwaters_adj.gpkg`. +- Fixed a bug in `adjust_thalweg_lateral.py` lateral elevation replacement threshold; changed threshold to 3 meters. +- Updated `aggregate_vector_inputs.py` to log intermediate processes. + +

+ +## v3.0.19.0 - 2021-06-10 - [PR #415](https://github.com/NOAA-OWP/cahaba/pull/415) + +Feature to evaluate performance of alternative CatFIM techniques. + +## Additions +- Added `eval_catfim_alt.py` to evaluate performance of alternative CatFIM techniques. + +

+## v3.0.18.0 - 2021-06-09 - [PR #404](https://github.com/NOAA-OWP/cahaba/pull/404) + +To help analyze the memory consumption of the Fim Run process, the python module `memory-profiler` has been added to give insights into where peak memory usage is with in the codebase. + +In addition, the Dockerfile was previously broken due to the Taudem dependency removing the version that was previously being used by FIM. To fix this, and allow new docker images to be built, the Taudem version has been updated to the newest version on the Github repo and thus needs to be thoroughly tested to determine if this new version has affected the overall FIM outputs. + +## Additions +- Added `memory-profiler` to `Pipfile` and `Pipfile.lock`. +- Added `mprof` (memory-profiler cli utility) call to the `time_and_tee_run_by_unit.sh` to create overall memory usage graph location in the `/logs/{HUC}_memory.png` of the outputs directory. +- Added `@profile` decorator to all functions within scripts used in the `run_by_unit.sh` script to allow for memory usage tracking, which is then recorded in the `/logs/{HUC}.log` file of the outputs directory. + +## Changes +- Changed the Taudem version in `Dockerfile.dev` to `98137bb6541a0d0077a9c95becfed4e56d0aa0ac`. +- Changed all calls of python scripts in `run_by_unit.s` to be called with the `-m memory-profiler` argument to allow scripts to also track memory usage. + +

+## v3.0.17.1 - 2021-06-04 - [PR #395](https://github.com/NOAA-OWP/cahaba/pull/395) + +Bug fix to the `generate_nws_lid.py` script + +## Changes +- Fixes incorrectly assigned attribute field "is_headwater" for some sites in the `nws_lid.gpkg` layer. +- Updated `agg_nhd_headwaters_adj.gpkg`, `agg_nhd_streams_adj.gpkg`, `nwm_flows.gpkg`, and `nwm_catchments.gpkg` input layers using latest NWS LIDs. + +

+## v3.0.17.0 - 2021-06-04 - [PR #393](https://github.com/NOAA-OWP/cahaba/pull/393) +BARC updates to cap the bathy calculated xsec area in `bathy_rc_adjust.py` and allow user to choose input bankfull geometry. + +## Changes + +- Added new env variable to control which input file is used for the bankfull geometry input to bathy estimation workflow. +- Modified the bathymetry cross section area calculation to cap the additional area value so that it cannot exceed the bankfull cross section area value for each stream segment (bankfull value obtained from regression equation dataset). +- Modified the `rating_curve_comparison.py` plot output to always put the FIM rating curve on top of the USGS rating curve (avoids USGS points covering FIM). +- Created a new aggregate csv file (aggregates for all hucs) for all of the `usgs_elev_table.csv` files (one per huc). +- Evaluate the FIM Bathymetry Adjusted Rating Curve (BARC) tool performance using the estimated bankfull geometry dataset derived for the NWM route link dataset. + +

+## v3.0.16.3 - 2021-05-21 - [PR #388](https://github.com/NOAA-OWP/cahaba/pull/388) + +Enhancement and bug fixes to `synthesize_test_cases.py`. + +## Changes +- Addresses a bug where AHPS sites without benchmark data were receiving a CSI of 0 in the master metrics CSV produced by `synthesize_test_cases.py`. +- Includes a feature enhancement to `synthesize_test_cases.py` that allows for the inclusion of user-specified testing versions in the master metrics CSV. +- Removes some of the print statements used by `synthesize_test_cases.py`. + +

+## v3.0.16.2 - 2021-05-18 - [PR #384](https://github.com/NOAA-OWP/cahaba/pull/384) + +Modifications and fixes to `run_test_case.py`, `eval_plots.py`, and AHPS preprocessing scripts. + +## Changes +- Comment out return statement causing `run_test_case.py` to skip over sites/hucs when calculating contingency rasters. +- Move bad sites list and query statement used to filter out bad sites to the `tools_shared_variables.py`. +- Add print statements in `eval_plots.py` detailing the bad sites used and the query used to filter out bad sites. +- Update AHPS preprocessing scripts to produce a domain shapefile. +- Change output filenames produced in ahps preprocessing scripts. +- Update workarounds for some sites in ahps preprocessing scripts. + +

+## v3.0.16.1 - 2021-05-11 - [PR #380](https://github.com/NOAA-OWP/cahaba/pull/380) + +The current version of Eventlet used in the Connector module of the FIM API is outdated and vulnerable. This update bumps the version to the patched version. + +## Changes +- Updated `api/node/connector/requirements.txt` to have the Eventlet version as 0.31.0 + +

+## v3.0.16.0 - 2021-05-07 - [PR #378](https://github.com/NOAA-OWP/cahaba/pull/378) + +New "Release" feature added to the FIM API. This feature will allow for automated FIM, CatFIM, and relevant metrics to be generated when a new FIM Version is released. See [#373](https://github.com/NOAA-OWP/cahaba/issues/373) for more detailed steps that take place in this feature. + +## Additions +- Added new window to the UI in `api/frontend/gui/templates/index.html`. +- Added new job type to `api/node/connector/connector.py` to allow these release jobs to run. +- Added additional logic in `api/node/updater/updater.py` to run the new eval and CatFIM scripts used in the release feature. + +## Changes +- Updated `api/frontend/output_handler/output_handler.py` to allow for copying more broad ranges of file paths instead of only the `/data/outputs` directory. + +

+## v3.0.15.10 - 2021-05-06 - [PR #375](https://github.com/NOAA-OWP/cahaba/pull/375) +Remove Great Lakes coastlines from WBD buffer. + +## Changes +- `gl_water_polygons.gpkg` layer is used to mask out Great Lakes boundaries and remove NHDPlus HR coastline segments. + +

+## v3.0.15.9 - 2021-05-03 - [PR #372](https://github.com/NOAA-OWP/cahaba/pull/372) + +Generate `nws_lid.gpkg`. + +## Additions +- Generate `nws_lid.gpkg` with attributes indicating if site is a headwater `nws_lid` as well as if it is co-located with another `nws_lid` which is referenced to the same `nwm_feature_id` segment. + +

+## v3.0.15.8 - 2021-04-29 - [PR #371](https://github.com/NOAA-OWP/cahaba/pull/371) + +Refactor NHDPlus HR preprocessing workflow. Resolves issue #238 + +## Changes +- Consolidate NHD streams, NWM catchments, and headwaters MS and FR layers with `mainstem` column. +- HUC8 intersections are included in the input headwaters layer. +- `clip_vectors_to_wbd.py` removes incoming stream segment from the selected layers. + +

+## v3.0.15.7 - 2021-04-28 - [PR #367](https://github.com/NOAA-OWP/cahaba/pull/367) + +Refactor synthesize_test_case.py to handle exceptions during multiprocessing. Resolves issue #351 + +## Changes +- refactored `inundation.py` and `run_test_case.py` to handle exceptions without using `sys.exit()`. + +

+## v3.0.15.6 - 2021-04-23 - [PR #365](https://github.com/NOAA-OWP/cahaba/pull/365) + +Implement CatFIM threshold flows to Sierra test and add AHPS benchmark preprocessing scripts. + +## Additions +- Produce CatFIM flows file when running `rating_curve_get_usgs_gages.py`. +- Several scripts to preprocess AHPS benchmark data. Requires numerous file dependencies not available through Cahaba. + +## Changes +- Modify `rating_curve_comparison.py` to ingest CatFIM threshold flows in calculations. +- Modify `eval_plots.py` to save all site specific bar plots in same parent directory instead of in subdirectories. +- Add variables to `env.template` for AHPS benchmark preprocessing. + +

+## v3.0.15.5 - 2021-04-20 - [PR #363](https://github.com/NOAA-OWP/cahaba/pull/363) + +Prevent eval_plots.py from erroring out when spatial argument enabled if certain datasets not analyzed. + +## Changes +- Add check to make sure analyzed dataset is available prior to creating spatial dataset. + +

+## v3.0.15.4 - 2021-04-20 - [PR #356](https://github.com/NOAA-OWP/cahaba/pull/356) + +Closing all multiprocessing Pool objects in repo. + +

+## v3.0.15.3 - 2021-04-19 - [PR #358](https://github.com/NOAA-OWP/cahaba/pull/358) + +Preprocess NHDPlus HR rasters for consistent projections, nodata values, and convert from cm to meters. + +## Additions +- `preprocess_rasters.py` reprojects raster, converts to meters, and updates nodata value to -9999. +- Cleaned up log messages from `bathy_rc_adjust.py` and `usgs_gage_crosswalk.py`. +- Outputs paths updated in `generate_categorical_fim_mapping.py` and `generate_categorical_fim.py`. +- `update_raster_profile` cleans up raster crs, blocksize, nodata values, and converts elevation grids from cm to meters. +- `reproject_dem.py` imports gdal to reproject elevation rasters because an error was occurring when using rasterio. + +## Changes +- `burn_in_levees.py` replaces the `gdal_calc.py` command to resolve inconsistent outputs with burned in levee values. + +

+## v3.0.15.2 - 2021-04-16 - [PR #359](https://github.com/NOAA-OWP/cahaba/pull/359) + +Hotfix to preserve desired files when production flag used in `fim_run.sh`. + +## Changes + +- Fixed production whitelisted files. + +

## v3.0.15.1 - 2021-04-13 - [PR #355](https://github.com/NOAA-OWP/cahaba/pull/355) -Sierra test considered all USGS gage locations to be mainstems even though many actually occurred with tributaries. This resulted in unrealistic comparisons as incorrect gages were assigned to mainstems segments. This feature branch identifies gages that are on mainstems via attribute field. +Sierra test considered all USGS gage locations to be mainstems even though many actually occurred with tributaries. This resulted in unrealistic comparisons as incorrect gages were assigned to mainstems segments. This feature branch identifies gages that are on mainstems via attribute field. + ## Changes -- Modifies `usgs_gage_crosswalk.py` to filter out gages from the `usgs_gages.gpkg` layer such that for a "MS" run, only consider gages that contain rating curve information (via `curve` attribute) and are also mainstems gages (via `mainstems` attribute). -- Modifies `usgs_gage_crosswalk.py` to filter out gages from the `usgs_gages.gpkg` layer such that for a "FR" run, only consider gages that contain rating curve information (via `curve` attribute) and are not mainstems gages (via `mainstems` attribute). -- Modifies how mainstems segments are determined by using the `nwm_flows_ms.gpkg` as a lookup to determine if the NWM segment specified by WRDS for a gage site is a mainstems gage. +- Modifies `usgs_gage_crosswalk.py` to filter out gages from the `usgs_gages.gpkg` layer such that for a "MS" run, only consider gages that contain rating curve information (via `curve` attribute) and are also mainstems gages (via `mainstems` attribute). +- Modifies `usgs_gage_crosswalk.py` to filter out gages from the `usgs_gages.gpkg` layer such that for a "FR" run, only consider gages that contain rating curve information (via `curve` attribute) and are not mainstems gages (via `mainstems` attribute). +- Modifies how mainstems segments are determined by using the `nwm_flows_ms.gpkg` as a lookup to determine if the NWM segment specified by WRDS for a gage site is a mainstems gage. ## Additions -- Adds a `mainstem` attribute field to `usgs_gages.gpkg` that indicates whether a gage is located on a mainstems river. +- Adds a `mainstem` attribute field to `usgs_gages.gpkg` that indicates whether a gage is located on a mainstems river. - Adds `NWM_FLOWS_MS` variable to the `.env` and `.env.template` files. - Adds the `extent` argument specified by user when running `fim_run.sh` to `usgs_gage_crosswalk.py`.

- ## v3.0.15.0 - 2021-04-08 - [PR #340](https://github.com/NOAA-OWP/cahaba/pull/340) Implementing a prototype technique to estimate the missing bathymetric component in the HAND-derived synthetic rating curves. The new Bathymetric Adjusted Rating Curve (BARC) function is built within the `fim_run.sh` workflow and will ingest bankfull geometry estimates provided by the user to modify the cross section area used in the synthetic rating curve generation. @@ -33,6 +237,7 @@ Implementing a prototype technique to estimate the missing bathymetric component - Imports the existing synthetic rating curve table and the bankfull geometry input data (topwidth and cross section area per COMID). - Performs new synthetic rating curve calculations with bathymetry estimation modifications. - Flags issues with the thalweg-notch artifact. +

## v3.0.14.0 - 2021-04-05 - [PR #338](https://github.com/NOAA-OWP/cahaba/pull/338) @@ -49,6 +254,7 @@ Create tool to retrieve rating curves from USGS sites and convert to elevation ( 1) `usgs_rating_curves.csv`: A csv file that contains rating curves (including converted to NAVD88 elevation) for USGS gages in a format that is compatible with `rating_curve_comparisons.py`. As it is is currently configured, only gages within CONUS will have rating curve data. 2) `log.csv`: A log file that records status for each gage and includes error messages. 3) `usgs_gages.gpkg`: A geospatial layer (in FIM projection) of all active USGS gages that meet a predefined criteria. Additionally, the `curve` attribute indicates whether a rating curve is found in the `usgs_rating_curves.csv`. This spatial file is only generated if the `all` option is passed with the `-l` argument. +

## v3.0.13.0 - 2021-04-01 - [PR #332](https://github.com/NOAA-OWP/cahaba/pull/332) @@ -62,8 +268,8 @@ Created tool to compare synthetic rating curve with benchmark rating curve (Sier ### Additions - `usgs_gage_crosswalk.py`: generates `usgs_elev_table.csv` in `run_by_unit.py` with elevation and additional attributes at USGS gages. - `rating_curve_comparison.py`: post-processing script to plot and calculate metrics between synthetic rating curves and USGS rating curve data. -

+

## v3.0.12.1 - 2021-03-31 - [PR #336](https://github.com/NOAA-OWP/cahaba/pull/336) Fix spatial option in `eval_plots.py` when creating plots and spatial outputs. @@ -76,8 +282,8 @@ Fix spatial option in `eval_plots.py` when creating plots and spatial outputs. ### Additions - Creates `fim_performance_points.shp`: this layer consists of all evaluated ahps points (with metrics). Spatial data retrieved from WRDS on the fly. - Creates `fim_performance_polys.shp`: this layer consists of all evaluated huc8s (with metrics). Spatial data retrieved from WBD layer. -

+

## v3.0.12.0 - 2021-03-26 - [PR #327](https://github.com/NOAA-OWP/cahaba/pull/237) Add more detail/information to plotting capabilities. @@ -89,8 +295,8 @@ Add more detail/information to plotting capabilities. ### Additions - Optional argument to create barplots of CSI for each individual site. - Create a csv containing the data used to create the scatterplots. -

+

## v3.0.11.0 - 2021-03-22 - [PR #319](https://github.com/NOAA-OWP/cahaba/pull/298) Improvements to CatFIM service source data generation. @@ -103,16 +309,16 @@ Improvements to CatFIM service source data generation. ### Additions - Added `generate_categorical_fim.py` to wrap `generate_categorical_fim_flows.py` and `generate_categorical_fim_mapping.py`. - Create new `nws_lid_sites` shapefile located in same directory as the `catfim_library` shapefile. -

+

## v3.0.10.1 - 2021-03-24 - [PR #320](https://github.com/NOAA-OWP/cahaba/pull/320) Patch to synthesize_test_cases.py. ### Changes - Bug fix to `synthesize_test_cases.py` to allow comparison between `testing` version and `official` versions. -

+

## v3.0.10.0 - 2021-03-12 - [PR #298](https://github.com/NOAA-OWP/cahaba/pull/298) Preprocessing of flow files for Categorical FIM. @@ -126,8 +332,8 @@ Preprocessing of flow files for Categorical FIM. ### Changes - Stability fixes to `generate_categorical_fim.py`. -

+

## v3.0.9.0 - 2021-03-12 - [PR #297](https://github.com/NOAA-OWP/cahaba/pull/297) Enhancements to FIM API. diff --git a/Pipfile b/Pipfile index 219f69ba2..48f8da27d 100755 --- a/Pipfile +++ b/Pipfile @@ -21,6 +21,7 @@ grass-session = "==0.5" seaborn = "==0.11.0" python-dotenv = "*" natsort = "*" +memory-profiler = "*" [requires] python_version = "3.8" diff --git a/Pipfile.lock b/Pipfile.lock index 1250dcc4b..e1db9681c 100644 --- a/Pipfile.lock +++ b/Pipfile.lock @@ -1,7 +1,7 @@ { "_meta": { "hash": { - "sha256": "f74c6d08c7c900c6cea86a5ea4661791356aac573ed738c9c3a1405303a4200c" + "sha256": "dbdc717b9aea8c37d6aecaa1b57bf669689c63758542bea1d47bb4e02fddc663" }, "pipfile-spec": 6, "requires": { @@ -25,18 +25,18 @@ }, "attrs": { "hashes": [ - "sha256:31b2eced602aa8423c2aea9c76a724617ed67cf9513173fd3a4f03e3a929c7e6", - "sha256:832aa3cde19744e49938b91fea06d69ecb9e649c93ba974535d08ad92164f700" + "sha256:149e90d6d8ac20db7a955ad60cf0e6881a3f20d37096140088356da6c716b0b1", + "sha256:ef6aaac3ca6cd92904cdd0d83f629a15f18053ec84e6432106f7a4d04ae4f5fb" ], - "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", - "version": "==20.3.0" + "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4'", + "version": "==21.2.0" }, "certifi": { "hashes": [ - "sha256:1a4995114262bffbc2413b159f2a1a480c969de6e6eb13ee966d470af86af59c", - "sha256:719a74fb9e33b9bd44cc7f3a8d94bc35e4049deebe19ba7d8e108280cfd59830" + "sha256:2bbf76fd432960138b3ef6dda3dde0544f27cbf8546c458e60baf371917ba9ee", + "sha256:50b1e4f8446b06f41be7dd6338db18e0990601dce795c2b1686458aa7e8fa7d8" ], - "version": "==2020.12.5" + "version": "==2021.5.30" }, "click": { "hashes": [ @@ -55,11 +55,11 @@ }, "cligj": { "hashes": [ - "sha256:07171c1e287f45511f97df4ea071abc5d19924153413d5683a8e4866369bc676", - "sha256:b2f1f7247d59a5387bd3013a08b9ed6829e96fafa4a6e6292341efdb46fe6220" + "sha256:a4bc13d623356b373c2c27c53dbd9c68cae5d526270bfa71f6c6fa69669c6b27", + "sha256:c1ca117dbce1fe20a5809dc96f01e1c2840f6dcc939b3ddbb1111bf330ba82df" ], "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3' and python_version < '4'", - "version": "==0.7.1" + "version": "==0.7.2" }, "cycler": { "hashes": [ @@ -162,34 +162,35 @@ }, "matplotlib": { "hashes": [ - "sha256:1de0bb6cbfe460725f0e97b88daa8643bcf9571c18ba90bb8e41432aaeca91d6", - "sha256:1e850163579a8936eede29fad41e202b25923a0a8d5ffd08ce50fc0a97dcdc93", - "sha256:215e2a30a2090221a9481db58b770ce56b8ef46f13224ae33afe221b14b24dc1", - "sha256:348e6032f666ffd151b323342f9278b16b95d4a75dfacae84a11d2829a7816ae", - "sha256:3d2eb9c1cc254d0ffa90bc96fde4b6005d09c2228f99dfd493a4219c1af99644", - "sha256:3e477db76c22929e4c6876c44f88d790aacdf3c3f8f3a90cb1975c0bf37825b0", - "sha256:451cc89cb33d6652c509fc6b588dc51c41d7246afdcc29b8624e256b7663ed1f", - "sha256:46b1a60a04e6d884f0250d5cc8dc7bd21a9a96c584a7acdaab44698a44710bab", - "sha256:5f571b92a536206f7958f7cb2d367ff6c9a1fa8229dc35020006e4cdd1ca0acd", - "sha256:672960dd114e342b7c610bf32fb99d14227f29919894388b41553217457ba7ef", - "sha256:7310e353a4a35477c7f032409966920197d7df3e757c7624fd842f3eeb307d3d", - "sha256:746a1df55749629e26af7f977ea426817ca9370ad1569436608dc48d1069b87c", - "sha256:7c155437ae4fd366e2700e2716564d1787700687443de46bcb895fe0f84b761d", - "sha256:9265ae0fb35e29f9b8cc86c2ab0a2e3dcddc4dd9de4b85bf26c0f63fe5c1c2ca", - "sha256:94bdd1d55c20e764d8aea9d471d2ae7a7b2c84445e0fa463f02e20f9730783e1", - "sha256:9a79e5dd7bb797aa611048f5b70588b23c5be05b63eefd8a0d152ac77c4243db", - "sha256:a17f0a10604fac7627ec82820439e7db611722e80c408a726cd00d8c974c2fb3", - "sha256:a1acb72f095f1d58ecc2538ed1b8bca0b57df313b13db36ed34b8cdf1868e674", - "sha256:aa49571d8030ad0b9ac39708ee77bd2a22f87815e12bdee52ecaffece9313ed8", - "sha256:c24c05f645aef776e8b8931cb81e0f1632d229b42b6d216e30836e2e145a2b40", - "sha256:cf3a7e54eff792f0815dbbe9b85df2f13d739289c93d346925554f71d484be78", - "sha256:d738acfdfb65da34c91acbdb56abed46803db39af259b7f194dc96920360dbe4", - "sha256:e15fa23d844d54e7b3b7243afd53b7567ee71c721f592deb0727ee85e668f96a", - "sha256:ed4a9e6dcacba56b17a0a9ac22ae2c72a35b7f0ef0693aa68574f0b2df607a89", - "sha256:f44149a0ef5b4991aaef12a93b8e8d66d6412e762745fea1faa61d98524e0ba9" + "sha256:0bea5ec5c28d49020e5d7923c2725b837e60bc8be99d3164af410eb4b4c827da", + "sha256:1c1779f7ab7d8bdb7d4c605e6ffaa0614b3e80f1e3c8ccf7b9269a22dbc5986b", + "sha256:21b31057bbc5e75b08e70a43cefc4c0b2c2f1b1a850f4a0f7af044eb4163086c", + "sha256:32fa638cc10886885d1ca3d409d4473d6a22f7ceecd11322150961a70fab66dd", + "sha256:3a5c18dbd2c7c366da26a4ad1462fe3e03a577b39e3b503bbcf482b9cdac093c", + "sha256:5826f56055b9b1c80fef82e326097e34dc4af8c7249226b7dd63095a686177d1", + "sha256:6382bc6e2d7e481bcd977eb131c31dee96e0fb4f9177d15ec6fb976d3b9ace1a", + "sha256:6475d0209024a77f869163ec3657c47fed35d9b6ed8bccba8aa0f0099fbbdaa8", + "sha256:6a6a44f27aabe720ec4fd485061e8a35784c2b9ffa6363ad546316dfc9cea04e", + "sha256:7a58f3d8fe8fac3be522c79d921c9b86e090a59637cb88e3bc51298d7a2c862a", + "sha256:7ad19f3fb6145b9eb41c08e7cbb9f8e10b91291396bee21e9ce761bb78df63ec", + "sha256:85f191bb03cb1a7b04b5c2cca4792bef94df06ef473bc49e2818105671766fee", + "sha256:956c8849b134b4a343598305a3ca1bdd3094f01f5efc8afccdebeffe6b315247", + "sha256:a9d8cb5329df13e0cdaa14b3b43f47b5e593ec637f13f14db75bb16e46178b05", + "sha256:b1d5a2cedf5de05567c441b3a8c2651fbde56df08b82640e7f06c8cd91e201f6", + "sha256:b26535b9de85326e6958cdef720ecd10bcf74a3f4371bf9a7e5b2e659c17e153", + "sha256:c541ee5a3287efe066bbe358320853cf4916bc14c00c38f8f3d8d75275a405a9", + "sha256:d8d994cefdff9aaba45166eb3de4f5211adb4accac85cbf97137e98f26ea0219", + "sha256:df815378a754a7edd4559f8c51fc7064f779a74013644a7f5ac7a0c31f875866" ], - "markers": "python_version >= '3.6'", - "version": "==3.3.4" + "markers": "python_version >= '3.7'", + "version": "==3.4.2" + }, + "memory-profiler": { + "hashes": [ + "sha256:01385ac0fec944fcf7969814ec4406c6d8a9c66c079d09276723c5a7680f44e5" + ], + "index": "pypi", + "version": "==0.58.0" }, "munch": { "hashes": [ @@ -233,33 +234,33 @@ }, "numpy": { "hashes": [ - "sha256:032be656d89bbf786d743fee11d01ef318b0781281241997558fa7950028dd29", - "sha256:104f5e90b143dbf298361a99ac1af4cf59131218a045ebf4ee5990b83cff5fab", - "sha256:125a0e10ddd99a874fd357bfa1b636cd58deb78ba4a30b5ddb09f645c3512e04", - "sha256:12e4ba5c6420917571f1a5becc9338abbde71dd811ce40b37ba62dec7b39af6d", - "sha256:13adf545732bb23a796914fe5f891a12bd74cf3d2986eed7b7eba2941eea1590", - "sha256:2d7e27442599104ee08f4faed56bb87c55f8b10a5494ac2ead5c98a4b289e61f", - "sha256:3bc63486a870294683980d76ec1e3efc786295ae00128f9ea38e2c6e74d5a60a", - "sha256:3d3087e24e354c18fb35c454026af3ed8997cfd4997765266897c68d724e4845", - "sha256:4ed8e96dc146e12c1c5cdd6fb9fd0757f2ba66048bf94c5126b7efebd12d0090", - "sha256:60759ab15c94dd0e1ed88241fd4fa3312db4e91d2c8f5a2d4cf3863fad83d65b", - "sha256:65410c7f4398a0047eea5cca9b74009ea61178efd78d1be9847fac1d6716ec1e", - "sha256:66b467adfcf628f66ea4ac6430ded0614f5cc06ba530d09571ea404789064adc", - "sha256:7199109fa46277be503393be9250b983f325880766f847885607d9b13848f257", - "sha256:72251e43ac426ff98ea802a931922c79b8d7596480300eb9f1b1e45e0543571e", - "sha256:89e5336f2bec0c726ac7e7cdae181b325a9c0ee24e604704ed830d241c5e47ff", - "sha256:89f937b13b8dd17b0099c7c2e22066883c86ca1575a975f754babc8fbf8d69a9", - "sha256:9c94cab5054bad82a70b2e77741271790304651d584e2cdfe2041488e753863b", - "sha256:9eb551d122fadca7774b97db8a112b77231dcccda8e91a5bc99e79890797175e", - "sha256:a1d7995d1023335e67fb070b2fae6f5968f5be3802b15ad6d79d81ecaa014fe0", - "sha256:ae61f02b84a0211abb56462a3b6cd1e7ec39d466d3160eb4e1da8bf6717cdbeb", - "sha256:b9410c0b6fed4a22554f072a86c361e417f0258838957b78bd063bde2c7f841f", - "sha256:c26287dfc888cf1e65181f39ea75e11f42ffc4f4529e5bd19add57ad458996e2", - "sha256:c91ec9569facd4757ade0888371eced2ecf49e7982ce5634cc2cf4e7331a4b14", - "sha256:ecb5b74c702358cdc21268ff4c37f7466357871f53a30e6f84c686952bef16a9" + "sha256:1676b0a292dd3c99e49305a16d7a9f42a4ab60ec522eac0d3dd20cdf362ac010", + "sha256:16f221035e8bd19b9dc9a57159e38d2dd060b48e93e1d843c49cb370b0f415fd", + "sha256:43909c8bb289c382170e0282158a38cf306a8ad2ff6dfadc447e90f9961bef43", + "sha256:4e465afc3b96dbc80cf4a5273e5e2b1e3451286361b4af70ce1adb2984d392f9", + "sha256:55b745fca0a5ab738647d0e4db099bd0a23279c32b31a783ad2ccea729e632df", + "sha256:5d050e1e4bc9ddb8656d7b4f414557720ddcca23a5b88dd7cff65e847864c400", + "sha256:637d827248f447e63585ca3f4a7d2dfaa882e094df6cfa177cc9cf9cd6cdf6d2", + "sha256:6690080810f77485667bfbff4f69d717c3be25e5b11bb2073e76bb3f578d99b4", + "sha256:66fbc6fed94a13b9801fb70b96ff30605ab0a123e775a5e7a26938b717c5d71a", + "sha256:67d44acb72c31a97a3d5d33d103ab06d8ac20770e1c5ad81bdb3f0c086a56cf6", + "sha256:6ca2b85a5997dabc38301a22ee43c82adcb53ff660b89ee88dded6b33687e1d8", + "sha256:6e51534e78d14b4a009a062641f465cfaba4fdcb046c3ac0b1f61dd97c861b1b", + "sha256:70eb5808127284c4e5c9e836208e09d685a7978b6a216db85960b1a112eeace8", + "sha256:830b044f4e64a76ba71448fce6e604c0fc47a0e54d8f6467be23749ac2cbd2fb", + "sha256:8b7bb4b9280da3b2856cb1fc425932f46fba609819ee1c62256f61799e6a51d2", + "sha256:a9c65473ebc342715cb2d7926ff1e202c26376c0dcaaee85a1fd4b8d8c1d3b2f", + "sha256:c1c09247ccea742525bdb5f4b5ceeacb34f95731647fe55774aa36557dbb5fa4", + "sha256:c5bf0e132acf7557fc9bb8ded8b53bbbbea8892f3c9a1738205878ca9434206a", + "sha256:db250fd3e90117e0312b611574cd1b3f78bec046783195075cbd7ba9c3d73f16", + "sha256:e515c9a93aebe27166ec9593411c58494fa98e5fcc219e47260d9ab8a1cc7f9f", + "sha256:e55185e51b18d788e49fe8305fd73ef4470596b33fc2c1ceb304566b99c71a69", + "sha256:ea9cff01e75a956dbee133fa8e5b68f2f92175233de2f88de3a682dd94deda65", + "sha256:f1452578d0516283c87608a5a5548b0cdde15b99650efdfd85182102ef7a7c17", + "sha256:f39a995e47cb8649673cfa0579fbdd1cdd33ea497d1728a6cb194d6252268e48" ], "markers": "python_version >= '3.7'", - "version": "==1.20.1" + "version": "==1.20.3" }, "pandas": { "hashes": [ @@ -285,42 +286,77 @@ }, "pillow": { "hashes": [ - "sha256:15306d71a1e96d7e271fd2a0737038b5a92ca2978d2e38b6ced7966583e3d5af", - "sha256:1940fc4d361f9cc7e558d6f56ff38d7351b53052fd7911f4b60cd7bc091ea3b1", - "sha256:1f93f2fe211f1ef75e6f589327f4d4f8545d5c8e826231b042b483d8383e8a7c", - "sha256:30d33a1a6400132e6f521640dd3f64578ac9bfb79a619416d7e8802b4ce1dd55", - "sha256:328240f7dddf77783e72d5ed79899a6b48bc6681f8d1f6001f55933cb4905060", - "sha256:46c2bcf8e1e75d154e78417b3e3c64e96def738c2a25435e74909e127a8cba5e", - "sha256:5762ebb4436f46b566fc6351d67a9b5386b5e5de4e58fdaa18a1c83e0e20f1a8", - "sha256:5a2d957eb4aba9d48170b8fe6538ec1fbc2119ffe6373782c03d8acad3323f2e", - "sha256:5cf03b9534aca63b192856aa601c68d0764810857786ea5da652581f3a44c2b0", - "sha256:5daba2b40782c1c5157a788ec4454067c6616f5a0c1b70e26ac326a880c2d328", - "sha256:63cd413ac52ee3f67057223d363f4f82ce966e64906aea046daf46695e3c8238", - "sha256:6efac40344d8f668b6c4533ae02a48d52fd852ef0654cc6f19f6ac146399c733", - "sha256:71b01ee69e7df527439d7752a2ce8fb89e19a32df484a308eca3e81f673d3a03", - "sha256:71f31ee4df3d5e0b366dd362007740106d3210fb6a56ec4b581a5324ba254f06", - "sha256:72027ebf682abc9bafd93b43edc44279f641e8996fb2945104471419113cfc71", - "sha256:74cd9aa648ed6dd25e572453eb09b08817a1e3d9f8d1bd4d8403d99e42ea790b", - "sha256:81b3716cc9744ffdf76b39afb6247eae754186838cedad0b0ac63b2571253fe6", - "sha256:8565355a29655b28fdc2c666fd9a3890fe5edc6639d128814fafecfae2d70910", - "sha256:87f42c976f91ca2fc21a3293e25bd3cd895918597db1b95b93cbd949f7d019ce", - "sha256:89e4c757a91b8c55d97c91fa09c69b3677c227b942fa749e9a66eef602f59c28", - "sha256:8c4e32218c764bc27fe49b7328195579581aa419920edcc321c4cb877c65258d", - "sha256:903293320efe2466c1ab3509a33d6b866dc850cfd0c5d9cc92632014cec185fb", - "sha256:90882c6f084ef68b71bba190209a734bf90abb82ab5e8f64444c71d5974008c6", - "sha256:98afcac3205d31ab6a10c5006b0cf040d0026a68ec051edd3517b776c1d78b09", - "sha256:a01da2c266d9868c4f91a9c6faf47a251f23b9a862dce81d2ff583135206f5be", - "sha256:aeab4cd016e11e7aa5cfc49dcff8e51561fa64818a0be86efa82c7038e9369d0", - "sha256:b07c660e014852d98a00a91adfbe25033898a9d90a8f39beb2437d22a203fc44", - "sha256:bead24c0ae3f1f6afcb915a057943ccf65fc755d11a1410a909c1fefb6c06ad1", - "sha256:d1d6bca39bb6dd94fba23cdb3eeaea5e30c7717c5343004d900e2a63b132c341", - "sha256:e2cd8ac157c1e5ae88b6dd790648ee5d2777e76f1e5c7d184eaddb2938594f34", - "sha256:e5739ae63636a52b706a0facec77b2b58e485637e1638202556156e424a02dc2", - "sha256:f36c3ff63d6fc509ce599a2f5b0d0732189eed653420e7294c039d342c6e204a", - "sha256:f91b50ad88048d795c0ad004abbe1390aa1882073b1dca10bfd55d0b8cf18ec5" + "sha256:01425106e4e8cee195a411f729cff2a7d61813b0b11737c12bd5991f5f14bcd5", + "sha256:031a6c88c77d08aab84fecc05c3cde8414cd6f8406f4d2b16fed1e97634cc8a4", + "sha256:083781abd261bdabf090ad07bb69f8f5599943ddb539d64497ed021b2a67e5a9", + "sha256:0d19d70ee7c2ba97631bae1e7d4725cdb2ecf238178096e8c82ee481e189168a", + "sha256:0e04d61f0064b545b989126197930807c86bcbd4534d39168f4aa5fda39bb8f9", + "sha256:12e5e7471f9b637762453da74e390e56cc43e486a88289995c1f4c1dc0bfe727", + "sha256:22fd0f42ad15dfdde6c581347eaa4adb9a6fc4b865f90b23378aa7914895e120", + "sha256:238c197fc275b475e87c1453b05b467d2d02c2915fdfdd4af126145ff2e4610c", + "sha256:3b570f84a6161cf8865c4e08adf629441f56e32f180f7aa4ccbd2e0a5a02cba2", + "sha256:463822e2f0d81459e113372a168f2ff59723e78528f91f0bd25680ac185cf797", + "sha256:4d98abdd6b1e3bf1a1cbb14c3895226816e666749ac040c4e2554231068c639b", + "sha256:5afe6b237a0b81bd54b53f835a153770802f164c5570bab5e005aad693dab87f", + "sha256:5b70110acb39f3aff6b74cf09bb4169b167e2660dabc304c1e25b6555fa781ef", + "sha256:5cbf3e3b1014dddc45496e8cf38b9f099c95a326275885199f427825c6522232", + "sha256:624b977355cde8b065f6d51b98497d6cd5fbdd4f36405f7a8790e3376125e2bb", + "sha256:63728564c1410d99e6d1ae8e3b810fe012bc440952168af0a2877e8ff5ab96b9", + "sha256:66cc56579fd91f517290ab02c51e3a80f581aba45fd924fcdee01fa06e635812", + "sha256:6c32cc3145928c4305d142ebec682419a6c0a8ce9e33db900027ddca1ec39178", + "sha256:8b56553c0345ad6dcb2e9b433ae47d67f95fc23fe28a0bde15a120f25257e291", + "sha256:8bb1e155a74e1bfbacd84555ea62fa21c58e0b4e7e6b20e4447b8d07990ac78b", + "sha256:95d5ef984eff897850f3a83883363da64aae1000e79cb3c321915468e8c6add5", + "sha256:a013cbe25d20c2e0c4e85a9daf438f85121a4d0344ddc76e33fd7e3965d9af4b", + "sha256:a787ab10d7bb5494e5f76536ac460741788f1fbce851068d73a87ca7c35fc3e1", + "sha256:a7d5e9fad90eff8f6f6106d3b98b553a88b6f976e51fce287192a5d2d5363713", + "sha256:aac00e4bc94d1b7813fe882c28990c1bc2f9d0e1aa765a5f2b516e8a6a16a9e4", + "sha256:b91c36492a4bbb1ee855b7d16fe51379e5f96b85692dc8210831fbb24c43e484", + "sha256:c03c07ed32c5324939b19e36ae5f75c660c81461e312a41aea30acdd46f93a7c", + "sha256:c5236606e8570542ed424849f7852a0ff0bce2c4c8d0ba05cc202a5a9c97dee9", + "sha256:c6b39294464b03457f9064e98c124e09008b35a62e3189d3513e5148611c9388", + "sha256:cb7a09e173903541fa888ba010c345893cd9fc1b5891aaf060f6ca77b6a3722d", + "sha256:d68cb92c408261f806b15923834203f024110a2e2872ecb0bd2a110f89d3c602", + "sha256:dc38f57d8f20f06dd7c3161c59ca2c86893632623f33a42d592f097b00f720a9", + "sha256:e98eca29a05913e82177b3ba3d198b1728e164869c613d76d0de4bde6768a50e", + "sha256:f217c3954ce5fd88303fc0c317af55d5e0204106d86dea17eb8205700d47dec2" ], "markers": "python_version >= '3.6'", - "version": "==8.1.2" + "version": "==8.2.0" + }, + "psutil": { + "hashes": [ + "sha256:0066a82f7b1b37d334e68697faba68e5ad5e858279fd6351c8ca6024e8d6ba64", + "sha256:02b8292609b1f7fcb34173b25e48d0da8667bc85f81d7476584d889c6e0f2131", + "sha256:0ae6f386d8d297177fd288be6e8d1afc05966878704dad9847719650e44fc49c", + "sha256:0c9ccb99ab76025f2f0bbecf341d4656e9c1351db8cc8a03ccd62e318ab4b5c6", + "sha256:0dd4465a039d343925cdc29023bb6960ccf4e74a65ad53e768403746a9207023", + "sha256:12d844996d6c2b1d3881cfa6fa201fd635971869a9da945cf6756105af73d2df", + "sha256:1bff0d07e76114ec24ee32e7f7f8d0c4b0514b3fae93e3d2aaafd65d22502394", + "sha256:245b5509968ac0bd179287d91210cd3f37add77dad385ef238b275bad35fa1c4", + "sha256:28ff7c95293ae74bf1ca1a79e8805fcde005c18a122ca983abf676ea3466362b", + "sha256:36b3b6c9e2a34b7d7fbae330a85bf72c30b1c827a4366a07443fc4b6270449e2", + "sha256:52de075468cd394ac98c66f9ca33b2f54ae1d9bff1ef6b67a212ee8f639ec06d", + "sha256:5da29e394bdedd9144c7331192e20c1f79283fb03b06e6abd3a8ae45ffecee65", + "sha256:61f05864b42fedc0771d6d8e49c35f07efd209ade09a5afe6a5059e7bb7bf83d", + "sha256:6223d07a1ae93f86451d0198a0c361032c4c93ebd4bf6d25e2fb3edfad9571ef", + "sha256:6323d5d845c2785efb20aded4726636546b26d3b577aded22492908f7c1bdda7", + "sha256:6ffe81843131ee0ffa02c317186ed1e759a145267d54fdef1bc4ea5f5931ab60", + "sha256:74f2d0be88db96ada78756cb3a3e1b107ce8ab79f65aa885f76d7664e56928f6", + "sha256:74fb2557d1430fff18ff0d72613c5ca30c45cdbfcddd6a5773e9fc1fe9364be8", + "sha256:90d4091c2d30ddd0a03e0b97e6a33a48628469b99585e2ad6bf21f17423b112b", + "sha256:90f31c34d25b1b3ed6c40cdd34ff122b1887a825297c017e4cbd6796dd8b672d", + "sha256:99de3e8739258b3c3e8669cb9757c9a861b2a25ad0955f8e53ac662d66de61ac", + "sha256:c6a5fd10ce6b6344e616cf01cc5b849fa8103fbb5ba507b6b2dee4c11e84c935", + "sha256:ce8b867423291cb65cfc6d9c4955ee9bfc1e21fe03bb50e177f2b957f1c2469d", + "sha256:d225cd8319aa1d3c85bf195c4e07d17d3cd68636b8fc97e6cf198f782f99af28", + "sha256:ea313bb02e5e25224e518e4352af4bf5e062755160f77e4b1767dd5ccb65f876", + "sha256:ea372bcc129394485824ae3e3ddabe67dc0b118d262c568b4d2602a7070afdb0", + "sha256:f4634b033faf0d968bb9220dd1c793b897ab7f1189956e1aa9eae752527127d3", + "sha256:fcc01e900c1d7bee2a37e5d6e4f9194760a93597c97fee89c4ae51701de03563" + ], + "markers": "python_version >= '2.6' and python_version not in '3.0, 3.1, 3.2, 3.3'", + "version": "==5.8.0" }, "pygeos": { "hashes": [ @@ -354,34 +390,28 @@ }, "pyproj": { "hashes": [ - "sha256:09bead60769e69b592e8cb3ac51b5215f75e9bb9c213ce575031961deb48d6da", - "sha256:09db64a8088b23f001e574d92bcc3080bf7de44ddca152d0282a2b50c918a64a", - "sha256:1be7d54900eb7e2d1e637319080b3a047c70d1fb2f3c12d3400c0fa8a90cf440", - "sha256:36ba436675f9dea4ab3db7d9a32d3ff11c2fbb4d6690a83454d2f3c5c0b54041", - "sha256:3e7e851e6d58c16ac2cd920a1bacb7fbb24758a6fcd7f234d594a88ebae04ec9", - "sha256:489a96da87d8846c34c90da90e637544e4f4f50a13589b5aac54297f5ee1b01d", - "sha256:4a333f3e46fe8b2eb4647a3daa3a2cec52ddc6c107c653b45880526114942ee8", - "sha256:708d6e01b9ff3d6dc62a5ad2d2ba1264a863eaa657c1a9bf713a10cc35d34553", - "sha256:7ae8e7052f18fde1884574da449010e94fa205ad27aeeaa34a097f49a1ed6a2b", - "sha256:7bfaa34e8bb0510d4380310374deecd9e4328b9cf556925cfb45b5a94d5bbdbe", - "sha256:81c06df20d09d621e52791c19ce3c880695fb430061e59c2472fa5467e890391", - "sha256:86ef2fcd584a3222bf73e2befc24b2badd139b3371f4a1e88649978ef7649540", - "sha256:9666d01faf4e758ac68f2c16695c90de49c3170e3760988bf76a34aae11f4e15", - "sha256:9b845510255f9580d7e226dd3321a51c468cefb7be24e46415caf67caa4287c4", - "sha256:9e2ef75401f17062166d3fe53c555cd62c9577697a2f5ded916b23c54e5db497", - "sha256:a3805e026a5547be205a5e322c08e3069f0a48c63bbd53dbc7a8e3499bc66d58", - "sha256:a3a8ab19232bf4f4bb2590536538881b7bd0c07df23e0c2a792402ca2476c197", - "sha256:aa0a2981b25145523ca17a643c5be077fe13e514fdca9b6d1c412a95d723a5a5", - "sha256:bc70b6adcfa713d89bc561673cb57af5fb3a1718cd7d57ec537430cd1007a864", - "sha256:bfbac35490dd17f706700673506eeb8170f8a2a63fb5878171d4e6eef242d141", - "sha256:c658afc8a6115b58b02aa53d27bf2a67c1b00b55067edb1b7711c6c7391cfaa9", - "sha256:cba99e171d744969e13a865ad28fa9c949c4400b0e9c431a802cdd804f52f632", - "sha256:d27d40ec541ef69a5107bfcd85f40170e9e122ceb6315ce508cd44d199983d41", - "sha256:f942a976ea3de6a519cf48be30a12f465e44d0ac0c38a0d820ab3acfcc0a48a6", - "sha256:fee7517bd389a1db7b8bebb18838d04dedca9eaacda01d353d98f5ee421f263e" + "sha256:04c185102e659439c5bd428ac5473d36ef795fca8e225bbbe78e20643d804ec0", + "sha256:10dad599b9f7ce2194996dc25f1000e0aa15754ecef9db46b624713959c67957", + "sha256:1e88ebc4e08e661e9011b5c1ebfb32f0d311963a9824a6effb4168c7e07918b1", + "sha256:4f3ad09cf3352bf5664794042b28d98781362ec8d9774ad73f28a1a0101a27f1", + "sha256:5f8a8d982bde211e65dc2de1f8f36cf162f9cc7fcd8a7625046ea265284e5e65", + "sha256:67b94f4e694ae33fc90dfb7da0e6b5ed5f671dd0acc2f6cf46e9c39d56e16e1a", + "sha256:808f5992320e9631b2e45444028a65cd6ba3ee40229292934178ef07020a5ffd", + "sha256:8eda240225971b5cd0bac2d399ed6222068f0598ee92d5f6e847bd2019d2c8b0", + "sha256:911d773da9fa4d4f3f7580173858c391e3ee0b61acaf0be303baab323d2eae78", + "sha256:9cc464a1c51baad28ffb7a233116e8d4ce4c560b32039fa986d0f992ac3c431f", + "sha256:a162ed199cd2ec392cffe20b2fa3381b68e7a166d55f3f060eceb8d517e4f46d", + "sha256:aa87df0982aa0f4477478899d9c930cc0f97cd6d8a4ce84c43ac88ccf86d1da7", + "sha256:ae237492767e0225f99b53a0fd7110fde2b7e7cabc105bbc243c151a7497de88", + "sha256:ae5534fa7a3b74f20534694d297fce6f7483890ff6ca404394ecf372f3c589d4", + "sha256:b635e7e21fea5af74e90fc9e54d1a4c27078efdce6f214101c98dd93afae599a", + "sha256:b6c74bbec679199746a3e02c0e0fad093c3652df96dd63e086a2fbf2afe9dc0e", + "sha256:c4193e1069d165476b2d0f7d882b7712b3eab6e2e6fe2a0a78ef40de825a1f28", + "sha256:da88abc5e2f6a8fb07533855a57ca2a31845f58901a87f821b68b0db6b023978", + "sha256:ebbba7707fe83a01e54bce8e3e7342feb0b3e0d74ff8c28df12f8bc59b76827c" ], - "markers": "python_version >= '3.6'", - "version": "==3.0.1" + "markers": "python_version >= '3.7'", + "version": "==3.1.0" }, "python-dateutil": { "hashes": [ @@ -393,11 +423,11 @@ }, "python-dotenv": { "hashes": [ - "sha256:0c8d1b80d1a1e91717ea7d526178e3882732420b03f08afea0406db6402e220e", - "sha256:587825ed60b1711daea4832cf37524dfd404325b7db5e25ebe88c495c9f807a0" + "sha256:00aa34e92d992e9f8383730816359647f358f4a3be1ba45e5a5cefd27ee91544", + "sha256:b1ae5e9643d5ed987fc57cc2583021e38db531946518130777734f9589b3141f" ], "index": "pypi", - "version": "==0.15.0" + "version": "==0.17.1" }, "pytz": { "hashes": [ @@ -451,28 +481,28 @@ }, "scipy": { "hashes": [ - "sha256:0c8a51d33556bf70367452d4d601d1742c0e806cd0194785914daf19775f0e67", - "sha256:0e5b0ccf63155d90da576edd2768b66fb276446c371b73841e3503be1d63fb5d", - "sha256:2481efbb3740977e3c831edfd0bd9867be26387cacf24eb5e366a6a374d3d00d", - "sha256:33d6b7df40d197bdd3049d64e8e680227151673465e5d85723b3b8f6b15a6ced", - "sha256:5da5471aed911fe7e52b86bf9ea32fb55ae93e2f0fac66c32e58897cfb02fa07", - "sha256:5f331eeed0297232d2e6eea51b54e8278ed8bb10b099f69c44e2558c090d06bf", - "sha256:5fa9c6530b1661f1370bcd332a1e62ca7881785cc0f80c0d559b636567fab63c", - "sha256:6725e3fbb47da428794f243864f2297462e9ee448297c93ed1dcbc44335feb78", - "sha256:68cb4c424112cd4be886b4d979c5497fba190714085f46b8ae67a5e4416c32b4", - "sha256:794e768cc5f779736593046c9714e0f3a5940bc6dcc1dba885ad64cbfb28e9f0", - "sha256:83bf7c16245c15bc58ee76c5418e46ea1811edcc2e2b03041b804e46084ab627", - "sha256:8e403a337749ed40af60e537cc4d4c03febddcc56cd26e774c9b1b600a70d3e4", - "sha256:a15a1f3fc0abff33e792d6049161b7795909b40b97c6cc2934ed54384017ab76", - "sha256:a423533c55fec61456dedee7b6ee7dce0bb6bfa395424ea374d25afa262be261", - "sha256:a5193a098ae9f29af283dcf0041f762601faf2e595c0db1da929875b7570353f", - "sha256:bd50daf727f7c195e26f27467c85ce653d41df4358a25b32434a50d8870fc519", - "sha256:c4fceb864890b6168e79b0e714c585dbe2fd4222768ee90bc1aa0f8218691b11", - "sha256:e79570979ccdc3d165456dd62041d9556fb9733b86b4b6d818af7a0afc15f092", - "sha256:f46dd15335e8a320b0fb4685f58b7471702234cba8bb3442b69a3e1dc329c345" - ], - "markers": "python_version >= '3.7'", - "version": "==1.6.1" + "sha256:01b38dec7e9f897d4db04f8de4e20f0f5be3feac98468188a0f47a991b796055", + "sha256:10dbcc7de03b8d635a1031cb18fd3eaa997969b64fdf78f99f19ac163a825445", + "sha256:19aeac1ad3e57338723f4657ac8520f41714804568f2e30bd547d684d72c392e", + "sha256:1b21c6e0dc97b1762590b70dee0daddb291271be0580384d39f02c480b78290a", + "sha256:1caade0ede6967cc675e235c41451f9fb89ae34319ddf4740194094ab736b88d", + "sha256:23995dfcf269ec3735e5a8c80cfceaf384369a47699df111a6246b83a55da582", + "sha256:2a799714bf1f791fb2650d73222b248d18d53fd40d6af2df2c898db048189606", + "sha256:3274ce145b5dc416c49c0cf8b6119f787f0965cd35e22058fe1932c09fe15d77", + "sha256:33d1677d46111cfa1c84b87472a0274dde9ef4a7ef2e1f155f012f5f1e995d8f", + "sha256:44d452850f77e65e25b1eb1ac01e25770323a782bfe3a1a3e43847ad4266d93d", + "sha256:9e3302149a369697c6aaea18b430b216e3c88f9a61b62869f6104881e5f9ef85", + "sha256:a75b014d3294fce26852a9d04ea27b5671d86736beb34acdfc05859246260707", + "sha256:ad7269254de06743fb4768f658753de47d8b54e4672c5ebe8612a007a088bd48", + "sha256:b30280fbc1fd8082ac822994a98632111810311a9ece71a0e48f739df3c555a2", + "sha256:b79104878003487e2b4639a20b9092b02e1bad07fc4cf924b495cf413748a777", + "sha256:d449d40e830366b4c612692ad19fbebb722b6b847f78a7b701b1e0d6cda3cc13", + "sha256:d647757373985207af3343301d89fe738d5a294435a4f2aafb04c13b4388c896", + "sha256:f68eb46b86b2c246af99fcaa6f6e37c7a7a413e1084a794990b877f2ff71f7b6", + "sha256:fdf606341cd798530b05705c87779606fcdfaf768a8129c348ea94441da15b04" + ], + "markers": "python_version < '3.10' and python_version >= '3.7'", + "version": "==1.6.3" }, "seaborn": { "hashes": [ @@ -560,11 +590,11 @@ }, "six": { "hashes": [ - "sha256:30639c035cdb23534cd4aa2dd52c3bf48f06e5f4a941509c8bafd8ce11080259", - "sha256:8b74bedcbbbaca38ff6d7491d76f2b06b3592611af620f8426e82dddb04a5ced" + "sha256:1e61c37477a1626458e36f7b1d82aa5c9b094fa4802892072e49de9c60c4c926", + "sha256:8abb2f1d86890a2dfb989f9a77cfcfd3e47c2a354b01111771326f8aa26e0254" ], "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3'", - "version": "==1.15.0" + "version": "==1.16.0" }, "snuggs": { "hashes": [ @@ -592,18 +622,19 @@ }, "decorator": { "hashes": [ - "sha256:41fa54c2a0cc4ba648be4fd43cff00aedf5b9465c9bf18d64325bc225f08f760", - "sha256:e3a62f0520172440ca0dcc823749319382e377f37f140a0b99ef45fecb84bfe7" + "sha256:6e5c199c16f7a9f0e3a61a4a54b3d27e7dad0dbdde92b944426cb20914376323", + "sha256:72ecfba4320a893c53f9706bebb2d55c270c1e51a28789361aa93e4a21319ed5" ], - "version": "==4.4.2" + "markers": "python_version >= '3.5'", + "version": "==5.0.9" }, "ipython": { "hashes": [ - "sha256:04323f72d5b85b606330b6d7e2dc8d2683ad46c3905e955aa96ecc7a99388e70", - "sha256:34207ffb2f653bced2bc8e3756c1db86e7d93e44ed049daae9814fed66d408ec" + "sha256:a171caa3d3d4c819a1c0742e3abecfd5a2b8ab525ca1c9f114b40b76b0679ab1", + "sha256:f86788eef439891438af3498525094cc2acbdbea4f2aa2f8895782d4ff471341" ], "index": "pypi", - "version": "==7.21.0" + "version": "==7.24.0" }, "ipython-genutils": { "hashes": [ @@ -620,13 +651,21 @@ "markers": "python_version >= '3.6'", "version": "==0.18.0" }, + "matplotlib-inline": { + "hashes": [ + "sha256:5cf1176f554abb4fa98cb362aa2b55c500147e4bdbb07e3fda359143e1da0811", + "sha256:f41d5ff73c9f5385775d5c0bc13b424535c8402fe70ea8210f93e11f3683993e" + ], + "markers": "python_version >= '3.5'", + "version": "==0.1.2" + }, "parso": { "hashes": [ - "sha256:15b00182f472319383252c18d5913b69269590616c947747bc50bf4ac768f410", - "sha256:8519430ad07087d4c997fda3a7918f7cfa27cb58972a8c89c2a0295a1c940e9e" + "sha256:12b83492c6239ce32ff5eed6d3639d6a536170723c6f3f1506869f1ace413398", + "sha256:a8c4922db71e4fdb90e0d0bc6e50f9b273d3397925e5e60a717e719201778d22" ], "markers": "python_version >= '3.6'", - "version": "==0.8.1" + "version": "==0.8.2" }, "pexpect": { "hashes": [ @@ -645,11 +684,11 @@ }, "prompt-toolkit": { "hashes": [ - "sha256:0fa02fa80363844a4ab4b8d6891f62dd0645ba672723130423ca4037b80c1974", - "sha256:62c811e46bd09130fb11ab759012a4ae385ce4fb2073442d1898867a824183bd" + "sha256:bf00f22079f5fadc949f42ae8ff7f05702826a97059ffcc6281036ad40ac6f04", + "sha256:e1b4f11b9336a28fa11810bc623c357420f69dfdb6d2dac41ca2c21a55c033bc" ], "markers": "python_full_version >= '3.6.1'", - "version": "==3.0.16" + "version": "==3.0.18" }, "ptyprocess": { "hashes": [ @@ -660,11 +699,11 @@ }, "pygments": { "hashes": [ - "sha256:2656e1a6edcdabf4275f9a3640db59fd5de107d88e8663c5d4e9a0fa62f77f94", - "sha256:534ef71d539ae97d4c3a4cf7d6f110f214b0e687e92f9cb9d2a3b0d3101289c8" + "sha256:a18f47b506a429f6f4b9df81bb02beab9ca21d0a5fee38ed15aef65f0545519f", + "sha256:d66e804411278594d764fc69ec36ec13d9ae9147193a1740cd34d272ca383b8e" ], "markers": "python_version >= '3.5'", - "version": "==2.8.1" + "version": "==2.9.0" }, "traitlets": { "hashes": [ diff --git a/README.md b/README.md index c067ea29d..8bbeaeb96 100644 --- a/README.md +++ b/README.md @@ -38,6 +38,7 @@ The following input data sources should be downloaded and preprocessed prior to ### NHDPlus HR datasets - `acquire_and_preprocess_inputs.py` +- `preprocess_rasters.py` - `aggregate_nhd_hr_streams.py` **Please note:** For the following two datasets, please contact Mark Glaudemans (mark.glaudemans@noaa.gov). We are currently working on a long-term data sharing solution for the in-house NOAA data. @@ -68,11 +69,11 @@ docker run --rm -it -v :/data -v :/foss_fim -c /foss_fim/config/ -n ``` diff --git a/api/frontend/gui/templates/index.html b/api/frontend/gui/templates/index.html index 4dd2c2fc3..421050dbd 100644 --- a/api/frontend/gui/templates/index.html +++ b/api/frontend/gui/templates/index.html @@ -59,6 +59,7 @@ color: black; background-color: white; + font-size: 0.9rem; transition: all 0.1s linear; @@ -137,6 +138,27 @@ background-color: #093568; } + #release-button { + display: flex; + width: 100%; + height: 100%; + + align-items: center; + justify-content: center; + + font-size: 1.2rem; + color: white; + background-color: #28be99; + cursor: pointer; + + transition: all 0.5s linear; + pointer-events: inherit; + } + + #release-button:hover, #release-button.active { + background-color: #09685b; + } + .form-field { display: flex; width: 0; @@ -224,15 +246,40 @@
+
Release
FIM Run
Calibration
Pre-processing
+

Basic

- + @@ -271,7 +318,7 @@

Extent

-
Not Connected
+
Not Connected