Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
Manual Entry: Recalculate Area-derived Values on Land Cover Updates #3151
MapShed operates on a large dictionary of values, whose fields can have static, lookup, user-provided, or calculated values. A number of these values are calculated using rasters geoprocessed for the given area of interest. One of these rasters in the NLCD raster, which is provides a count of cells for each land cover type. These counts are stored in the
As of #2977, users can override the land cover counts for a given scenario. This modification is saved on the scenario. When running MapShed, these modifications are combined with the
Unfortunately, so far we have been simply overriding the
Keys whose values are affected by the
This is done on the back-end for facilitating improved GWLF-E calculations, and on the front-end for allowing Land Cover modifications to inform Conservation Practice modifications.
See commit messages for details.
$ ./scripts/manage.sh test_mapshed + ARGS=test_mapshed + [[ 1 -eq 1 ]] + [[ test_mapshed == runserver ]] + vagrant ssh app -c 'cd /opt/app && envdir /etc/mmw.d/env ./manage.py test_mapshed' . ---------------------------------------------------------------------- Ran 1 test in 1.124s OK Connection to 127.0.0.1 closed.
If the land use is overridden:
The conservation practice also updates:
There are two exceptions to the fields updated, one overt and one subtle:
Despite being computed from
A number of other geoprocessing operations combine NLCD with another raster to get a grouped count, such as
Recalculating these fields from the modifications is complicated, because the modifications are only changing the proportion of land use distribution, not the actual distribution in space. Without the spatial updates, we cannot run raster operations. This will need to be clarified and addressed at a later date.
Additionally, there are some concerns outlined here about the implications of modifications affecting each other.
Using a sample area of interes and observed output, add a test that ensures the output matches. This is useful when making changes to the calculation to ensure that the numbers do not change significantly when the calcs are updated. The dictdiffer package is brought in which allows for deep comparison of dicts, with a configurable tolerance for numeric comparison. The tests are currently set to tolerate differences smaller than 1e-15. The test is implemented as a management command rather than a unit test because it needs access to the special MapShed tables to run the calculations, which are not available in the Unit Test database. To add the large tables as fixtures to the Unit Test database would be too big of a slowdown for the sake of this one test, and would considerably bloat the repo size. To ensure the test does not run with the others, it is tagged and excluded from the test script.
Previously, when the user would override Land Use values, we would only update the Area list in the MapShed dict. However, there are a number of other fields in the dict whose values are derived from the Area list, which were not updated. The derivation of those fields is now extracted into its own method, which is called in the initialization of the dict, as well as during ingestion of modifications from the user. If the set of modifications contains updates to the land use distribution, then the area derived fields are recalculated. Notably, the LS calculation is not included in this refactor. This is because while it does derive its value from Area, it is also dependent on the geoprocessing result of `lu_stream_pct`, which isn't saved anywhere. Because of that, we cannot adjust its value with new proportions when the user updates land use changes. Amending this would require a larger refactor, which is deferred to a later date. The test introduced in the previous commit serves to ensure that this refactoring does not alter the calculations.
The changes made in the previous commit recalculate the derived values for running the MapShed model. The changes in this commit recalculate them for the data model in the front-end, which allows for other modifications to pick up on the land use changes. For example, if the user changes the area of Cropland in the Land Cover modal, the Area of Row Crops should change correspondingly for the respective Conservation Practices. This will now happen, as `n23` (the model field representing the Area of Row Crops), amongst others, is now updated.