yoon-etal_2024_hess

Representing Farmer Irrigated Crop Area Adaptation in a Large-Scale Hydrological Model

Jim Yoon^1*, Nathalie Voisin¹, Christian Klassert¹, Travis Thurber¹, Wenwei Xu¹

¹ Pacific Northwest National Laboratory, Richland, WA., USA

* corresponding author: jim.yoon@pnnl.gov

Abstract

Large-scale hydrological models (LHMs) are commonly used for regional and global assessment of future water shortage outcomes under climate and socioeconomic scenarios. The irrigation of croplands, which accounts for the lion’s share of human water consumption, is critical in understanding these water shortage trajectories. Despite irrigation’s defining role, LHM frameworks typically impose trajectories of land use that underlie irrigation demand, neglecting potential dynamic feedbacks in the form of human instigation of and subsequent adaptation to water shortage via irrigated crop area changes. We extend an LHM, MOSART-WM, with adaptive farmer agents, applying the model to the Continental United States to explore water shortage outcomes that emerge from the interplay between hydrologic-driven surface water availability, reservoir management, and farmer irrigated crop area adaptation. The extended modeling framework is used to conduct hypothetical computational experiment comparing differences between a model run with and without the incorporation of adaptive farmer agents. These comparative simulations reveal that accounting for farmer adaptation via irrigated crop area changes substantially alters modeled water shortage outcomes, with U.S.-wide annual water shortage reduced by as much as 42 percent when comparing adaptive and non-adaptive versions of the model forced with U.S. climatology from 1950-2009.

Journal reference

Yoon, J., Voisin, N., Klassert, C., Thurber, T., and Xu, W.: Representing Farmer Irrigated Crop Area Adaptation in a Large-Scale Hydrological Model, EGUsphere [preprint], https://doi.org/10.5194/egusphere-2023-1604, 2023.

Data reference

Input data

1. Heimlich, Ralph. USDA Agricultural Information Bulletin No. (AIB-760) 2 pp. (2000). Farm Resource Regions [Online]. Available at https://www.ers.usda.gov/publications/pub-details/?pubid=42299 (accessed 2021-04-12; verified 2021-04-12). USDA-NASS, Washington, DC.
2. USDA 2012 Census of Agriculture. (2013 and 2008). Table 4. Estimated Quantity of Water Applied By Source [Online]. Available at https://agcensus.library.cornell.edu/wp-content/uploads/2012-Farm-and-Ranch-Irrigation-Survey-fris13_1_004_004.pdf (accessed 2021-04-12; verified 2024-02-21). USDA-NASS, Washington, DC.
3. USDA 2012 Census of Agriculture. (2013 and 2008). Table 12. On-Farm Energy Expense for Pumping Irrigation Water by Water Source and Type of Energy [Online]. Available at https://agcensus.library.cornell.edu/wp-content/uploads/2012-Farm-and-Ranch-Irrigation-Survey-fris13_1_012_012.pdf (accessed 2021-04-12; verified 2024-02-21). USDA-NASS, Washington, DC.
4. USDA 2012 Census of Agriculture. (2013 and 2008). Table 35. Crops Harvested in the Open from Irrigated Farms [Online]. Available at https://agcensus.library.cornell.edu/wp-content/uploads/2012-Farm-and-Ranch-Irrigation-Survey-fris13_2_035_035.pdf (accessed 2021-04-12; verified 2024-02-21). USDA-NASS, Washington, DC.
5. USDA 2012 Census of Agriculture. (2013 and 2008). Table 36. Estimated Quantity of Water Applied and Primary Method of Distribution by Selected Crops Harvested in the Open [Online]. Available at https://agcensus.library.cornell.edu/wp-content/uploads/2012-Farm-and-Ranch-Irrigation-Survey-fris13_2_036_036.pdf (accessed 2021-04-12; verified 2024-02-21). USDA-NASS, Washington, DC.
6. USDA Economic Research Service using data from USDA’s Agricultural Resource Management Survey (ARMS) and other sources. (1995-2017). Published Commodity Costs and Returns [Online]. Available at https://www.ers.usda.gov/data-products/commodity-costs-and-returns/commodity-costs-and-returns (accessed 2021-05-12; verified 2021-05-12). USDA-NASS, Washington, DC
7. USDA National Agricultural Statistics Service Cropland Data Layer. (2008-2018). Published crop-specific data layer [Online]. Available at https://nassgeodata.gmu.edu/CropScape/ (accessed 2021-04-05; verified 2021-04-05). USDA-NASS, Washington, DC.
8. Voisin, N. et al. (2018). MOSART-WM-ABM Input Data [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4836886.

Output data

9. Yoon, J. (2024). Model run output for WM-ABM 2024 HESS Publication (Version v1) [Data set]. MSD-LIVE Data Repository. https://doi.org/10.57931/2293596.

Code reference

10. Jim Yoon, & Travis Thurber. (2024). IMMM-SFA/wm-abm_data_processing: v1.1.0 (v1.1.0). Zenodo. https://doi.org/10.5281/zenodo.10689455.
11. Yoon, Jim, & Thurber, Travis. (2021). IMMM-SFA/iwmm: MOSART-WM-ABM (Version v1.1.2.abm). Zenodo. http://doi.org/10.5281/zenodo.4739516.
12. Jim Yoon, & Travis Thurber. (2024). IMMM-SFA/wm-abm_pmp: v1.1.0 (v1.1.0). Zenodo. https://doi.org/10.5281/zenodo.10689430.

Contributing modeling software

Model	Version	Repository Link	DOI
wm-abm_pmp	v1.1.0	https://github.com/IMMM-SFA/wm-abm_pmp	https://doi.org/10.5281/zenodo.5570500
wm-abm_data_processing	v1.1.0	https://github.com/IMMM-SFA/wm-abm_data_processing	https://doi.org/10.5281/zenodo.5570863
MOSART-WM-ABM	v1.1.2.abm	https://github.com/IMMM-SFA/iwmm	https://doi.org/10.5281/zenodo.4739516

Reproduce my experiment

0a. Preprocess the USDA data to produce these files:

• usda farm budget summary (machine readable).xlsx - Combine the data for relevant crops from [6] into a single Excel file.
• usda irrigation summary.xlsx - Reformat the data from [4] into a machine-readable format.
• usda irrigation water requirement.xlsx - Reformat the data from [5] into a machine-readable format.
• nldas_states_counties_regions.csv - Map grid cells to U.S. state IDs, county IDs, and ERS farm regions; developed as part of this experiment using data from [1] and spatial joins in ArcMap.
• water_proportions.csv - Reformat the data from [2] into a machine-readable format.
• water_costs.csv - Aggregate surface water costs from [5] and aggregate groundwater costs by dividing volume of groundwater irrigation [2] by total groundwater energy cost [3].
• cdl_gcam_lookup_v3_notavailcorr.csv - Map CDL crop types to Global Change Analysis Model (GCAM) crop types; developed as part of this experiment.

0b. Download the Cropland Data Layer (CDL) files for 2008-2018 from [7] and preprocess:

• For each year, sum the pixels for each CDL Crop category within each 1/8 degree North American Land Data Assimilation System (NLDAS) grid cell using ArcMap or similar tool; producing files like cdl_{year}_clean.csv
• Run the script cdl_processing.py from [10] to combine the yearly files into a single file in the expected format: all_nldas_cdl_data_v3.txt

0c. Preprocess the output from steps 0a and 0b for use in MOSART-WM-ABM:

• crop_ids_by_farm.p - Pickled dictionary mapping farm ID to the list of crop IDs on the farm; developed as part of this experiment.
• NLDAS_Grid_Reference.csv - Maps NLDAS IDs to latitude/longitude coordinates; developed as part of this experiment by spatially joining an NLDAS shapefile with a U.S. counties shapefile in GIS.
• nldas.txt - Subset of columns from NLDAS_Grid_Reference.csv.
• nldas_ids.p - Pickled list of NLDAS IDs to be included in the optimization; developed as part of this experiment.
• Run the script wmabm_data_process.py from [10] to generate the following files:
  • max_land_constr_20201102_protocol2.p
  • MOSART_WM_PMP_inputs_20220323_GW
• Run the script hist_water_availability_abm.py from [12] which ingests some output files from a historical MOSART-WM simulation to generate the following files:
  • hist_dependent_storage.csv
  • hist_avail_bias_correction_20201102.csv
• Run the script MOSART_WM_PMP_stage1_noloop_gwalt.py from [12] to generate the following files:
  • gammas_new_20201102_protocol2.p
  • net_prices_new_20201102_protocol2.p

1. The results of step 0a are available as part of the code repository [10]; the results of step 0b can be downloaded from [9]; the results of step 0c are available as part of the code repository [12].

2. Download and unpack the MOSART-WM-ABM input data from [8] into a supercomputing environment.

3. Clone the MOSART-WM-ABM repository [11] into a supercomputing environment; modifications to the source code may be necessary to run on unsupported machines.

4. From the repository root directory, setup the model with the following shell commands (replacing sections surrounded by curly braces with the specifics of your environment):

• ./cime/script/create_newcase --case {absolute path to a directory to create for your run} --res NLDAS_NLDAS --compset mosart_runoff_driven --project {cost center for your use of supercomputer}
• cd {absolute path to your run directory}
• ./xmlchange DIN_LOC_ROOT={absolute path to your unpacked data directory from step 2}
• ./xmlchange DLND_CPLHIST_DIR={absolute path to your unpacked data directory from step 2}
• ./xmlchange LND_DOMAIN_PATH={absolute path to your unpacked data directory from step 2}
• ./xmlchange RUN_STARTDATE=1940-01-01
• ./xmlchange STOP_OPTION=nyears
• ./xmlchange STOP_N=70
• ./xmlchange JOB_WALLCLOCK_TIME=35:00:00 (you may need more or less wallclock time depending on machine)
• ./xmlchange JOB_QUEUE={name of a job queue in your supercomputing environment}
• ./case.setup

5. Make the following modifications to the file user_nl_mosart in the root of your run directory:

• Change the value of parafile to '{absolute path to your unpacked data directory from step 2}/US_reservoir_8th_NLDAS3_updated_CERF_Livneh_naturalflow.nc'.
• Add a new line with: frivinp_rtm = '{absolute path to your unpacked data directory from step 2}/MOSART_NLDAS_8th_20160426.nc'

6. Build the model with the shell command: ./case.build

7. Submit the model run to the job queue with the shell command: ./case.submit

8. If all is well, the model will eventually run, generating output and log files into your directory within the supercomputer's scratch file system, including the data found in [9].

Reproduce my figures

Reproduce my figures:

The figures for the manuscript are generated based on post-processing of MOSART-WM-ABM output files in Python, Tableau, QGIS, and Inkscape. The WM-ABM working directory contains two general types of output files: 1) monthly netCDF output files with the MOSART-WM results (e.g., river flows, water demand, water shortage, etc.) and, 2) csv files containing annual ABM results (crop types, crop areas, etc.). The general steps are laid out below:

1. The monthly netCDF files are post-processed into a consolidated csv file using the "WM_output_PIC_ncdf.py" script. The script reads in monthly the MOSART-WM netCDF files, calculates various annual summary statistics (e.g., average supply for each cell by year), saves the summary statistics into a pandas dataframe, and finally outputs the pandas dataframe as a csv (e.g., "wm_summary_results.csv").

2. The "abm_output_processing.py" script is used to further post-process output, taking the annual abm output csv files and the wm summary file generated in [1] above as inputs.

3. Figure 2a is generated based upon taking post-processed output from [2] and developing the final visualizations in Tableau. Final touch ups (e.g., drought labels) are added in Inkscape.

4. Figure 2b is generated based upon taking post-processed output from [2] and generating the final maps in QGIS. A Stamen Terrain Background map is used for the visualization using QuickMapServices in QGIS. Final touch ups (e.g., modification of label locations) are conducted in Inkscape.

5. The crop area bump charts (left side) of Figure 3 are generated based upon taking post-processed output from [2] and developing the final visualizations in Tableau. See guide here: http://www.datatableauandme.com/2017/08/how-to-area-bump-chart-in-tableau.html

6. The shortage and adaptivity maps (right side) of Figure 3 are generated taking post-processed output from [2] and developing the final visualizations in QGIS with final touch ups conducted in Inkscape.

7. Figure 4 is generated based upon taking post-processed output from [2] and developing the final visualizations in Tableau. Final touch ups (e.g., drought labels) are added in Inkscape.

8. Figure 1 is developed in Powerpoint and does not rely on any data inputs