Extending Caravan with new basins
This wiki page will guide you through the process of extending Caravan to new basins. After this succeeds, you will have generated files in the same format as Caravan for basins of your choice, including ERA5-Land timeseries and basin attributes.
You will need to run the code in two Jupyter notebooks (Caravan_part1_Earth_Engine.ipynb, Caravan_part2_local_postprocessing.ipynb. The notebooks include more explanations and have all necessary information and instructions - please use this Wiki page as a complementary resource.
- The first notebook uses Google Earth Engine to process ERA5-Land images and HydroAtlas characteristics for the basins of your choice into an intermediate format.
- After running the first notebook, the output files will be stored in a directory in your personal Google Drive account. You will then need to copy those files locally.
- The second notebook locally processes the intermediate files into the final format of files, similar to the ones provided in the Caravan dataset.
As part of the first notebook, you will need to upload to Earth Engine an "Asset", which is a shapefile including your basin polygons. This shapefile has to include one field with a unique basin id per polygon. Make sure that the name of this field is different to any HydroATLAS field. For example, you can use gauge_id or basin_id but not HYBAS_ID or PFAF_ID, which are both field names in HydroATLAS. For more details about the format, check the notebook itself. To upload an asset, you need to go to the Earth Engine Code Editor.
If this is your first time using Earth Engine, you will need to sign up. Click on your Google account, and you might see the following page:
Submit an application to use Google Earth Engine.
After submitting your application, try to use the Earth Engine Code Editor again - although the approval might take time, it will usually allow you to use Earth Engine immediately.
On the Earth Engine page, to upload your shapefile as an asset, click on Assets.
Then, Create home folder.
Pick a name for your folder as follows (using the default is recommended).
After your folder is created, click on New --> Shape files. Pick a name for the asset (you will later need to copy to it to the Jupyter notebook) and upload the .shp, .dbf and .shx files. The asset might take a few minutes to upload - you can track the progress as shown in the second image under the "Tasks" menu on the top left of your screen:
As part of the first notebook, you will need to allow writing access to Google Drive on your account. This process starts by running the following cell in the notebook:
During this phase you will need to accept and authorize the access at all steps. Click on the link that appears in the output of the cell. In the page that is opened, you will need to choose a cloud project:
Continue with creating a new cloud project:
Make sure to click continue and not "Back to safety":
Give the notebook the necessary permissions:
Copy the resulting Token into the Jupyter notebook, and continue running to the next code cells.
