Skip to content

Commit

Permalink
Merge pull request #656 from neteler/actinia_quickstart_updates
Browse files Browse the repository at this point in the history 
actinia_quickstart.rst: updates to current actinia version
  • Loading branch information
@kalxas
kalxas committed Jan 24, 2021
2 parents 187de38 + e37de76 commit 9fba45a
Showing 1 changed file with 138 additions and 93 deletions.
231 changes: 138 additions & 93 deletions doc/quickstart/actinia_quickstart.rst
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
:Author: Markus Neteler, mundialis
:Reviewer: Astrid Emde
:Reviewer: Felicity Brand (Google Season of Docs 2019)
:Version: osgeolive13.0
:Version: osgeolive14.0
:License: Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)
:Copyright: 2019 by The OSGeo Foundation
:Copyright: 2019-2021 by The OSGeo Foundation

@LOGO_actinia@
@OSGEO_KIND_actinia@
Expand Down Expand Up @@ -39,7 +39,7 @@ then execute it on the server.
Introduction to ace - actinia command execution
===============================================

The ``ace`` tool (`details <https://github.com/mundialis/actinia_core/tree/master/scripts>`_)
The ``ace`` tool (`details <https://github.com/mundialis/actinia_core/tree/master/scripts/README.md>`_)
allows the execution of a single GRASS GIS command or a
list of GRASS GIS commands on an actinia REST service
(https://actinia.mundialis.de/). In addition it provides job management,
Expand All @@ -60,6 +60,9 @@ in the **persistent** user database. It can be used with

Setup your environment
======================

Be sure to run the following commands in a GRASS GIS session.

The user must setup the following environmental variables to specify the
actinia server and credentials:

Expand All @@ -78,15 +81,16 @@ Selected datasets are available to the demo user. To list the locations you have
.. code:: bash
ace --list-locations
['latlong', 'nc_spm_08', 'utm_32n', 'latlong']
['latlong_wgs84', 'ECAD', 'nc_spm_08']
The following command lists mapsets of current location in the active
GRASS GIS session (nc_spm_08):

.. code:: bash
# running ace in the "nc_spm_08" location:
ace --list-mapsets
# running ace in the "nc_spm_08" location
# (the current location name is propagated to the server):
ace --location latlong_wgs84 --list-mapsets
['PERMANENT', 'landsat']
Access data from external sources
Expand All @@ -110,15 +114,15 @@ Currently available datasets are (organized by projections):

- Latitude-Longitude location (LatLong WGS84, EPSG:4326):

- empty (``latlong/PERMANENT/``)
- empty (``latlong_wgs84/PERMANENT/``)
- 16-days NDVI, MOD13C1, V006, CMG 0.05 deg res.
(``latlong/modis_ndvi_global/``; source:
(``latlong_wgs84/modis_ndvi_global/``; source:
https://lpdaac.usgs.gov/products/mod13c1v006/)
- LST growing degree days asia 2017 (``latlong/asia_gdd_2017/``;
- LST growing degree days asia 2017 (``latlong_wgs84/asia_gdd_2017/``;
source: https://www.mundialis.de/en/temperature-data/)
- LST tropical days asia 2017 (``latlong/asia_tropical_2017/``)
- LST tropical days asia 2017 (``latlong_wgs84/asia_tropical_2017/``)
- LST temperature daily asia 2017, including min, max and avg
(``latlong/asia_lst_daily_2017/``)
(``latlong_wgs84/asia_lst_daily_2017/``)

- Europe (EU LAEA CRS, EPSG:3035):

Expand All @@ -138,7 +142,7 @@ Inspect the REST call prior to submission
================================================================================

To generate the actinia process chain JSON request simply add the
–dry-run flag:
--dry-run flag:

.. code:: bash
Expand All @@ -152,8 +156,8 @@ It is very easy (and fast) to render a map:
::

# check amount of pixels, just FYI
ace --location latlong r.info globcover@globcover
ace --location latlong --render-raster globcover@globcover
ace --location latlong_wgs84 r.info globcover@globcover
ace --location latlong_wgs84 --render-raster globcover@globcover

.. figure:: /images/projects/actinia/esa_globcover_rendered_by_ace.png
:alt: ESA Globcover map shown by actinia
Expand All @@ -165,8 +169,8 @@ ephemeral mapset which will be removed after processing. You can export the
output of GRASS GIS modules to store the computational result for download and further analysis.
The following export formats are currently supported:

- raster: ``GTiff``
- vector: ``ESRI_Shapefile``, ``GeoJSON``, ``GML``
- raster: ``COG``, ``GTiff``
- vector: ``ESRI_Shapefile``, ``GeoJSON``, ``GML``, ``GPKG``
- table: ``CSV``, ``TXT``


Expand All @@ -186,91 +190,131 @@ Store the following script as text file ``ace_dtm_statistics.sh``:

.. code:: bash
# grass77 ~/grassdata/nc_spm_08/user1/
# grass78 ~/grassdata/nc_spm_08/user1/
# Import the web resource and set the region to the imported map
g.region raster=elev+https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif -ap
g.region raster=elev@https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif -ap
# Compute univariate statistics
r.univar map=elev
r.info elev
# Compute the slope of the imported map and mark it for export as geotiff file
r.slope.aspect elevation=elev slope=slope_elev+GTiff
# Compute the slope of the imported map and mark it for export as a COG file (Cloud Optimized GeoTIFF)
r.slope.aspect elevation=elev slope=slope_elev+COG
r.info slope_elev
Save the script in the text file to ``/tmp/ace_dtm_statistics.sh`` and
run the saved script as

.. code:: bash
ace --script /tmp/ace_dtm_statistics.sh
ace --location nc_spm_08 --script ace_dtm_statistics.sh
The results are provided as REST resources.
The results (messages, statistics, files) are provided as REST resources.

To generate the actinia process chain JSON request simply add the
–dry-run flag
--dry-run flag

.. code:: bash
ace --dry-run --script /tmp/ace_dtm_statistics.sh
ace --dry-run --location nc_spm_08 --script /tmp/ace_dtm_statistics.sh
The output should look like this:

.. code:: json
{
"version": "1",
"list": [
{
"module": "g.region",
"id": "g.region_1804289383",
"flags": "pa",
"inputs": [
{
"import_descr": {
"source": "https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif",
"type": "raster"
},
"param": "raster", "value": "elev"
}
]
},
{
"module": "r.univar",
"id": "r.univar_1804289383",
"inputs": [
{"param": "map", "value": "elev"},
{"param": "percentile", "value": "90"},
{"param": "separator", "value": "pipe"}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [{"param": "map", "value": "elev"}]
},
{
"module": "r.slope.aspect",
"id": "r.slope.aspect_1804289383",
"inputs": [
{"param": "elevation", "value": "elev"},
{"param": "format", "value": "degrees"},
{"param": "precision", "value": "FCELL"},
{"param": "zscale", "value": "1.0"},
{"param": "min_slope", "value": "0.0"}
],
"outputs": [
{
"export": {"format": "GTiff", "type": "raster"},
"param": "slope", "value": "slope_elev"
}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [{"param": "map", "value": "slope_elev"}]
}
]
}
{
"version": "1",
"list": [
{
"module": "g.region",
"id": "g.region_1804289383",
"flags": "pa",
"inputs": [
{
"import_descr": {
"source": "https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif",
"type": "raster"
},
"param": "raster",
"value": "elev"
}
]
},
{
"module": "r.univar",
"id": "r.univar_1804289383",
"inputs": [
{
"param": "map",
"value": "elev"
},
{
"param": "percentile",
"value": "90"
},
{
"param": "separator",
"value": "pipe"
}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [
{
"param": "map",
"value": "elev"
}
]
},
{
"module": "r.slope.aspect",
"id": "r.slope.aspect_1804289383",
"inputs": [
{
"param": "elevation",
"value": "elev"
},
{
"param": "format",
"value": "degrees"
},
{
"param": "precision",
"value": "FCELL"
},
{
"param": "zscale",
"value": "1.0"
},
{
"param": "min_slope",
"value": "0.0"
}
],
"outputs": [
{
"export": {
"format": "COG",
"type": "raster"
},
"param": "slope",
"value": "slope_elev"
}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [
{
"param": "map",
"value": "slope_elev"
}
]
}
]
}
Example 2: Orthophoto image segmentation with export
----------------------------------------------------
Expand All @@ -279,11 +323,11 @@ Store the following script as text file ``/tmp/ace_segmentation.sh``:

.. code:: bash
# grass77 ~/grassdata/nc_spm_08/user1/
# grass78 ~/grassdata/nc_spm_08/user1/
# Import the web resource and set the region to the imported map
# we apply a trick for the import of multi-band GeoTIFFs:
# install with: g.extension importer
importer raster=ortho2010+https://apps.mundialis.de/workshops/osgeo_ireland2017/north_carolina/ortho2010_t792_subset_20cm.tif
# install with: g.extension importer url=https://github.com/mundialis/importer
importer raster=ortho2010@https://apps.mundialis.de/workshops/osgeo_ireland2017/north_carolina/ortho2010_t792_subset_20cm.tif
# The importer has created three new raster maps, one for each band in the geotiff file
# stored them in an image group
r.info map=ortho2010.1
Expand All @@ -292,17 +336,18 @@ Store the following script as text file ``/tmp/ace_segmentation.sh``:
# Set the region and resolution
g.region raster=ortho2010.1 res=1 -p
# Note: the RGB bands are organized as a group
i.segment group=ortho2010 threshold=0.25 output=ortho2010_segment_25+GTiff goodness=ortho2010_seg_25_fit+GTiff
# export as a as COG file (Cloud Optimized GeoTIFF)
i.segment group=ortho2010 threshold=0.25 output=ortho2010_segment_25+COG goodness=ortho2010_seg_25_fit+COG
# Finally vectorize segments with r.to.vect and export as a GeoJSON file
r.to.vect input=ortho2010_segment_25 type=area output=ortho2010_segment_25+GeoJSON
Run the script saved in a text file as

.. code:: bash
ace --script /tmp/ace_segmentation.sh
ace --location nc_spm_08 --script /tmp/ace_segmentation.sh
The results are provided as REST resources.
The results (messages, statistics, files) are provided as REST resources.

Examples for persistent processing
----------------------------------
Expand All @@ -324,7 +369,7 @@ Run the commands from the statistic script in the new persistent mapset

.. code:: bash
ace --location nc_spm_08 --persistent test_mapset --script /path/to/ace_dtm_statistics.sh
ace --location nc_spm_08 --persistent test_mapset --script /tmp/ace_dtm_statistics.sh
Show all raster maps that have been created with the script in
test_mapset
Expand All @@ -340,13 +385,13 @@ Show information about raster map elev and slope_elev
ace --location nc_spm_08 --persistent test_mapset r.info elev@test_mapset
ace --location nc_spm_08 --persistent test_mapset r.info slope_elev@test_mapset
Delete the test_mapset
Delete the test_mapset (always double check names when deleting)

.. code:: bash
ace --location nc_spm_08 --delete-mapset test_mapset
If the active GRASS GIS session has identical location/mapset settings,
If the active GRASS GIS session has identical location/mapset names,
then an alias can be used to avoid the persistent option in each single
command call:

Expand Down Expand Up @@ -375,25 +420,25 @@ Create new locations
.. code:: bash
# create new location
ace --create-location latlon 4326
ace --create-location latlon_test 4326
# create new mapset within location
ace --location latlon --create-mapset user1
ace --location latlon_test --create-mapset user1
Install GRASS GIS addons (extensions)
--------------------------------------------------------------------------------
.. code:: bash
# list existing addons, see also
# https://grass.osgeo.org/grass7/manuals/addons/
ace --location latlon g.extension -l
ace --location latlon_test g.extension -l
# install machine learning addon r.learn.ml
ace --location latlon g.extension r.learn.ml
ace --location latlon_test g.extension r.learn.ml
What next?
==========
* Visit the actinia website at `https://actinia.mundialis.de <https://actinia.mundialis.de>`_
* actinia tutorial: `https://neteler.gitlab.io/actinia-introduction <https://neteler.gitlab.io/actinia-introduction>`_
* Further reading: Neteler, M., Gebbert, S., Tawalika, C., Bettge, A., Benelcadi, H., Löw, F., Adams, T, Paulsen, H. (2019). Actinia: cloud based geoprocessing. In Proc. of the 2019 conference on Big Data from Space (BiDS'2019) (pp. 41–44). EUR 29660 EN, Publications Office of the European Union 5, Luxembourg: P. Soille, S. Loekken, and S. Albani (Eds.). (`DOI <https://zenodo.org/record/2631917>`__)
* Further reading: Neteler, M., Gebbert, S., Tawalika, C., Bettge, A., Benelcadi, H., Löw, F., Adams, T, Paulsen, H. (2019). Actinia: cloud based geoprocessing. In Proc. of the 2019 conference on Big Data from Space (BiDS'2019) (pp. 41-44). EUR 29660 EN, Publications Office of the European Union 5, Luxembourg: P. Soille, S. Loekken, and S. Albani (Eds.). (`DOI <https://zenodo.org/record/2631917>`__)

0 comments on commit 9fba45a

Please sign in to comment.