Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

actinia_quickstart.rst: updates to current actinia version #656

Merged
merged 3 commits into from
Jan 24, 2021
Merged

actinia_quickstart.rst: updates to current actinia version #656

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
8d4109e
actinia_quickstart.rst: updates to current actinia state
neteler Jan 5, 2021
0787782
actinia_quickstart.rst: further updates
neteler Jan 22, 2021
e37de76
Merge branch 'master' into actinia_quickstart_updates
neteler Jan 22, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
231 changes: 138 additions & 93 deletions doc/quickstart/actinia_quickstart.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
:Author: Markus Neteler, mundialis
:Reviewer: Astrid Emde
:Reviewer: Felicity Brand (Google Season of Docs 2019)
:Version: osgeolive13.0
:Version: osgeolive14.0
:License: Creative Commons Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0)
:Copyright: 2019 by The OSGeo Foundation
:Copyright: 2019-2021 by The OSGeo Foundation

@LOGO_actinia@
@OSGEO_KIND_actinia@
Expand Down Expand Up @@ -39,7 +39,7 @@ then execute it on the server.
Introduction to ace - actinia command execution
===============================================

The ``ace`` tool (`details <https://github.com/mundialis/actinia_core/tree/master/scripts>`_)
The ``ace`` tool (`details <https://github.com/mundialis/actinia_core/tree/master/scripts/README.md>`_)
allows the execution of a single GRASS GIS command or a
list of GRASS GIS commands on an actinia REST service
(https://actinia.mundialis.de/). In addition it provides job management,
Expand All @@ -60,6 +60,9 @@ in the **persistent** user database. It can be used with

Setup your environment
======================

Be sure to run the following commands in a GRASS GIS session.

The user must setup the following environmental variables to specify the
actinia server and credentials:

Expand All @@ -78,15 +81,16 @@ Selected datasets are available to the demo user. To list the locations you have
.. code:: bash

ace --list-locations
['latlong', 'nc_spm_08', 'utm_32n', 'latlong']
['latlong_wgs84', 'ECAD', 'nc_spm_08']

The following command lists mapsets of current location in the active
GRASS GIS session (nc_spm_08):

.. code:: bash

# running ace in the "nc_spm_08" location:
ace --list-mapsets
# running ace in the "nc_spm_08" location
# (the current location name is propagated to the server):
ace --location latlong_wgs84 --list-mapsets
['PERMANENT', 'landsat']

Access data from external sources
Expand All @@ -110,15 +114,15 @@ Currently available datasets are (organized by projections):

- Latitude-Longitude location (LatLong WGS84, EPSG:4326):

- empty (``latlong/PERMANENT/``)
- empty (``latlong_wgs84/PERMANENT/``)
- 16-days NDVI, MOD13C1, V006, CMG 0.05 deg res.
(``latlong/modis_ndvi_global/``; source:
(``latlong_wgs84/modis_ndvi_global/``; source:
https://lpdaac.usgs.gov/products/mod13c1v006/)
- LST growing degree days asia 2017 (``latlong/asia_gdd_2017/``;
- LST growing degree days asia 2017 (``latlong_wgs84/asia_gdd_2017/``;
source: https://www.mundialis.de/en/temperature-data/)
- LST tropical days asia 2017 (``latlong/asia_tropical_2017/``)
- LST tropical days asia 2017 (``latlong_wgs84/asia_tropical_2017/``)
- LST temperature daily asia 2017, including min, max and avg
(``latlong/asia_lst_daily_2017/``)
(``latlong_wgs84/asia_lst_daily_2017/``)

- Europe (EU LAEA CRS, EPSG:3035):

Expand All @@ -138,7 +142,7 @@ Inspect the REST call prior to submission
================================================================================

To generate the actinia process chain JSON request simply add the
–dry-run flag:
--dry-run flag:

.. code:: bash

Expand All @@ -152,8 +156,8 @@ It is very easy (and fast) to render a map:
::

# check amount of pixels, just FYI
ace --location latlong r.info globcover@globcover
ace --location latlong --render-raster globcover@globcover
ace --location latlong_wgs84 r.info globcover@globcover
ace --location latlong_wgs84 --render-raster globcover@globcover

.. figure:: /images/projects/actinia/esa_globcover_rendered_by_ace.png
:alt: ESA Globcover map shown by actinia
Expand All @@ -165,8 +169,8 @@ ephemeral mapset which will be removed after processing. You can export the
output of GRASS GIS modules to store the computational result for download and further analysis.
The following export formats are currently supported:

- raster: ``GTiff``
- vector: ``ESRI_Shapefile``, ``GeoJSON``, ``GML``
- raster: ``COG``, ``GTiff``
- vector: ``ESRI_Shapefile``, ``GeoJSON``, ``GML``, ``GPKG``
- table: ``CSV``, ``TXT``


Expand All @@ -186,91 +190,131 @@ Store the following script as text file ``ace_dtm_statistics.sh``:

.. code:: bash

# grass77 ~/grassdata/nc_spm_08/user1/
# grass78 ~/grassdata/nc_spm_08/user1/
# Import the web resource and set the region to the imported map
g.region raster=elev+https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif -ap
g.region raster=elev@https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif -ap
# Compute univariate statistics
r.univar map=elev
r.info elev
# Compute the slope of the imported map and mark it for export as geotiff file
r.slope.aspect elevation=elev slope=slope_elev+GTiff
# Compute the slope of the imported map and mark it for export as a COG file (Cloud Optimized GeoTIFF)
r.slope.aspect elevation=elev slope=slope_elev+COG
r.info slope_elev

Save the script in the text file to ``/tmp/ace_dtm_statistics.sh`` and
run the saved script as

.. code:: bash

ace --script /tmp/ace_dtm_statistics.sh
ace --location nc_spm_08 --script ace_dtm_statistics.sh

The results are provided as REST resources.
The results (messages, statistics, files) are provided as REST resources.

To generate the actinia process chain JSON request simply add the
–dry-run flag
--dry-run flag

.. code:: bash

ace --dry-run --script /tmp/ace_dtm_statistics.sh
ace --dry-run --location nc_spm_08 --script /tmp/ace_dtm_statistics.sh

The output should look like this:

.. code:: json

{
"version": "1",
"list": [
{
"module": "g.region",
"id": "g.region_1804289383",
"flags": "pa",
"inputs": [
{
"import_descr": {
"source": "https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif",
"type": "raster"
},
"param": "raster", "value": "elev"
}
]
},
{
"module": "r.univar",
"id": "r.univar_1804289383",
"inputs": [
{"param": "map", "value": "elev"},
{"param": "percentile", "value": "90"},
{"param": "separator", "value": "pipe"}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [{"param": "map", "value": "elev"}]
},
{
"module": "r.slope.aspect",
"id": "r.slope.aspect_1804289383",
"inputs": [
{"param": "elevation", "value": "elev"},
{"param": "format", "value": "degrees"},
{"param": "precision", "value": "FCELL"},
{"param": "zscale", "value": "1.0"},
{"param": "min_slope", "value": "0.0"}
],
"outputs": [
{
"export": {"format": "GTiff", "type": "raster"},
"param": "slope", "value": "slope_elev"
}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [{"param": "map", "value": "slope_elev"}]
}
]
}
{
"version": "1",
"list": [
{
"module": "g.region",
"id": "g.region_1804289383",
"flags": "pa",
"inputs": [
{
"import_descr": {
"source": "https://storage.googleapis.com/graas-geodata/elev_ned_30m.tif",
"type": "raster"
},
"param": "raster",
"value": "elev"
}
]
},
{
"module": "r.univar",
"id": "r.univar_1804289383",
"inputs": [
{
"param": "map",
"value": "elev"
},
{
"param": "percentile",
"value": "90"
},
{
"param": "separator",
"value": "pipe"
}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [
{
"param": "map",
"value": "elev"
}
]
},
{
"module": "r.slope.aspect",
"id": "r.slope.aspect_1804289383",
"inputs": [
{
"param": "elevation",
"value": "elev"
},
{
"param": "format",
"value": "degrees"
},
{
"param": "precision",
"value": "FCELL"
},
{
"param": "zscale",
"value": "1.0"
},
{
"param": "min_slope",
"value": "0.0"
}
],
"outputs": [
{
"export": {
"format": "COG",
"type": "raster"
},
"param": "slope",
"value": "slope_elev"
}
]
},
{
"module": "r.info",
"id": "r.info_1804289383",
"inputs": [
{
"param": "map",
"value": "slope_elev"
}
]
}
]
}


Example 2: Orthophoto image segmentation with export
----------------------------------------------------
Expand All @@ -279,11 +323,11 @@ Store the following script as text file ``/tmp/ace_segmentation.sh``:

.. code:: bash

# grass77 ~/grassdata/nc_spm_08/user1/
# grass78 ~/grassdata/nc_spm_08/user1/
# Import the web resource and set the region to the imported map
# we apply a trick for the import of multi-band GeoTIFFs:
# install with: g.extension importer
importer raster=ortho2010+https://apps.mundialis.de/workshops/osgeo_ireland2017/north_carolina/ortho2010_t792_subset_20cm.tif
# install with: g.extension importer url=https://github.com/mundialis/importer
importer raster=ortho2010@https://apps.mundialis.de/workshops/osgeo_ireland2017/north_carolina/ortho2010_t792_subset_20cm.tif
# The importer has created three new raster maps, one for each band in the geotiff file
# stored them in an image group
r.info map=ortho2010.1
Expand All @@ -292,17 +336,18 @@ Store the following script as text file ``/tmp/ace_segmentation.sh``:
# Set the region and resolution
g.region raster=ortho2010.1 res=1 -p
# Note: the RGB bands are organized as a group
i.segment group=ortho2010 threshold=0.25 output=ortho2010_segment_25+GTiff goodness=ortho2010_seg_25_fit+GTiff
# export as a as COG file (Cloud Optimized GeoTIFF)
i.segment group=ortho2010 threshold=0.25 output=ortho2010_segment_25+COG goodness=ortho2010_seg_25_fit+COG
# Finally vectorize segments with r.to.vect and export as a GeoJSON file
r.to.vect input=ortho2010_segment_25 type=area output=ortho2010_segment_25+GeoJSON

Run the script saved in a text file as

.. code:: bash

ace --script /tmp/ace_segmentation.sh
ace --location nc_spm_08 --script /tmp/ace_segmentation.sh

The results are provided as REST resources.
The results (messages, statistics, files) are provided as REST resources.

Examples for persistent processing
----------------------------------
Expand All @@ -324,7 +369,7 @@ Run the commands from the statistic script in the new persistent mapset

.. code:: bash

ace --location nc_spm_08 --persistent test_mapset --script /path/to/ace_dtm_statistics.sh
ace --location nc_spm_08 --persistent test_mapset --script /tmp/ace_dtm_statistics.sh

Show all raster maps that have been created with the script in
test_mapset
Expand All @@ -340,13 +385,13 @@ Show information about raster map elev and slope_elev
ace --location nc_spm_08 --persistent test_mapset r.info elev@test_mapset
ace --location nc_spm_08 --persistent test_mapset r.info slope_elev@test_mapset

Delete the test_mapset
Delete the test_mapset (always double check names when deleting)

.. code:: bash

ace --location nc_spm_08 --delete-mapset test_mapset

If the active GRASS GIS session has identical location/mapset settings,
If the active GRASS GIS session has identical location/mapset names,
then an alias can be used to avoid the persistent option in each single
command call:

Expand Down Expand Up @@ -375,25 +420,25 @@ Create new locations
.. code:: bash

# create new location
ace --create-location latlon 4326
ace --create-location latlon_test 4326
# create new mapset within location
ace --location latlon --create-mapset user1
ace --location latlon_test --create-mapset user1

Install GRASS GIS addons (extensions)
--------------------------------------------------------------------------------
.. code:: bash

# list existing addons, see also
# https://grass.osgeo.org/grass7/manuals/addons/
ace --location latlon g.extension -l
ace --location latlon_test g.extension -l

# install machine learning addon r.learn.ml
ace --location latlon g.extension r.learn.ml
ace --location latlon_test g.extension r.learn.ml



What next?
==========
* Visit the actinia website at `https://actinia.mundialis.de <https://actinia.mundialis.de>`_
* actinia tutorial: `https://neteler.gitlab.io/actinia-introduction <https://neteler.gitlab.io/actinia-introduction>`_
* Further reading: Neteler, M., Gebbert, S., Tawalika, C., Bettge, A., Benelcadi, H., Löw, F., Adams, T, Paulsen, H. (2019). Actinia: cloud based geoprocessing. In Proc. of the 2019 conference on Big Data from Space (BiDS'2019) (pp. 41–44). EUR 29660 EN, Publications Office of the European Union 5, Luxembourg: P. Soille, S. Loekken, and S. Albani (Eds.). (`DOI <https://zenodo.org/record/2631917>`__)
* Further reading: Neteler, M., Gebbert, S., Tawalika, C., Bettge, A., Benelcadi, H., Löw, F., Adams, T, Paulsen, H. (2019). Actinia: cloud based geoprocessing. In Proc. of the 2019 conference on Big Data from Space (BiDS'2019) (pp. 41-44). EUR 29660 EN, Publications Office of the European Union 5, Luxembourg: P. Soille, S. Loekken, and S. Albani (Eds.). (`DOI <https://zenodo.org/record/2631917>`__)