Input/Outputs from database to feature/calculation module

[zarch]

If I understood correctly the general architecture: we will have all the data (from the default data set and/or provided by the users) store in the database.

Each module we will receive a REST request to do a certain task.
For example: lets imagine a calculation module that extract some statistics from a list of raster maps for a selected area.
The inputs for this calculation module are:

• a list of raster map
• a multi-polyon geometry
• a list of strings with the statistics that we want to extract.

The output of the calculation module could be a json file like:

{
  "rastname0": {
    "sum": 505000,
    "mean": 500,
    "min": 0,
    "max": 5000
  },
  "rastname1": {
    "sum": 4040,
    "mean": 40,
    "min": 0,
    "max": 40
  }
}

How can the calculation module access to the raster values stored in the database to extract and provide back to the user those numbers?
The REST API will receive what kind of information? I can receive the raster data as GeoTif or as a JSON-like, see the specifications (http://www.finds.jp/rstprv/docs/rasterjson/index.html.en).

{
  "rastername0": {
    "type": "raster",
    "transform": [1, 0, 0, -1, 135, 35],
    "crs": "EPSG:4612",
    "nodata_values": [11],
    "data_types": ["float32"],
    "values": [
      [
        [ 11, 12, 13 ],
        [14, 15, 16]
      ]
    ]
  },
  "rastername1": {
    "type": "raster",
    "transform": [1, 0, 0, -1, 135, 35],
    "crs": "EPSG:4612",
    "nodata_values": [11],
    "data_types": ["float32"],
    "values": [
      [
        [ 21, 22, 23 ],
        [24, 25, 26]
      ]
    ]
  }
}

but this method seems not every efficient because we have to read the value from the database, convert the value to the geotif/json format, in the calculation module we have to import/parse the json to have back the raster data (a lot of IO activities).

Another possibilities could be to have json file with only the database parameters like the rast_id and then the calculation module can use directly this function in the database:

http://www.postgis.org/documentation/manual-svn/RT_ST_SummaryStats.html

to query the database and generate the json file, with this solution we are only reading from the database.

We can have the same problem with the outputs I can generate a geotif or json file and then you have to read and import the file to the database or I can provide to you the raster_id in the database.

[lesly-houndole]

Please find bellow the IT Architecture of Hotmaps:

As you can see main web service module from WP4 (CREM/HES SO) is the orchestrator of all requests from the user interface. Main web service is the only module that have access to the main database(POSTGIS).
When a request is generated from the user interface, main web service will root it to the right module with some specific parameters added at the user request that the module needs for computation.

In the case of a heavy file like a raster file the main web service will send a link URL as parameter of the request to the module. This link allows the module to download the data (like raster file) then computes the result.