Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New vector layer sampling endpoint "sampling_sync" #8

Merged
merged 12 commits into from
Mar 22, 2022
Merged

New vector layer sampling endpoint "sampling_sync" #8

merged 12 commits into from
Mar 22, 2022

Conversation

neteler
Copy link
Member

@neteler neteler commented Mar 8, 2022

This PR adds a new endpoint
ACTINIA_URL/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync

which somewhat corresponds to the existing timeseries entpoint "Sample a strds by point coordinates, asynchronous call" (see
https://redocly.github.io/redoc/?url=https://actinia.mundialis.de/latest/swagger.json#tag/STRDS-Sampling/paths/~1locations~1{location_name}~1mapsets~1{mapset_name}~1strds~1{strds_name}~1sampling_async/post) as well as the raster_layers sampling_sync endpoint proposed in #7.

It also includes a test case based on the full North Carolina sample dataset (nc_spm_08 location).

/api/v3/swagger.json now shows (see last two lines):

...
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/render
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_async
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_async_geojson
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_sync
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_sync_geojson
...
/locations/{location_name}/mapsets/{mapset_name}/vector_layers
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}/render
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}/sampling_async
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}/sampling_sync
...
cat test_vector_sample_data.json
{
  "points": [
    [
      "p1",
      "638684.0",
      "220210.0"
    ],
    [
      "p2",
      "635676.0",
      "226371.0"
    ]
  ]
}

curl --no-progress-meter ${AUTH} -X POST -H "content-type: application/json" \
     "${actinia}/api/v3/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync" \
     -d @test_vector_sample_data.json

Result:

{
  "accept_datetime": "2022-03-17 13:41:43.981103", 
  "accept_timestamp": 1647524503.9810963, 
  "api_info": {
    "endpoint": "syncephemeralvectorsamplingresource", 
    "method": "POST", 
    "path": "/api/v3/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync", 
    "request_url": "http://127.0.0.1:8088/api/v3/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync"
  }, 
  "datetime": "2022-03-17 13:41:44.467395", 
  "http_code": 200, 
  "message": "Processing successfully finished", 
  "process_chain_list": [
    {
      "list": [
        {
          "flags": "p", 
          "id": "g_region", 
          "inputs": [
            {
              "param": "vector", 
              "value": "zipcodes_wake@PERMANENT"
            }
          ], 
          "module": "g.region"
        }, 
        {
          "flags": "ag", 
          "id": "v_what", 
          "inputs": [
            {
              "param": "map", 
              "value": "zipcodes_wake@PERMANENT"
            }, 
            {
              "param": "coordinates", 
              "value": "638684.0,220210.0,635676.0,226371.0"
            }
          ], 
          "module": "v.what", 
          "stdout": {
            "delimiter": "|", 
            "format": "list", 
            "id": "info"
          }
        }
      ], 
      "version": "1"
    }
  ], 
  "process_log": [
    {
      "executable": "g.region", 
      "id": "g_region", 
      "mapset_size": 485, 
      "parameter": [
        "vector=zipcodes_wake@PERMANENT", 
        "-p"
      ], 
      "return_code": 0, 
      "run_time": 0.10039043426513672, 
      "stderr": [
        ""
      ], 
      "stdout": "projection: 99 (Lambert Conformal Conic)\nzone:       0\ndatum:      nad83\nellipsoid:  a=6378137 es=0.006694380022900787\nnorth:      258102.57214598\nsouth:      196327.52090104\nwest:       610047.86645109\neast:       677060.680666\nnsres:      498.18589714\newres:      500.09562847\nrows:       124\ncols:       134\ncells:      16616\n"
    }, 
    {
      "executable": "v.what", 
      "id": "v_what", 
      "mapset_size": 485, 
      "parameter": [
        "map=zipcodes_wake@PERMANENT", 
        "coordinates=638684.0,220210.0,635676.0,226371.0", 
        "-ag"
      ], 
      "return_code": 0, 
      "run_time": 0.10032153129577637, 
      "stderr": [
        ""
      ], 
      "stdout": "East=638684\nNorth=220210\n\nMap=zipcodes_wake\nMapset=PERMANENT\nType=Area
...
    }
  ], 
  "process_results": [
    {
      "p1": {
        "Acres": "32340.135", 
        "Category": "40", 
        "Database": "/actinia_core/workspace/temp_db/gisdbase_5ce6c4cf9b8f47628f816e89b7767819/nc_spm_08/PERMANENT/sqlite/sqlite.db", 
        "Driver": "sqlite", 
        "East": "638684", 
        "Hectares": "13087.588", 
        "Key_column": "cat", 
        "Layer": "1", 
        "Map": "zipcodes_wake", 
        "Mapset": "PERMANENT", 
        "NAME": "RALEIGH", 
        "North": "220210", 
        "OBJECTID": "286", 
        "PERIMETER": "282815.79339", 
        "SHAPE_Area": "1408742751.36", 
        "SHAPE_Leng": "285693.495599", 
        "Sq_Meters": "130875884.223", 
        "Sq_Miles": "50.5315", 
        "Table": "zipcodes_wake", 
        "Type": "Area", 
        "WAKE_ZIPCO": "1285870010.66", 
        "ZIPCODE": "RALEIGH_27603", 
        "ZIPCODE_": "37", 
        "ZIPCODE_ID": "66", 
        "ZIPNAME": "RALEIGH", 
        "ZIPNUM": "27603", 
        "cat": "40"
      }
    }, 
    {
      "p2": {
        "Acres": "15609.488", 
        "Category": "42", 
        "Database": "/actinia_core/workspace/temp_db/gisdbase_5ce6c4cf9b8f47628f816e89b7767819/nc_spm_08/PERMANENT/sqlite/sqlite.db", 
        "Driver": "sqlite", 
        "East": "635676", 
        "Hectares": "6316.936", 
        "Key_column": "cat", 
        "Layer": "1", 
        "Map": "zipcodes_wake", 
        "Mapset": "PERMANENT", 
        "NAME": "RALEIGH", 
        "North": "226371", 
        "OBJECTID": "298", 
        "PERIMETER": "230773.26059", 
        "SHAPE_Area": "679989401.948", 
        "SHAPE_Leng": "212707.32257", 
        "Sq_Meters": "63169356.527", 
        "Sq_Miles": "24.3898", 
        "Table": "zipcodes_wake", 
        "Type": "Area", 
        "WAKE_ZIPCO": "829874917.625", 
        "ZIPCODE": "RALEIGH_27606", 
        "ZIPCODE_": "39", 
        "ZIPCODE_ID": "2", 
        "ZIPNAME": "RALEIGH", 
        "ZIPNUM": "27606", 
        "cat": "42"
      }
    }
  ], 
  "progress": {
    "num_of_steps": 2, 
    "step": 2
  }, 
  "resource_id": "resource_id-6527a077-a74d-4195-a44c-90a75692bd22", 
  "status": "finished", 
  "time_delta": 0.4863121509552002, 
  "timestamp": 1647524504.4673853, 
  "urls": {
    "resources": [], 
    "status": "http://127.0.0.1:8088/api/v3/resources/actinia-gdi/resource_id-6527a077-a74d-4195-a44c-90a75692bd22"
  }, 
  "user_id": "actinia-gdi"
}

@neteler neteler added the enhancement New feature or request label Mar 8, 2022
This PR adds a new endpoint
`ACTINIA_URL/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync`

which somewhat corresponds to the existing timeseries entpoint "Sample a strds by point coordinates, asynchronous call" (see
https://redocly.github.io/redoc/?url=https://actinia.mundialis.de/latest/swagger.json#tag/STRDS-Sampling/paths/~1locations~1{location_name}~1mapsets~1{mapset_name}~1strds~1{strds_name}~1sampling_async/post) as well as the raster_layers sampling_sync endpoint proposed in #7.

It also includes a test case based on the full [North Carolina sample dataset](https://grass.osgeo.org/download/data/#NorthCarolinaDataset) (`nc_spm_08` location).

/api/v3/swagger.json now shows (see last two lines):
```
...
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/render
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_async
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_async_geojson
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_sync
/locations/{location_name}/mapsets/{mapset_name}/strds/{strds_name}/sampling_sync_geojson
...
/locations/{location_name}/mapsets/{mapset_name}/vector_layers
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}/render
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}/sampling_async
/locations/{location_name}/mapsets/{mapset_name}/vector_layers/{vector_name}/sampling_sync
...
```

```
cat test_vector_sample_data.json
{
  "points": [
    [
      "p1",
      "638684.0",
      "220210.0"
    ],
    [
      "p2",
      "635676.0",
      "226371.0"
    ]
  ]
}

curl --no-progress-meter ${AUTH} -X POST -H "content-type: application/json" \
     "${actinia}/api/v3/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync" \
     -d @test_vector_sample_data.json
```

Result:
```
{
  "accept_datetime": "2022-03-08 13:07:05.361027",
  "accept_timestamp": 1646744825.3610191,
  "api_info": {
    "endpoint": "syncephemeralvectorsamplingresource",
    "method": "POST",
    "path": "/api/v3/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync",
    "request_url": "http://127.0.0.1:8088/api/v3/locations/nc_spm_08/mapsets/PERMANENT/vector_layers/zipcodes_wake/sampling_sync"
  },
  "datetime": "2022-03-08 13:07:05.886810",
  "http_code": 200,
  "message": "Processing successfully finished",
  "process_chain_list": [
    {
      "list": [
        {
          "flags": "p",
          "id": "g_region",
          "inputs": [
            {
              "param": "vector",
              "value": "zipcodes_wake@PERMANENT"
            }
          ],
          "module": "g.region"
        },
        {
          "flags": "aj",
          "id": "v_what",
          "inputs": [
            {
              "param": "map",
              "value": "zipcodes_wake@PERMANENT"
            },
            {
              "param": "coordinates",
              "value": "638684.0,220210.0,635676.0,226371.0"
            }
          ],
          "module": "v.what",
          "stdout": {
            "delimiter": "|",
            "format": "list",
            "id": "info"
          }
        }
      ],
      "version": "1"
    }
  ],
  "process_log": [
    {
      "executable": "g.region",
      "id": "g_region",
      "mapset_size": 485,
      "parameter": [
        "vector=zipcodes_wake@PERMANENT",
        "-p"
      ],
      "return_code": 0,
      "run_time": 0.10027289390563965,
      "stderr": [
        ""
      ],
      "stdout": "projection: 99 (Lambert Conformal Conic)\nzone:       0\ndatum:      nad83\nellipsoid:  a=6378137 es=0.006694380022900787\nnorth:      258102.57214598\nsouth:      196327.52090104\nwest:       610047.86645109\neast:       677060.680666\nnsres:      498.18589714\newres:      500.09562847\nrows:       124\ncols:       134\ncells:      16616\n"
    },
    {
      "executable": "v.what",
      "id": "v_what",
      "mapset_size": 485,
      "parameter": [
        "map=zipcodes_wake@PERMANENT",
        "coordinates=638684.0,220210.0,635676.0,226371.0",
        "-aj"
      ],
      "return_code": 0,
      "run_time": 0.10026431083679199,
      "stderr": [
        ""
      ],
      "stdout": "{\"Coordinates\": {\"East\": \"638684\", \"North\": \"220210\"},\n\"Maps\": [\n{\"Map\": \"zipcodes_wake\",\n\"Mapset\": \"PERMANENT\",\n\"Type\": \"Area\",\n\"Sq_Meters\": 130875884.223,\n\"Hectares\": 13087.588,\n\"Acres\": 32340.135,\n\"Sq_Miles\": 50.5315,\n\"Categories\": [\n{\"Layer\": 1, \"Category\": 40,\n\"Driver\": \"sqlite\",\n\"Database\": \"/actinia_core/workspace/temp_db/gisdbase_09643934cbdf4adfb6b956281d43ee2d/nc_spm_08/PERMANENT/sqlite/sqlite.db\",\n\"Table\": \"zipcodes_wake\",\n\"Key_column\": \"cat\",\n\"Attributes\": {\"cat\": \"40\",\n\"OBJECTID\": \"286\",\n\"WAKE_ZIPCO\": \"1285870010.66\",\n\"PERIMETER\": \"282815.79339\",\n\"ZIPCODE_\": \"37\",\n\"ZIPCODE_ID\": \"66\",\n\"ZIPNAME\": \"RALEIGH\",\n\"ZIPNUM\": \"27603\",\n\"ZIPCODE\": \"RALEIGH 27603\",\n\"NAME\": \"RALEIGH\",\n\"SHAPE_Leng\": \"285693.495599\",\n\"SHAPE_Area\": \"1408742751.36\"}}]}]}\n{\"Coordinates\": {\"East\": \"635676\", \"North\": \"226371\"},\n\"Maps\": [\n{\"Map\": \"zipcodes_wake\",\n\"Mapset\": \"PERMANENT\",\n\"Type\": \"Area\",\n\"Sq_Meters\": 63169356.527,\n\"Hectares\": 6316.936,\n\"Acres\": 15609.488,\n\"Sq_Miles\": 24.3898,\n\"Categories\": [\n{\"Layer\": 1, \"Category\": 42,\n\"Driver\": \"sqlite\",\n\"Database\": \"/actinia_core/workspace/temp_db/gisdbase_09643934cbdf4adfb6b956281d43ee2d/nc_spm_08/PERMANENT/sqlite/sqlite.db\",\n\"Table\": \"zipcodes_wake\",\n\"Key_column\": \"cat\",\n\"Attributes\": {\"cat\": \"42\",\n\"OBJECTID\": \"298\",\n\"WAKE_ZIPCO\": \"829874917.625\",\n\"PERIMETER\": \"230773.26059\",\n\"ZIPCODE_\": \"39\",\n\"ZIPCODE_ID\": \"2\",\n\"ZIPNAME\": \"RALEIGH\",\n\"ZIPNUM\": \"27606\",\n\"ZIPCODE\": \"RALEIGH 27606\",\n\"NAME\": \"RALEIGH\",\n\"SHAPE_Leng\": \"212707.32257\",\n\"SHAPE_Area\": \"679989401.948\"}}]}]}\n"
    }
  ],
  "process_results": {
    "info": [
      "{\"Coordinates\": {\"East\": \"638684\", \"North\": \"220210\"},",
      "\"Maps\": [",
      "{\"Map\": \"zipcodes_wake\",",
      "\"Mapset\": \"PERMANENT\",",
      "\"Type\": \"Area\",",
      "\"Sq_Meters\": 130875884.223,",
      "\"Hectares\": 13087.588,",
      "\"Acres\": 32340.135,",
...
      "\"ZIPCODE_ID\": \"66\",",
      "\"ZIPNAME\": \"RALEIGH\",",
      "\"ZIPNUM\": \"27603\",",
      "\"ZIPCODE\": \"RALEIGH 27603\",",
      "\"NAME\": \"RALEIGH\",",
      "\"SHAPE_Leng\": \"285693.495599\",",
      "\"SHAPE_Area\": \"1408742751.36\"}}]}]}",
      "{\"Coordinates\": {\"East\": \"635676\", \"North\": \"226371\"},",
      "\"Maps\": [",
      "{\"Map\": \"zipcodes_wake\",",
...
      "\"ZIPCODE\": \"RALEIGH 27606\",",
      "\"NAME\": \"RALEIGH\",",
      "\"SHAPE_Leng\": \"212707.32257\",",
      "\"SHAPE_Area\": \"679989401.948\"}}]}]}"
    ]
  },
  "progress": {
    "num_of_steps": 2,
    "step": 2
  },
  "resource_id": "resource_id-fe65998d-f357-4aba-9e56-06bf15b61629",
  "status": "finished",
  "time_delta": 0.525813102722168,
  "timestamp": 1646744825.8867915,
  "urls": {
    "resources": [],
    "status": "http://127.0.0.1:8088/api/v3/resources/actinia-gdi/resource_id-fe65998d-f357-4aba-9e56-06bf15b61629"
  },
  "user_id": "actinia-gdi"
}

```

TODO: improve output formatting.
Co-authored-by: Anika Weinmann <weinmann@mundialis.de
@neteler neteler marked this pull request as ready for review March 17, 2022 13:56
@neteler neteler changed the title Draft: New vector layer sampling endpoint "sampling_sync" New vector layer sampling endpoint "sampling_sync" Mar 17, 2022
Copy link
Member

@marcjansen marcjansen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks nice to my untrained eye. I have added a couple of smaller comments, questions & remarks; none of these are blocking, please take from them what you agree with.

I wouldn't consider this a proper review.

src/actinia_statistic_plugin/endpoints.py Show resolved Hide resolved
src/actinia_statistic_plugin/endpoints.py Outdated Show resolved Hide resolved
src/actinia_statistic_plugin/vector_sampling.py Outdated Show resolved Hide resolved
src/actinia_statistic_plugin/vector_sampling.py Outdated Show resolved Hide resolved
tests/test_vector_sample.py Outdated Show resolved Hide resolved
tests/test_vector_sample.py Outdated Show resolved Hide resolved
tests/test_vector_sample.py Outdated Show resolved Hide resolved
src/actinia_statistic_plugin/endpoints.py Outdated Show resolved Hide resolved
@neteler neteler mentioned this pull request Mar 21, 2022
@neteler neteler merged commit 42aae87 into actinia-org:master Mar 22, 2022
@neteler neteler deleted the new_vector_sampling_endpoint branch March 22, 2022 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants