[Feature] Handle WPS-1.x/2.x execute request as WPS-3 'job' #126
Labels
process/wps1
Issue related to WPS 1.x processes support
process/wps2
Issue related to WPS 2.x processes support
process/wps3
Issue related to WPS 3.x (REST-JSON) processes support
project/DACCS
Related to DACCS project (https://github.com/orgs/DACCS-Climate)
triage/feature
New requested feature.
Route
/ows/wps
(default path) offers the WPS-1.x/2.x XML interface (maybe also its 2.0 JSON interface with #125).Since this route is served as a standard PyWPS service, it is possible to call an
Execute
request (either with GET/POST variants), but this will run the process directly on an API 'manager' thread instead of some dedicated celery worker thread/docker image. This both consumes the few resources for handling REST-API requests and possibly limit how fast/many job request the actual API can process, as each 'manager' thread becomes synchronously blocked/used until the corresponding process completes (could be a very long workflow).Even worst, because the WPS process is executed directly from the PyWPS service, no actual
job
gets registered inWeaver
and it is therefore impossible to obtain any feedback from the WPS 3.x routes. The only access to process status is via the base PyWPS status URL, which will consume yet another API-manager thread. Finally, when the process completes, no log/exceptions/results will be available, except the bare WPS Status route.Therefore, WPS-1/2 Execute requests should be redirected to the corresponding
/processes/{id}/jobs
call, so that the job gets created and processed using the normal celery worker that will pick it up eventually. Feature #21 must be implemented at the same time as the redirect will have to work the other way around of current implementation (ie: WPS-1/2 Execute => WPS-REST 'celery' execute).Without simultaneous implementation of the features, we will generate a circular loop of execute calls between implementations.
Given that redirections between WPS-1/2 and WPS-3 calls would be implemented here, features in #125 could be done at the same time as it implies similar requirements.
Relates to double-layer monitoring #21.
The text was updated successfully, but these errors were encountered: