workdir is removed before process has started

[cehbrecht]

Description

The process workdir is removed before the process has started.

The suspicious line of code is here:

pywps/pywps/response/execute.py

Line 209 in fbd5b3a

self.process.clean()

When I disable it then it works.

@gschwind @jachym Could you please check the reasoning for this code line? Otherwise I would just comment it with a PR.

If your process has ComplexInputs then the workdir is created again and you might not notice this bug.

Steps to Reproduce

Try a process with no ComplexInputs and which creates a file in self.workdir:
https://github.com/cehbrecht/ecas-wps-demo/blob/759a83d390ef28641825eaa1aafde791a4c4a815/ecaswps/processes/wps_index_tn.py#L77

Additional Information

The mentioned line of code was introduced in PR #358.

[gschwind]

Hello, as first thought, I think cleanup should happen as the queue of process is working, i.e. cleaning directories each time services are called, checking for obsolete directories and removing them.

There is also a lot of issue related to the size of the temporary directory.

[cehbrecht]

@gschwind what shall we do now to get this working for the release? Further steps on cleanup handling could follow after the 4.2 release.

[jachym]

Would something like

209                if self.status == WPS_STATUS.FAILED or self.status == WPS_STATUS.SUCCEEDED:
                        .....

help ?

[jachym]

Did this apply for synchronous process call as well or only for async mode?

[cehbrecht]

I have tested this with the chomsky process of Emu:
http://localhost:5000/wps?service=wps&version=1.0.0&request=Execute&identifier=chomsky&DataInputs=times=1&status=true&storeExecuteResponse=true&

The failure happens in the async mode. I have tried your fix and it works:
if self.status == WPS_STATUS.FAILED or self.status == WPS_STATUS.SUCCEEDED:

In the sync mode the temp folder pywps_ is not removed.

[cehbrecht]

There is already a method calling clean:

pywps/pywps/response/execute.py

Line 79 in 2c48403

if (self.status == WPS_STATUS.SUCCEEDED or self.status == WPS_STATUS.FAILED) and clean:

Maybe the status updates are not called in the right way.