cleanup unused functions from management service

[iakkus]

[Environment]: bare metal, Kubernetes (maybe)
[Known affected releases]: master (includes all releases)

The management service has the following functions that are either never called or don't do anything meaningful:

1. clearWorkflowLog (does nothing)
2. clearAllWorkflowLogs (does nothing)
3. prepareWorkflowLog (does nothing)
4. prepareAllWorkflowLogs (does nothing)
5. retrieveWorkflowLog (never called, provides functionality of subset of retrieveAllWorkflowLogs)

retrieveAllWorkflowLogs returns all workflow related logs and it is being used throughout GUI as well as SDK. One can modify it to use an optional parameter to return only a particular log type if needed, making retrieveWorkflowLog fully redundant.

Besides the management service, the GUI needs to be updated to skip the redundant calls to prepareAllWorkflowLogs and directly call retrieveAllWorkflowLogs. SDK already uses retrieveAllWorkflowLogs directly.

This would reduce the resource usage of the management service sandbox.

[manuelstein]

Backwards compatibility
The management service constitutes our API, so removing functions (without a deprecation period) means we're breaking backward compatibility. That's okayish as we haven't reached out first major release, just pointing it out.

clearWorkflowLog IIUC is a placeholder that hasn't been implemented yet with Elasticsearch. Otherwise, I think we need a solution to manage log data in Elasticsearch. AS a side note, we should consider removing log data when removing a function.

[iakkus]

You mean, if somebody has written a piece of code that utilizes the Management service APIs (besides our code, like the SDK)? Sure, we could add the appropriate responses.

clearWorkflowLog and clearAllWorkflowLogs: we can modify them to indeed the delete the logs from Elasticsearch, but I think it might be better if we could automatically delete all logs after a certain amount of time (e.g., 30 days). For example,

• (https://blog.chesterwood.io/2019/06/how-to-delete-old-elasticsearch-logs.html)
• (https://www.elastic.co/guide/en/elasticsearch/client/curator/5.8/ex_delete_indices.html).

@paarijaat, what do you think?

[iakkus]

Implementation started on branch feature/cleanup_management.

[iakkus]

AS a side note, we should consider removing log data when removing a function.

I think perhaps removing the logs of a deleted workflow would make more sense.

[manuelstein]

Yes, of course, as we're not executing functions but workflows.

[iakkus]

New issue to track deletion of workflow logs: #25