(KB20267) Add a task to clean up index locks

[jarretlavallee]

Add a very simple task to find and delete index.lock files in the file sync cache directory. It doesn't have any conditional logic, so let me know if you want me to modify it.

[abottchen]

Should we check to confirm that pe-puppetserver is not running before executing the task? My concern is customers running this task when they aren't having the problem and corrupting a normal deploy by deleting the locks. If pe-puppetserver is dead, we know a deploy cannot be running, thus avoiding this possibility.

[jarretlavallee]

@abottchen Good idea. Should we stop the service as a part of the task or fail if the puppetserver service is running?

[abottchen]

If I'm understanding the bug properly, the failed sync kills the puppetserver. If that is true, then I think we should fail if it is running. since that would imply they are not hitting the bug and thus shouldn't be running this task.

However, if it is possible for puppetserver to still be running in spite of this having happened, then I think it makes sense to stop it at the beginning of this task to ensure no deploys happen during it.

[MartyEwings]

I agree I think we should stop the service. If the task is run in error and we remove a valid lock file we could get some weird results, but if we stop the service, there really shouldn't be a lock file unless you have hit this bug, so no harm done

[MartyEwings]

I think some logic to make sure we dont remove a valid lock file would be best. Maybe stopping the server just to be sure? or even checking for the offending message in the recent logs before proceeding

[jarretlavallee]

This change just blatantly stops the service, removes any locks and starts the service. I feel like it may be better to have some conditional logic and a force parameter, but tasks are kind of a pain. It would be nice to use puppet resources to get the status across all platforms without having to parse the DSL. Maybe it is easier to just use the puppet resources with an output in yaml. Let me know what you think of the blunt approach or if I should implement some more logic and parameters in it.

[MartyEwings]

i am a blunt change sort of guy, but echoing @abottchen comments maybe good foundations is what we need to go for here, id love to strike the balance but im also biased with getting this off the ground fast! :)

[abottchen]

I think this is fine. As long as we document the fact that it will stop the service so users know, we are good. There is still the overall question I have about sending output to stdout rather than structured json output, but I can save that for a larger conversation.