Please sign in to comment.
CDAP-13795 clean up run resources on remote hadoop provisioner
Added some cleanup logic to the remote hadoop provisioner to delete the run directory on the remote host when 'deprovisioning' the cluster. Ideally, this logic would reside within CDAP instead of in the provisioner, as CDAP is the one that knows where it copies the files to. However, it requires a bit more work to make sure the cleanup logic always runs, even across master death, due to the asynchronous nature of how run status is recorded and how deprovisioning is kicked off. There would also ideally need to be some way for other provisioners to tell CDAP that it does not need to run cleanup.
- Loading branch information...
Showing with 14 additions and 1 deletion.