Make job deletion configurable

[saurabhnanda]

Currently there are only two options here:

• cfgDeleteSuccessfulJobs: True or False

I propose something like the following:

data Config = Config
  { -- | This function will be evaluated every time a job is attempted (at the __end__ of the attempt)
   cfgImmediateJobDeletion :: Job -> IO Bool 

   -- | Another polling thread will be setup to run every 'cfgPollingInterval' seconds to execute this query
  , cfgDelayedJobDeletionSql :: PGS.Query
  , ...
  } 

defaultImmediateJobDeletion :: Job -> IO Bool
defaultImmedateJobDeletion Job{jobStatus} = 
  pure $ jobStatus == Success

defaultDeleayedJobDeletionSql :: PGS.Query
defaultDeleayedJobDeletionSql = 
  "DELETE FROM jobs WHERE status in ('success', 'failed') AND run_at < current_timestamp - '7 days'::interval"

Use cases

• cleanup of failed jobs after enough time has passed (this is the same as Implement reaper for failed/stale jobs #22)
• delay a job's deletion for cases where an ephemeral job result
• make job deletion logic highly configurable (useful for advanced use-cases like "workflows" -- coming up in a future PR)

[saurabhnanda]

Folks, if anyone has a strong objection to the above proposal, please let me know.

@ivb-supercede @tfausak @jappeace @tchoutri

Please tag any other active users of odd-jobs as well.

[tchoutri]

LGTM 👍

[jappeace]

would be nice this could be changed without breaking existing code

[saurabhnanda]

@jappeace if the fields of Config are changed but the type-signature of OddJobs.ConfigBuilder.mkConfig is maintained, even , would that work for you?

cc: @tfausak

[jappeace]

I think that's fine.