Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete/Purge sketches and timelines #1602

Open
berggren opened this issue Feb 9, 2021 · 1 comment
Open

Delete/Purge sketches and timelines #1602

berggren opened this issue Feb 9, 2021 · 1 comment

Comments

@berggren
Copy link
Contributor

berggren commented Feb 9, 2021

Batch job to purge sketches and accompanying timelines.

  • Archive before deletion
  • Export and archive zip file before deletion
  • Delete after X days, both sketch, timelines and ES indices
@kiddinn
Copy link
Contributor

kiddinn commented Feb 9, 2021

This means to essentially:

  • Add a new script that checks and executes jobs.
  • Add a daily cron to the docker images that runs the new script.
  • Add a DB schema for these jobs, so that they can be configured.
  • Add an entry into the config file, "PURGE_LABEL = ''" and "DAYS_BEFORE_PURGING" or something like that, which can be used by the script to know what purge actions should take place.
  • The script will have sections in it, which defines the actions it can take, each action there (action class) can read from the database and understand what it needs to do (db schema defines "action name", which maps to an action class and then a JSON structure for configuring the job.
  • As soon as the script is run, it will check the DB for any actions to be taken. Each action then reads the appropriate configuration and does what it needs to do.
  • Pruning is one of these actions, it will start by listing all sketches with the PRUNE label on it, and then check the dates, to see if the label was applied DAYS_BEFORE_PURGING days ago and if so, will export the sketch (that directory needs to be mapped to the host in the docker files) and then purge it.
  • Leave room for other administrative actions, such as archiving sketches, etc.

@jaegeral jaegeral added this to the Future milestone Jul 7, 2021
@berggren berggren removed this from the Future milestone Sep 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants