Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: improve delete queries for janitor command #2540

Merged
merged 17 commits into from
Aug 4, 2021

Commits on May 20, 2021

  1. perf(janitor): improve delete queries

    Improve delete queries by separating the data extraction from actual delete.
    Extraction is made with a configurable limit, using --limit flag, then deletes are made in batch mode with a configurable batch size, using --batch-size flag.
    Default value for limit is 100000 records and default value for batch size is 100 records.
    flavioleggio committed May 20, 2021
    Configuration menu
    Copy the full SHA
    5baea31 View commit details
    Browse the repository at this point in the history
  2. fix(janitor): include unhandled requests in login and consent cleanup

    This uses LEFT JOIN to select also login and consent requests which
    did not result in a complete authentication, i.e. user requested login
    but timed out or user logged in and timed out at consent.
    flavioleggio committed May 20, 2021
    Configuration menu
    Copy the full SHA
    8b6cb9e View commit details
    Browse the repository at this point in the history
  3. fix(janitor): split login and consent requests extraction

    This splits in two independent SELECTs the extraction of login and consent
    requests eligible for deletion. This solves a bug in the single SELECT
    causing deletion of consent requests where matching login requests were
    eligible for deletion and vice versa. With independent SELECTs we keep
    consent requests even if matching login request gets deleted and vice
    versa.
    flavioleggio committed May 20, 2021
    Configuration menu
    Copy the full SHA
    6943655 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    d5bd252 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    1b27346 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2021

  1. test: ensure that user is passing limit and batch size greater than 0

    This adds a check in janitor command handler to ensure that user is not
    passing wrong values for limit anch batch flags. This also adds tests
    for these command line arguments.
    flavioleggio committed May 21, 2021
    Configuration menu
    Copy the full SHA
    d55622b View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    b5b0791 View commit details
    Browse the repository at this point in the history

Commits on Jun 25, 2021

  1. Configuration menu
    Copy the full SHA
    e526182 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    254a271 View commit details
    Browse the repository at this point in the history
  3. fix(janitor): ensure that records to delete are selected in order

    This fix avoids unexpected behaviors if the janitor cli is executed in parallel. In that case, if we do not order records to delete, we might incur in duplicate deletes.
    flavioleggio committed Jun 25, 2021
    Configuration menu
    Copy the full SHA
    744b1c5 View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    e77b610 View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8157887 View commit details
    Browse the repository at this point in the history
  6. Configuration menu
    Copy the full SHA
    84a7794 View commit details
    Browse the repository at this point in the history

Commits on Jul 16, 2021

  1. Configuration menu
    Copy the full SHA
    0ff8afe View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    9103f42 View commit details
    Browse the repository at this point in the history

Commits on Jul 17, 2021

  1. Configuration menu
    Copy the full SHA
    b54d3e4 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2021

  1. Configuration menu
    Copy the full SHA
    12661f2 View commit details
    Browse the repository at this point in the history