Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
privacyidea-token-janitor delete action does not delete all matching tokens #1322
I just stumbled upon this problem, which can be reproduced as follows:
The underlying problem is that we modify the database while doing a paginated search, and this is not only the case for token deletion, but also for things like unassign, disable :-/
Looking about this some more, it turns out to be quite nasty :-/
The bug only happens if the number of tokens is greater than the chunksize (otherwise, pagination has no effect).
But the bug doesn't occur in the following cases:
For the cases in which the action interferes with the filter criterion, we would need to use a solution like in
Here, we do not use the pagination feature, but just delete matching tokens until no tokens are found anymore.
We could fix the bug by checking ahead-of-time if the filter criterion and action will interfere, and using a special implementation in these cases. But checking for such conditions seems pretty hard and error-prone to me.
We originally introduced the feature in #1224 to speed up the search for orphaned tokens. So I have the following proposal: We only use paginated search for actions which do not change the database at all (i.e. CSV export,
Or: we could save the total number of matching tokens in the first invocation of
But this could have unexpected results if some user just deletes or disables some token via the WebUI in parallel.
added a commit
Nov 27, 2018
referenced this issue
Nov 27, 2018
could You explain this a little more? We check if the
is the database locked during the pagination requests? Otherwise this problem would arise there, too.
I'll try :) You're right, for
We could think of falling back to non-paginated queries for cases in which we aren't sure, but this could be complicated too :)
Good point! No, the database is not locked, so all sorts of funny things could happen anyway.