Should "nikola clean" clear the pydoit cache to deal with stuck states?

[nhthn]

Environment

Python Version: 3.11.0

Nikola Version: 8.2.4

Operating System: Windows

Description:

I somehow got Nikola into a stuck state where I kept getting a cryptic JSONDecodeError any time I invoked nikola build. I did some debugging and found that somewhere deep in the pydoit cache files, there was a JSON string that had an extra closing curly brace at the end. I apologize as I didn't think to save the error message nor was I able to figure out what caused it in the first place. I do remember that it was related to one of the image files, and I wasn't doing anything particularly unusual to achieve this state.

What made this troublesome is that nikola clean didn't fix it, and after a bunch of trial and error I found that deleting the .doit.db.* files fixed it.

I will report back if I ever manage to reproduce the underlying issue, but in the meantime I have a suggestion. Would it make sense for nikola clean to clear the pydoit cache to help users get unblocked from such states if they come up again?

Thanks to all devs for your hard work on Nikola.

[Kwpolska]

nikola clean is based largely on the doit clean command, and it requires the doit database files so that it knows what to delete. I don’t think we can reliably do it here; and I would suggest proposing this as a doit feature first.

[nhthn]

Thanks for the response. Looking into this further, this likely occurred because pydoit uses dbm, which on windows seems to fall back on the dbm.dumb backend. This backend does not support concurrent access as far as I can tell and data corruption has been reported. I don't think there's anything Nikola can reasonably do about this, so I'm closing this.