Editing entries with multiple simultaneous jrnl processes overwrites journal

[micahellison]

Bug Report

Environment

• jrnl --diagnostic output:

jrnl: v2.5
Python: 3.7.5 (tags/v3.7.5:5c02a39a0b, Oct 15 2019, 00:11:34) [MSC v.1916 64 bit (AMD64)]
OS: Windows 10

• Install method: pipx

Current Behavior

In two different terminals, use jrnl --edit with different search parameters to start editing different entries in your journal. Save your edits in one session and watch it end. Then save your edits in another session. All edits from the first session will be lost.

Relates to #1106 but it's not the same and the solution for this issue is likely to be much more complex.

Expected Behavior

Ideally, the edits from both editor sessions should be preserved. This could get tricky if the entries overlap between the two sessions, however. Should the entries be overwritten because the edits are intentional? Or should jrnl create new entries rather than losing data?

Repro Steps

With an empty journal:

jrnl First entry
jrnl Second entry
jrnl --edit -contains "First"
(in a different terminal): jrnl --edit -contains "Second"

Modify the second entry, save, and close. Then modify the first entry, save, and close. The modifications to the second entry will be lost.

Other Information

It seems like we might need unique IDs to deal with this problem. Another method could be to generate those IDs on the fly and use those to identify individual entries.

[KarimPwnz]

We should have unique IDs in Entry with the following saving flow:

1. Check if journal modified (through checksum or last modification date)
2. If False: continue normal flow
3. If True: get latest journal version and check if modified entries overlap
4. If False: update entries in latest journal version and save
5. If True: put latest version of the entries in a temp file, open the temp file in configured editor, and then apply changes to the temp file's entries; that way, the editor handles overlaps

What do you think?

[wren]

We thrown around the idea of unique IDs before (it would help a lot of other features). I'm all for it, but the current jrnl storage format doesn't support any additional metadata. Changing the storage format is a big enough change that we'd have to increment to v3.0. I still want to do it, but I want to be sure we think through whatever the new format should look like before we change it. This would also affect some other issues (#949, #1026, #1051, and #696 come to mind), so we should keep that in mind.

[MinchinWeb]

Does this still happen if you use a "DayOne" jrnl type? (as it is already using unique per-entry identifiers).

[wren]

So, it looks like @johnystar ran into this (sorry, again!). I still think that the ultimate fix for this will have to happen in the v3 format (whatever that ends up looking like), but I'm trying to think of how to best mitigate this issue sooner if at all possible.

@micahellison and I were thinking we might be able to read the jrnl file twice (once when the edit command is run, and again as we save). This would require implementing a matching algorithm, which is sure to have some pitfalls (maybe we can do some brainstorming on gitter), but might be better than the current state (especially for new users). This could mostly fix the issue for v2, and buy us some more time to plan out v3.

[wren]

@MinchinWeb I do think this also happens on DayOne types because we read the files into memory when the initial command is run.

[wren]

Oh, also, we could not write anything to the file if nothing was edited (much like we do when writing a new entry). This could also reduce the impact of this bug.

[bytesmith-ahmad]

It has been 3 years and this problem still persists :(
Sad to learn you can't properly use jrnl in separate terminals at once. No solution in sight?