OfflineIMAP occasionally downloads duplicates for a whole folder.

[petelewis]

I am experiencing an intermittent and seemingly randomly triggered issue, whereby OfflineIMAP redownloads an entire folder, creating duplicates of all messages. This has happened against both a Dovecot IMAP server and an Exchange server working in IMAP mode (and so far, it only appears to have happened with inbox folders, but this could just be related to the fact that they change / are used the most). The issue sometimes occurs after a few days, and sometimes a couple of times in a day. There doesn't appear to be any data loss, and my workaround is to ask Mutt to remove all duplicates and OffineIMAP then happily syncs the deletion of those duplicates. After this, OfflineIMAP carries on as normal (until the next time), but the whole thing obviously causes a lot of additional network traffic.

I did a git bisect, and ended up with the first bad commit being 9e63fa3. This sounds plausible to me, since the comments in the code refer to forgetting what we know about messages.

Note that since this is an intermittent fault, there is some uncertainty in the bisecting process. The way I did it was to mark a commit bad after observing the fault, and mark a commit good after about a week or two of daily usage without observing the issue.

[a3nm]

Couldn't this be the same issue as #174?

[nicolas33]

Not sure. The git bisect tend to show something else. This would require more digging.

[nicolas33]

I could not find logic errors in 9e63fa3. However, I've improved the code factorization in this area in 657f258.

You might like to test current next branch.

[nicolas33]

I wonder this might occur if one IMAP server updates the UIDs without changing UIDVALIDITY.

[nicolas33]

No feedback. Please, open a new issue if it is still valid.

[a3nm]

Sorry for the lack of feedback. I've been using offlineimap 6.7.0 from Debian on two machines intermittently over the last few months, and the problem hasn't recurred yet. (I used to use maxconnections with a setting >1 and I set it to 4, so this may also be why I'm no longer hitting the issue.)

[nicolas33]

Ok, thank you much!