After sync messages appears as new #75

vkuznet · 2014-02-20T16:53:50Z

I have the following problem, if I read my emails and save them into sub-folders in my mutt session and then invoke offlineimap sometimes for some emails it will mark them as new in my sub-folders. I'm attaching screen shot of mutt session where I read all emails, but after the sync some of them appeared as new. I use offlineimap version 6.5.5_0 from OSX MacPorts.
Thanks,
Valentin.

nicolas33 · 2015-01-09T11:49:34Z

Could you dig into this to find the conditions when this occurs, please?

vkuznet · 2015-01-09T14:50:56Z

Hi Nicolas,
I can help if I'll know where to look and what to trigger. So far, here is a
scenario I see that happens. I usually run mutt email client which uses
offlineimap. Within mutt normally I use "offlineimap -u Quiet -qf INBOX" mode
to fetch emails. Then once a week I run "offlineimap -u TTYUI" command to
synchronize my emails with my IMAP server (both commands I invoke via
key-binding shortcut). When I invoke last command I often see couple of emails
within different mailboxes which are marked as new. Then I go to those
mailboxes and "read" those emails. I am confident now that all of them I read
before within my inbox folder. Apart from that I don't see obvious "trigger
condition".

I'll be glad to check options, or anything else, to help with this debugging.
Just let me know.
Thanks,
Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

Could you dig into this to find the conditions when this occurs, please?

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-01-09T15:57:28Z

I think mutt does NOT sync mail changes to disk. There is a sync-mailbox command in mutt to write changes to disk ($ key binding, here).

What might happen is that you read your mails (marked as read in RAM by and for mutt only) but it doesn't sync immediately the changes on disk. For whatever reason, OfflineIMAP might "touch" the read file and mutt detect that the file was read/accessed. Then, I have suspicious that mutt bypass its own RAM cache to avoid conflicts and display the mail as it appears from disk after the OfflineIMAP sync.

Could you test if you still have the issue by adding the sync-mailbox command to your syncing key-bindings?

vkuznet · 2015-01-09T19:46:39Z

Hi Nicolas,
I added sync-mailbox to my key-binding, here how it looks now

macro index O "offlineimap -u TTYUI" "run offlineimap to sync all mail"
macro index o "offlineimap -u Quiet -qf INBOX" "run offlineimap to sync inbox"

but the problem is still there. For instance, I sent today several messages and
a few appeared in my Sent mailbox as new, while other as read. I don't see any
specific pattern. The message which was triggered as new I had CC to myself,
while other which were not triggered was sent to a person.

Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

I think mutt does NOT sync mail changes to disk. There is a sync-mailbox command in mutt to write changes to disk ($ key binding, here).

What might happen is that you read your mails (marked as read in RAM by and for mutt only) but it doesn't sync immediately the changes on disk. For whatever reason, OfflineIMAP might "touch" the read file and mutt detect that the file was read/accessed. Then, I have suspicious that mutt bypass its own RAM cache to avoid conflicts and display the mail as it appears from disk after the OfflineIMAP sync.

Could you test if you still have the issue by adding the sync-mailbox command to your syncing key-bindings?

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-01-09T22:33:21Z

On Fri, Jan 09, 2015 at 11:46:41AM -0800, Valentin Kuznetsov wrote:

I added sync-mailbox to my key-binding, here how it looks now
macro index O "offlineimap -u TTYUI" "run offlineimap to sync all mail"
macro index o "offlineimap -u Quiet -qf INBOX" "run offlineimap to sync inbox"
but the problem is still there.

Yes, mutt must sync its RAM cache to disk /before/ OfflineIMAP starts.
Something like:

macro index o "<sync-mailbox><shell-escape>offlineimap -u Quiet -qf INBOX<enter>" "run offlineimap to sync inbox"

specific pattern. The message which was triggered as new I had CC to myself,
while other which were not triggered was sent to a person.

Will think about that, too.

Nicolas Sebrecht

vkuznet · 2015-01-12T17:24:40Z

Nicolas,

Yes, mutt must sync its RAM cache to disk /before/ OfflineIMAP starts.
Something like:

macro index o "<sync-mailbox><shell-escape>offlineimap -u Quiet -qf INBOX<enter>" "run offlineimap to sync inbox"

I changed my macro definition as you suggested, so far I don't see the problem,
but level of emails I have right now is still low. I want to run in this mode
for a week or so and will report back to you in this ticket.
Valentin.

nicolas33 · 2015-01-12T18:27:14Z

On Mon, Jan 12, 2015 at 09:24:44AM -0800, Valentin Kuznetsov wrote:

I changed my macro definition as you suggested, so far I don't see
the problem, but level of emails I have right now is still low. I
want to run in this mode for a week or so and will report back to
you in this ticket.

No problem. Take the time you wish. This is not urging at all!

Nicolas Sebrecht

vkuznet · 2015-01-14T18:11:00Z

Hi Nicolas,
after a few days of using new macro definition I'd say that problem is still
there. I randomly get mails marked as new in different folders. Some emails
were CC to myself, but other were send to only single address, and some
I get from HyperNews mailing list I'm subscribed to.

If you have any other ideas please let me know and I can test them.
Thanks,
Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

On Mon, Jan 12, 2015 at 09:24:44AM -0800, Valentin Kuznetsov wrote:

I changed my macro definition as you suggested, so far I don't see
the problem, but level of emails I have right now is still low. I
want to run in this mode for a week or so and will report back to
you in this ticket.

No problem. Take the time you wish. This is not urging at all!

Nicolas Sebrecht

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-03-18T22:53:07Z

Could you check with v6.5.7-rc2? Do you know a version where you did not have this problem?

vkuznet · 2015-03-19T07:47:37Z

I'll check version you suggest later this week.
Meanwhile, I rely on macports which provides offlineimap version 6.5.6_0
Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

Could you check with v6.5.7-rc2? Do you know a version where you did not have this problem?

Reply to this email directly or view it on GitHub:
#75 (comment)

vkuznet · 2015-03-19T08:05:27Z

Nicolas,
I just downloaded v6.5.7-rc2 and tested my mutt with it. Unfortunately the
problem is still there.
Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

Could you check with v6.5.7-rc2? Do you know a version where you did not have this problem?

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-03-19T11:34:29Z

Thanks, could you send your configuration file, please?

vkuznet · 2015-03-19T15:18:42Z

Nicolas,
you can find my configuration files here:
https://www.dropbox.com/s/7pmexoui1w6q4ar/mutt.tar.gz?dl=0

In my muttrc I source bunch of other files. All of them can be found in
archive. I only deleted sensitive information.
Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

Thanks, could you send your configuration file, please?

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-03-19T15:41:04Z

I was talking about the offlineimaprc conf file.

vkuznet · 2015-03-19T15:48:29Z

Ahh, sorry, here it is
https://www.dropbox.com/s/h2h4qea79wm5xne/offlineimaprc?dl=0

On 0, Nicolas Sebrecht notifications@github.com wrote:

I was talking about the offlineimaprc conf file.

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-03-19T17:44:58Z

In an attempt to isolate the offending code:

Does new mails happen only after a move from a folder to another?
Try reducing maxconnections.
Try setting up a new configuration file for one of the accounts but without the mbnames thing (change the account name and Maildir path).

vkuznet · 2015-03-27T14:13:14Z

Nicoloas, I tried to reduce maxconnections to 1 and it didn't make any effect.
New mails indeed happen only after moving them from one folder to another.
I didn't tried yet your last suggestion though, still on my todo list.
I'll report later on this.
Thanks,
Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

In an attempt to isolate the offending code:

Does new mails happen only after a move from a folder to another?

Try reducing maxconnections.

Try setting up a new configuration file for one of the accounts but without the mbnames thing (change the account name and Maildir path).

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-03-27T15:00:13Z

Ok, so the downloaded mails reappear as NEW:

when previous sync was -q (quick mode);
for mails that was moved into another folder.

Will be great to know if mbname has impacts.

vkuznet · 2015-03-30T20:20:43Z

Nicolas, I setup single account config file and rerun mutt/offlinemap and still
see the same behavior, i.e. sometimes the messages which I've read and moved
from inbox into another folder appear in these folders as new.
Best,
Valentin.

On 0, Nicolas Sebrecht notifications@github.com wrote:

Ok, so the downloaded mails reappear as NEW:

when previous sync was -q (quick mode);

for mails that was moved into another folder.

Will be great to know if mbname has impacts.

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-03-30T20:33:34Z

Ok. I aim a deep code refactoring. Once done, inserting changes to track down such issues should be made easier. Because of this and because it's not critical, I'm postponing the resolution of this bug. Unless someone else wants to dig into it now.

vkuznet · 2015-03-30T21:25:35Z

I'm fine with that and it can wait.

On 0, Nicolas Sebrecht notifications@github.com wrote:

Ok. I aim a deep code refactoring. Once done, inserting changes to track down such issues should be made easier. Because of this and because it's not critical, I'm postponing the resolution of this bug. Unless someone else wants to dig into it now.

Reply to this email directly or view it on GitHub:
#75 (comment)

nicolas33 · 2015-10-13T16:42:08Z

Deep refactoring will not happen.

pprw · 2018-01-22T10:01:26Z

I have experiment that mutt does not save (to disk) the change of statut (unread -> read) before moving it.

My solution is to add before the moving command.

Here is suggested commands for archiving and deleting:

macro index,pager a <sync-mailbox>":set confirmappend=no delete=yes\n<save-message>=Petur/Archives\n<sync-mailbox>:set confirmappend=yes delete=yes\n"

bind index,pager d noop
macro index,pager d <sync-mailbox>":set confirmappend=no
delete=yes\n<save-message>+Petur/Trash<enter><sync-mailbox>:set
confirmappend=no delete=yes\n" "copy message to the trash"

jakobwuhrer · 2020-12-10T17:18:14Z

I've encountered very similar issues when using mu4e with offlineimap

nicolas33 added the bug label Jan 9, 2015

nicolas33 added this to the undefined milestone Jan 9, 2015

nicolas33 added the waiting submitter feedback label Jan 9, 2015

nicolas33 removed the waiting submitter feedback label Mar 9, 2015

nicolas33 closed this as completed Oct 13, 2015

nicolas33 added the need contributor! label Oct 13, 2015

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

After sync messages appears as new #75

After sync messages appears as new #75

vkuznet commented Feb 20, 2014

nicolas33 commented Jan 9, 2015

vkuznet commented Jan 9, 2015

nicolas33 commented Jan 9, 2015

vkuznet commented Jan 9, 2015

nicolas33 commented Jan 9, 2015

vkuznet commented Jan 12, 2015

nicolas33 commented Jan 12, 2015

vkuznet commented Jan 14, 2015

nicolas33 commented Mar 18, 2015

vkuznet commented Mar 19, 2015

vkuznet commented Mar 19, 2015

nicolas33 commented Mar 19, 2015

vkuznet commented Mar 19, 2015

nicolas33 commented Mar 19, 2015

vkuznet commented Mar 19, 2015

nicolas33 commented Mar 19, 2015

vkuznet commented Mar 27, 2015

nicolas33 commented Mar 27, 2015

vkuznet commented Mar 30, 2015

nicolas33 commented Mar 30, 2015

vkuznet commented Mar 30, 2015

nicolas33 commented Oct 13, 2015

pprw commented Jan 22, 2018 •

edited

Loading

jakobwuhrer commented Dec 10, 2020

After sync messages appears as new #75

After sync messages appears as new #75

Comments

vkuznet commented Feb 20, 2014

nicolas33 commented Jan 9, 2015

vkuznet commented Jan 9, 2015

nicolas33 commented Jan 9, 2015

vkuznet commented Jan 9, 2015

nicolas33 commented Jan 9, 2015

vkuznet commented Jan 12, 2015

nicolas33 commented Jan 12, 2015

vkuznet commented Jan 14, 2015

nicolas33 commented Mar 18, 2015

vkuznet commented Mar 19, 2015

vkuznet commented Mar 19, 2015

nicolas33 commented Mar 19, 2015

vkuznet commented Mar 19, 2015

nicolas33 commented Mar 19, 2015

vkuznet commented Mar 19, 2015

nicolas33 commented Mar 19, 2015

vkuznet commented Mar 27, 2015

nicolas33 commented Mar 27, 2015

vkuznet commented Mar 30, 2015

nicolas33 commented Mar 30, 2015

vkuznet commented Mar 30, 2015

nicolas33 commented Oct 13, 2015

pprw commented Jan 22, 2018 • edited Loading

jakobwuhrer commented Dec 10, 2020

pprw commented Jan 22, 2018 •

edited

Loading