duplicate entries in the mbnames file with autorefresh

[ernix]

Hi, I'm using offlineimap installed from Homebrew.
I found my offlineimap frequently hangs and produces huge mbnames file.
I need to reboot the service to sync mails again.

Could you point me out what I missed? Thanks.

mailboxes "+interlink/Archive" "+interlink/Archive" "+interlink/Archive" ...(x 580 times) "+interlink/Drafts" "+interlink/Drafts" ....

General informations

OSX sierra, 10.12.4
offlineimap v7.1.0, imaplib2 v2.57 (bundled), Python v2.7.13

Configuration file offlineimaprc

[general]

accounts = interlink
maxsyncaccounts = 2
ui = TTYUI
pythonfile = ~/.offlineimap.py
socktimeout = 90

[mbnames]

enabled = yes
filename = ~/.mutt/mailboxes
header = "mailboxes "
peritem = "+%(accountname)s/%(foldername)s"
sep = " "
footer = "\n"
mbnames_prune = 1


[Account interlink]

localrepository = interlink-local
remoterepository = interlink-remote
autorefresh = 1
quick = -1
postsynchook = XAPIAN_CJK_NGRAM=1 notmuch new
idlefolders = ['INBOX']


[Repository interlink-remote]

type = IMAP
remotehost = example.com
remoteuser = kojima
remotepasseval = osxkeychain("xxxxxxxxxxxxx")
ssl = yes
sslcacertfile = /usr/local/etc/openssl/cert.pem
holdconnectionopen = yes
idlefolders = ['INBOX']


[Repository interlink-local]

type = Maildir
localfolders = ~/Mail/interlink

pythonfile (if any)

import subprocess

def osxkeychain(account=None):
    command = [ '/usr/bin/security', 'find-generic-password', '-w', '-s', account ]
    password = subprocess.check_output(command, stderr=subprocess.STDOUT)
    return password.strip('\n')

[nicolas33]

Please check your .offlineimap/mbnames directory and report the content of the json files.

[ernix]

Should I attach the file?

$ ls -alh ~/.offlineimap/mbnames/interlink.json
-rw-r--r-- 1 kojima staff 1.5M 5 17 09:18 /Users/kojima/.offlineimap/mbnames/interlink.json

$ cat ~/.offlineimap/mbnames/interlink.json | jq -a '.[] | .foldername' | sort | uniq -c | sort -r | head -n3
539 "INBOX"
462 "xxxxx"
462 "wellnet"

$ cat ~/.offlineimap/mbnames/interlink.json | head -c 1000
[{"localfolders": "/Users/kojima/Mail/interlink", "foldername": "[Readdle]", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "x.xxxxxxxx", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "amazon", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "Archive", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "beijing", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "bill", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "briansakana", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "customer", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "cybozu", "accountname": "interlink"}, {"localfolders": "/Users/kojima/Mail/interlink", "foldername": "dom

[nicolas33]

Should I attach the file?

No, it's ok. It's enough to know that the JSON file has the issue.

Remove this file, it will be recreated on next run.

Then, please track its content for some time in a per-run basis and report if the problem is still there.

[ernix]

The json file is still growing at every syncing when offlinimap is run without -o option.

Since offlineimap -o works correctly, I guess it's related to autoreflesh options.

[nicolas33]

Ok. I'm not surprised it's related to autorefresh. I'll check the code.
Thank you.

[nicolas33]

I could reproduce and fix this issue. Could you please try the above version and report if this fixes the issue for you?

[ernix]

@nicolas33 Thanks, I confirmed that your patch fixes the issue.

Confirmation process:

✔ kojima:kojiMac offlineimap[master]$ git remote
origin
✔ kojima:kojiMac offlineimap[master]$ git remote add nicolas33 git@github.com:nicolas33/offlineimap.git
✔ kojima:kojiMac offlineimap[master]$ git fetch nicolas33
remote: Counting objects: 60, done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 60 (delta 41), reused 41 (delta 41), pack-reused 14
Unpacking objects: 100% (60/60), done.
From github.com:nicolas33/offlineimap
 * [new branch]      b367           -> nicolas33/b367
 * [new branch]      b384           -> nicolas33/b384
 * [new branch]      b401           -> nicolas33/b401
 * [new branch]      b413           -> nicolas33/b413
 * [new branch]      b467           -> nicolas33/b467
 * [new branch]      maint          -> nicolas33/maint
 * [new branch]      master         -> nicolas33/master
 * [new branch]      next           -> nicolas33/next
 * [new branch]      ns/fix-dryrun  -> nicolas33/ns/fix-dryrun
 * [new branch]      ns/threads     -> nicolas33/ns/threads
 * [new branch]      pu             -> nicolas33/pu
 * [new branch]      test-deletions -> nicolas33/test-deletions
✔ kojima:kojiMac offlineimap[master]$ git checkout nicolas33/b467
Note: checking out 'nicolas33/b467'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at f876205... mbnames: don't duplicate entries in autorefresh mode
✔ kojima:kojiMac offlineimap[(f876205...)]$ python offlineimap.py

[nicolas33]

Thank you much; patch applied.