Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Trying to create 'directory' error #413

Closed
Blind55 opened this issue Nov 28, 2016 · 11 comments
Closed

Trying to create 'directory' error #413

Blind55 opened this issue Nov 28, 2016 · 11 comments
Assignees
@nicolas33
Labels
bug

Comments

@Blind55
Copy link

Blind55 commented Nov 28, 2016 •
edited

General informations

  • system/distribution (with version): ArchLinux (up-to-date)
  • offlineimap version (offlineimap -V): offlineimap v7.0.10, imaplib2 v2.55 (bundled), Python v2.7.12
  • Python version: Python v2.7.12
  • server name or domain: n/a
  • CLI options: n/a

Configuration file offlineimaprc

[general]
accounts = main

[Account main]
remoterepository = GMail
localrepository = MyIMAP

[Repository MyIMAP]
type = IMAP
remotehost = imap.somewhere.com
remoteuser = me
remotepass = notyou
sslcacertfile = /etc/ssl/certs/ca-certificates.crt

[Repository GMail]
type = Gmail
remoteuser = me@gmail.com
remotepass = notyou
auth_mechanisms = PLAIN

nametrans = lambda folder: 'INBOX'
folderfilter = lambda folder: folder.startswith('[Gmail]/All Mail')
sslcacertfile = /etc/ssl/certs/ca-certificates.crt

Logs, error

$ offlineimap
OfflineIMAP 7.0.10
Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Account sync main:
*** Processing account main
Establishing connection to imap.gmail.com:993 (GMail)
Establishing connection to imap.somewhere.com:993 (MyIMAP)
Creating folder INBOX[MyIMAP]
ERROR: Creating folder INBOX on repository MyIMAP
Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])
ERROR: Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])
*** Finished account 'main' in 0:12
ERROR: Exceptions occurred during the run!
ERROR: Creating folder INBOX on repository MyIMAP
Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])

Traceback:
File "/usr/lib/python2.7/site-packages/offlineimap/repository/Base.py", line 267, in sync_folder_structure
local_repo.makefolder(local_name)
File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 551, in makefolder
OfflineImapError.ERROR.FOLDER)

ERROR: Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])

Traceback:
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 278, in syncrunner
self.__sync()
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 344, in __sync
remoterepos.sync_folder_structure(localrepos, statusrepos)
File "/usr/lib/python2.7/site-packages/offlineimap/repository/Base.py", line 267, in sync_folder_structure
local_repo.makefolder(local_name)
File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 551, in makefolder
OfflineImapError.ERROR.FOLDER)

Steps to reproduce the error

This setup ran fine with 7.0.9.
If I run the program as outlined above, I now get the error about INBOX already existing in MyIMAP.
@nicolas33
Copy link
Member

Actually, I'm surprised this configuration worked before v7.0.10:

nametrans = lambda folder: 'INBOX'
folderfilter = lambda folder: folder.startswith('[Gmail]/All Mail')

Here, you're telling offlineimap to not sync "[Gmail]/All Mail" and to translate any other folder name to 'INBOX'.

This is good news that the latest version is complaining.

@Blind55
Copy link
Author

Blind55 commented Nov 29, 2016

Well, I am confused then. The following seems to indicate that offlineimap will do exactly what I am requesting: copy the '[Gmail]/All Mail' folder to the INBOX folder on the other IMAP server.

So, I am curious what I am doing wrong when I try to do that.

$ offlineimap --info
OfflineIMAP 7.0.10
Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
imaplib2: 2.55 (bundled)
Remote repository 'GMail': type 'Gmail'
Host: imap.gmail.com Port: 993 SSL: True
Establishing connection to imap.gmail.com:993 (GMail)
Server supports ID extension.
Server welcome string: * OK Gimap ready for requests from XXX.YYY.ZZZ.AAA somehasnumber
Server capabilities: ('IMAP4REV1', 'UNSELECT', 'IDLE', 'NAMESPACE', 'QUOTA', 'ID', 'XLIST', 'CHILDREN', 'X-GM-EXT-1', 'UIDPLUS', 'COMPRESS=DEFLATE', 'ENABLE', 'MOVE', 'CONDSTORE', 'ESEARCH', 'UTF8=ACCEPT', 'LIST-EXTENDED', 'LIST-STATUS', 'LITERAL-', 'APPENDLIMIT=35651584')

folderfilter= lambda folder: folder.startswith('[Gmail]/All Mail')

nametrans= lambda folder: 'INBOX'

Folderlist:
Drafts -> INBOX (disabled)
INBOX (disabled)
Junk -> INBOX (disabled)
Notes -> INBOX (disabled)
Personal/A -> INBOX (disabled)
Personal/B -> INBOX (disabled)
Personal/C -> INBOX (disabled)
Personal/D -> INBOX (disabled)
Personal/E -> INBOX (disabled)
Personal/F -> INBOX (disabled)
Personal/G -> INBOX (disabled)
Personal/H -> INBOX (disabled)
Queue -> INBOX (disabled)
Sent -> INBOX (disabled)
Trash -> INBOX (disabled)
Work/A -> INBOX (disabled)
Work/B -> INBOX (disabled)
Work/C -> INBOX (disabled)
Work/D -> INBOX (disabled)
Work/E -> INBOX (disabled)
Work/F -> INBOX (disabled)
Work/G -> INBOX (disabled)
[Gmail]/All Mail -> INBOX
[Gmail]/Drafts -> INBOX (disabled)
[Gmail]/Important -> INBOX (disabled)
[Gmail]/Sent Mail -> INBOX (disabled)
[Gmail]/Trash -> INBOX (disabled)

Local repository 'NSOIMAP': type 'MappedIMAP'
Host: imap.somewhere.com Port: None SSL: True
Establishing connection to imap.somewhere.com:993 (MyIMAP)
Server supports ID extension.
Server welcome string: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
Server capabilities: ('IMAP4REV1', 'LITERAL+', 'SASL-IR', 'LOGIN-REFERRALS', 'ID', 'ENABLE', 'IDLE', 'SORT', 'SORT=DISPLAY', 'THREAD=REFERENCES', 'THREAD=REFS', 'THREAD=ORDEREDSUBJECT', 'MULTIAPPEND', 'URL-PARTIAL', 'CATENATE', 'UNSELECT', 'CHILDREN', 'NAMESPACE', 'UIDPLUS', 'LIST-EXTENDED', 'I18NLEVEL=1', 'CONDSTORE', 'QRESYNC', 'ESEARCH', 'ESORT', 'SEARCHRES', 'WITHIN', 'CONTEXT=SEARCH', 'LIST-STATUS', 'SPECIAL-USE', 'BINARY', 'MOVE')

Folderlist:
Archive
Drafts
INBOX
Junk
Sent
Trash

@nicolas33 nicolas33 added bug and removed question labels Nov 29, 2016
@nicolas33
Copy link
Member

My bad, this nametrans makes sense when all the folders are filtered but one.

@nicolas33
Copy link
Member

@aroig you were right about the wrong use of remote names while comparing structures. The translations are currently broken. Hence, v7.0.10 is broken for some users.

The reason is that we blindly assume that local_folder.getvisiblename() provides the remote name while it might not.

nicolas33 added a commit to nicolas33/offlineimap that referenced this issue Nov 29, 2016
@nicolas33
repository: Base: fix folder structure comparison
6b6fa4f 
481efa9 is wrong to assume the local nametrans is set up and working.

Now, we correctly check that the local folder name is the same once both
nametrans are applied to itself. This check is only done when we have to create
the folder on remote.

Github-ref: OfflineIMAP#413
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
@nicolas33
Copy link
Member

nicolas33 commented Nov 29, 2016 •
edited

I think I've fixed all known issues with this patch: nicolas33@6b6fa4f

@aroig @avar @Blind55 I look forward to hearing from you.

@nicolas33 nicolas33 self-assigned this Nov 29, 2016
@Blind55
Copy link
Author

Blind55 commented Nov 30, 2016 •
edited

EDIT 2: Yes. The patch fixed the issue for me.

EDIT 1: Duh - please ignore the below. I am recompiling with the correct branch.

Sorry to disappoint, I am still getting the error. I checked out branch b367, compiled and ran. gitk shows your patch.

The error seems to be that, for whatever reason, on the non-Google account offlineimap wants to create the 'INBOX' folder, that of course already exists...

Error is very similar:

$ offlineimap
OfflineIMAP 7.0.9
Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Account sync main:
*** Processing account main
Establishing connection to imap.gmail.com:993 (GMail)
Establishing connection to imap.somewhere.com:993 (MyIMAP)
Creating folder INBOX[MyIMAP]
ERROR: Creating folder INBOX on repository MyIMAP
Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])
ERROR: Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])
*** Finished account 'main' in 0:03
ERROR: Exceptions occurred during the run!
ERROR: Creating folder INBOX on repository MyIMAP
Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])

Traceback:
File "/usr/lib/python2.7/site-packages/offlineimap/repository/Base.py", line 266, in sync_folder_structure
local_repo.makefolder(local_name)
File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 551, in makefolder
OfflineImapError.ERROR.FOLDER)

ERROR: Folder 'INBOX'[MyIMAP] could not be created. Server responded: ('NO', ['[ALREADYEXISTS] Mailbox already exists'])

Traceback:
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 278, in syncrunner
self.__sync()
File "/usr/lib/python2.7/site-packages/offlineimap/accounts.py", line 344, in __sync
remoterepos.sync_folder_structure(localrepos, statusrepos)
File "/usr/lib/python2.7/site-packages/offlineimap/repository/Base.py", line 266, in sync_folder_structure
local_repo.makefolder(local_name)
File "/usr/lib/python2.7/site-packages/offlineimap/repository/IMAP.py", line 551, in makefolder
OfflineImapError.ERROR.FOLDER)

@aroig
Copy link
Member

aroig commented Nov 30, 2016

@nicolas33 nicolas33/offlineimap@6b6fa4f looks good. I've tested it with local nametrans and no problems there. Right now it is a pain to test it without local nametrans because of the fmd5 issue... so I have not tested it that way.

@nicolas33
Copy link
Member

@Blind55 I still don't get why it fails for you with nicolas33@6b6fa4f.

The offending check is local_name in local_hash.keys() which should return True at https://github.com/nicolas33/offlineimap/blob/6b6fa4f00d35f4684eb5eb831f89fd5086b2a345/offlineimap/repository/Base.py#L202.

For whatever reason, local_name is not INBOX or local_hash.keys() is missing "INBOX".

@nicolas33
Copy link
Member

Right now it is a pain to test it without local nametrans because of the fmd5 issue...

@aroig Are you saying that migrating from v7.0.9 to nicolas33@6b6fa4f implies FMD5 changes? Or you didn't try because of the previous issues you had with FMD5?

@aroig
Copy link
Member

aroig commented Nov 30, 2016

@nicolas33 @Blind55 is saying that nicolas33/offlineimap@6b6fa4f indeed works for him in an edit to his message.

@aroig Are you saying that migrating from v7.0.9 to nicolas33/offlineimap@6b6fa4f implies FMD5 changes?

No no. As far as I can see nicolas33/offlineimap@6b6fa4f is very ok.

I'm testing directly with all my email. Because of the issues the other day, I added a local nametrans, which invalidated all uids in my email because of the md5. Then I fixed the fmd5 manually for all 200k messages in my local mailbox.

Now I'm stuck with the local nametrans. If I want to test without it, either I should rename the files again, or setup a testing mailbox, which would be a prudent thing to do anyways... ;) I haven't done that.

@nicolas33
Copy link
Member

@Blind55 is saying that nicolas33/offlineimap@6b6fa4f indeed works for him in an edit to his message.

Alright! I've read the message the other way around.

I'm testing directly with all my email.

Yes, this is what I do too. However, I've put the maildirs and offlineimap cache into a filesystem supporting snapshots (btrfs) to revert from issues easily. On top of this I have a testing account.

Because of the issues the other day, I added a local nametrans, which invalidated all uids in my email because of the md5. Then I fixed the fmd5 manually for all 200k messages in my local mailbox.

Now I'm stuck with the local nametrans.

Hmm... The FMD5 is built from the translated local name! Ok. I understand more of #411. :-)

Thanks both for your feedback!

nicolas33 added a commit that referenced this issue Nov 30, 2016
@nicolas33
repository: Base: fix folder structure comparison
2e2684d 
481efa9 is wrong to assume the local nametrans is set up and working.

Now, we correctly check that the local folder name is the same once both
nametrans are applied to itself. This check is only done when we have to create
the folder on remote.

Github-ref: #413
Signed-off-by: Nicolas Sebrecht <nicolas.s-dev@laposte.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nicolas33 nicolas33

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@nicolas33 @Blind55 @aroig