Long-running backup fails by timeout

[C-Duv]

I am trying to backup a ~25Go mailbox and the script terminates (during backup) with one of the following errors:

• ERROR: 110 Connection timed out
• ERROR: 104 Connection reset by peer
• ERROR: command: FETCH => socket error: EOF

I guess it's because of the size of the mailbox?
Is there a way to define a longer wait time and/or reconnect to server?

[rcarmo]

Hi there! Thanks for reporting this. I added a -toption to set the global socket timeout that may fix this, try it out and re-open the issue if it doesn't (although I'm afraid that if your server is resetting the connection, the mailbox may indeed be too large, and you're probably best off splitting it into multiple smaller folders).

Re-connection is trickier, because if you're using the script as part of something bigger you may well want to handle re-connection yourself. Regardless, I hope this fixes it for you -- Happy New Year!

[C-Duv]

Hello, I fail to run script using the new -t option: The moment I add the -t option I get the Usage: erroneous message:

All the following commands prints the Usage: message:

python imapbackup.py --server=imap.example.com:993 --user=user@example.com -t 180
python imapbackup.py --server=imap.example.com:993 --user=user@example.com --timeout=180
python imapbackup.py -t 180 --server=imap.example.com:993 --user=user@example.com
python imapbackup.py --timeout=180 --server=imap.example.com:993 --user=user@example.com
python imapbackup.py -t 180 --server=imap.example.com:993 --user=user@example.com
python imapbackup.py ---timeout=180 --server=imap.example.com:993 --user=user@example.com

Here is the Usage: message I get:

Usage: imapbackup [OPTIONS] -s HOST -u USERNAME [-p PASSWORD]
 -a --append-to-mboxes     Append new messages to mbox files. (default)
 -y --yes-overwrite-mboxes Overwite existing mbox files instead of appending.
 -n --compress=none        Use one plain mbox file for each folder. (default)
 -z --compress=gzip        Use mbox.gz files.  Appending may be very slow.
 -b --compress=bzip2       Use mbox.bz2 files. Appending not supported: use -y.
 -f --=folder              Specifify which folders use.  Comma separated list.
 -e --ssl                  Use SSL.  Port defaults to 993.
 -k KEY --key=KEY          PEM private key file for SSL.  Specify cert, too.
 -c CERT --cert=CERT       PEM certificate chain for SSL.  Specify key, too.
                           Python's SSL module doesn't check the cert chain.
 -s HOST --server=HOST     Address of server, port optional, eg. mail.com:143
 -u USER --user=USER       Username to log into server
 -p PASS --pass=PASS       Prompts for password if not specified.
 -t SECS --timeout=SECS    Sets socket timeout to SECS seconds.
 --thunderbird             Create Mozilla Thunderbird compatible mailbox
 --nospinner               Disable spinner (makes output log-friendly)

NOTE: mbox files are created in the current working directory.

If I remove the -t/--timeout= part the commands starts just fine.
I might get something wrong...

Version: Python v2.7.12

(Edit: simplified the command examples to remove optional options that does not change the results)

[rcarmo]

Ooops, sorry. Merged this wrong. Should be fixed now — if not, I’ll check again during the weekend.

[C-Duv]

Yep: works now.

But I noticed the -t seems to be mandatory (even before e1ad2d3), running the command without it returns (after the password prompt):

Traceback (most recent call last):
  File "imapbackup.py", line 713, in <module>
    main()
  File "imapbackup.py", line 600, in main
    server = connect_and_login(config)
  File "imapbackup.py", line 547, in connect_and_login
    if config['timeout']:
KeyError: 'timeout

I guess it is missing from the part that fills config variable with default values, see PR #4