KeyError: X-GM-LABELS - Causing GmVault to stop sync

[dazbradbury]

Love the potential of GmVault, but when trying to run a gmvault sync, we keep getting errors, after syncing anywhere from the first 500-3500 emails that state:

Process email num XXX (imap_id:XXXXXXX) from None.
Error: 'X-GM-LABELS'.

=== Exception traceback ===
Traceback (most recent call last):
  File "gmv\gmv_cmd.pyc", line 741, in run
  File "gmv\gmv_cmd.pyc", line 661, in _sync
  File "gmv\gmvault.pyc", line 607, in sync
  File "gmv\gmvault.pyc", line 570, in _sync_emails
  File "gmv\gmvault.pyc", line 488, in _common_sync
KeyError: 'X-GM-LABELS'

=== End of Exception traceback ===

What version of Gmvault do you use ?

Latest (1.9)

On what Operating System (Linux flavour, MacOSX, Win7/XP/Vista) ?

Windows Server (Running on Amazon EC2 Large Instance)

In case of bug or issue, please detail how we can reproduce the problem:

Am using a sync of the format:

gmvault sync --type custom --imap-req 'Since 1-Jan-2015 Before 25-Apr-2015' 'foo.bar@gmail.com'

[gaubert]

@dazbradbury For some reasons, now the Gmail IMAP API can return an empty response.
I will modify the code to handle that issue and restart the request in case of issue.

[gaubert]

@dazbradbury I have implemented a fix that I am testing. I am trying to reproduce the error but it isn't easy in my case ? Will let you know when I have a version available for testing.

[dazbradbury]

Sounds great - thanks!

From my experience it does seem somewhat sporadic, and not actually related to any individual email ID - so can imagine it's hard to reproduce. Just running again now (with the same code base on the same machine) and I can't reproduce - so perhaps Google changed something too?

[gaubert]

I think they did change something because I cannot reproduce it anymore
while a week ago, it was much more frequent.
Anyway I will leave the fix and trigger the exception to be sure that it is
working correctly.

On 8 May 2015 at 12:25, Daz notifications@github.com wrote:

Sounds great - thanks!

From my experience it does seem somewhat sporadic, and not actually
related to any individual email ID - so can imagine it's hard to reproduce.
Just running again now (with the same code base on the same machine) and I
can't reproduce - so perhaps Google changed something too?

—
Reply to this email directly or view it on GitHub
#198 (comment).

[dazbradbury]

Sounds sensible - and thanks for the quick fix and responses!

[drcursor]

Just had the same issue :
Process email num 121088 (imap_id:219296) from 2011-01.
Process email num 121089 (imap_id:541145) from None.
Error: 'X-GM-LABELS'.

No matter how many times I try to run it, it seems to get this error on random emails...

[gaubert]

@drcursor It has been fixed in 1.9.1 which has not yet been released.

[rbrewer]

I also saw this problem just now, but it went away when I restarted with --resume

[rbrewer]

In backing up 100K emails, I've gotten this error 5 times, each time requiring a manual --resume. @gaubert, any chance you could release 1.9.1 with the fixes you have now, and roll whatever changes are holding back a release into 1.9.2? I guess I could fork and just cherry pick the fix for this issue...

[tyru]

@gaubert
I also met this error.
stacktrace is here:
https://gist.github.com/tyru/8e78c41f0af14ebdcf99

I suppose this may be fixed by checking the key exists like this?

Before

    #decode the labels that are received as utf7 => unicode
    new_data[the_id][imap_utils.GIMAPFetcher.GMAIL_LABELS] = \
    imap_utils.decode_labels(new_data[the_id][imap_utils.GIMAPFetcher.GMAIL_LABELS])

After

    #decode the labels that are received as utf7 => unicode
    if imap_utils.GIMAPFetcher.GMAIL_LABELS in new_data[the_id]:
        new_data[the_id][imap_utils.GIMAPFetcher.GMAIL_LABELS] = \
            imap_utils.decode_labels(new_data[the_id][imap_utils.GIMAPFetcher.GMAIL_LABELS])
    else:
        new_data[the_id][imap_utils.GIMAPFetcher.GMAIL_LABELS] = []

gmvault/src/gmv/gmvault.py

Lines 486 to 488 in 1ae1613

	#decode the labels that are received as utf7 => unicode
	new_data[the_id][imap_utils.GIMAPFetcher.GMAIL_LABELS] = \
	imap_utils.decode_labels(new_data[the_id][imap_utils.GIMAPFetcher.GMAIL_LABELS])

[voipmonitor]

I have the same problem. Is it possible to release at least beta version (macos) I'm not able to get pass this error

[kusmierz]

@gaubert any chance to release 1.9.1?