KeyError: '15bac9a5ba0b826b' in update_tags: self.mids[mid]

[julian-klode]

Traceback (most recent call last):
  File "/home/jak/bin/gmi", line 8, in <module>
    g.main ()
  File "/home/jak/Downloads/gmailieer/lieer/gmailieer.py", line 136, in main
    args.func (args)
  File "/home/jak/Downloads/gmailieer/lieer/gmailieer.py", line 190, in sync
    self.pull (args, True)
  File "/home/jak/Downloads/gmailieer/lieer/gmailieer.py", line 315, in pull
    self.partial_pull ()
  File "/home/jak/Downloads/gmailieer/lieer/gmailieer.py", line 463, in partial_pull
    r = self.local.update_tags (m, None, db)
  File "/home/jak/Downloads/gmailieer/lieer/local.py", line 303, in update_tags
    fname = self.mids[mid]
KeyError: '15bac9a5ba0b826b'```

[julian-klode]

I was just doing the usual sync thing.

[julian-klode]

I guess the correct thing to do is to just re-download the message if it went missing on the local disk for some reason (not sure why it went missing in the first place, but that's a different story).

[julian-klode]

A full sync fixed it, BTW. Maybe a full sync should be triggered automatically in such a case? Or maybe there should be a warning if the files are missing, but fetch them and continue (so that you can run a full sync eventually when you have time).

[gauteh]

If a message doesn't exist it is always fetched. Something went wrong in this case, and it was not scheduled for fetching. Did you first change labels of a message in web-ui and then delete a message? - gaute

[julian-klode]

OK, so the message was available on the remote side (queried the gid of the message via API explorer) and I'm not aware of me changing tags on it - maybe locally, but I don't recall touching the emails remotely. It only disappeared locally but I'm not sure why.

[julian-klode]

OK, I'm digging through the history list. Meta info from my log was:

Jun 02 00:38:09 jak-x230 gmi[9725]: remote historyId: 12103618
Jun 02 00:38:09 jak-x230 gmi[9725]: pull: partial synchronization.. (hid: 12103095)

In revision 12103168, the message and several others received the TRASH label:

  {
   "id": "12103168",
   "messages": [
    {
     "id": "15bac9a5ba0b826b",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15bac9a6ad3a52ce",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15baca3b0b0ba06c",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15baf137daaf5954",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15bb1608c62507f3",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15bb1bc17569b718",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15bb1fc16b05a649",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15c62fa47e0c5434",
     "threadId": "15bac9a5ba0b826b"
    },
    {
     "id": "15c65baf1cd5b23d",
     "threadId": "15bac9a5ba0b826b"
    }
   ],
   "labelsAdded": [
    {
     "message": {
      "id": "15bac9a5ba0b826b",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "Label_41",
       "CATEGORY_PERSONAL",
       "TRASH",
       "SENT"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15bac9a6ad3a52ce",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "Label_41",
       "CATEGORY_PERSONAL",
       "TRASH",
       "SENT"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15baca3b0b0ba06c",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "Label_41",
       "CATEGORY_PERSONAL",
       "TRASH"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15baf137daaf5954",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "Label_41",
       "CATEGORY_PERSONAL",
       "TRASH",
       "Label_12"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15bb1608c62507f3",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "Label_41",
       "CATEGORY_PERSONAL",
       "TRASH",
       "SENT"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15bb1bc17569b718",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "Label_41",
       "CATEGORY_PERSONAL",
       "TRASH",
       "SENT"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15bb1fc16b05a649",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "Label_41",
       "CATEGORY_PERSONAL",
       "TRASH",
       "Label_12"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15c62fa47e0c5434",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "IMPORTANT",
       "CATEGORY_PERSONAL",
       "TRASH",
       "SENT"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    },
    {
     "message": {
      "id": "15c65baf1cd5b23d",
      "threadId": "15bac9a5ba0b826b",
      "labelIds": [
       "CATEGORY_PERSONAL",
       "TRASH",
       "Label_12",
       "INBOX"
      ]
     },
     "labelIds": [
      "TRASH"
     ]
    }
   ]
  },

[julian-klode]

Ah no, there was an earlier failure:

Jun 02 00:23:09 jak-x230 gmi[12218]: remote historyId: 12103548
Jun 02 00:23:09 jak-x230 gmi[12218]: pull: partial synchronization.. (hid: 12103095)

I also just noticed that the trash label adding revision 12103168 is not the same as the one from the gmi output: 12103618.

[julian-klode]

So the changes to that message on the remote side were:

12103168 +TRASH
12103211 -TRASH
12103807 -Label_41

I guess the culprit might be the +TRASH -TRASH pair in the same pull?

[gauteh]

Julian Andres Klode writes on juni 2, 2017 9:02:

So the changes to that message on the remote side were: ``` 12103168 +TRASH 12103211 -TRASH 12103807 -Label_41 ``` I guess the culprit might be the +TRASH -TRASH pair in the same pull?

Yes, good catch - I think that might result in a bug! And, we know how to reproduce it!

[julian-klode]

I believe inside the if 'labelsRemoved' in h: case of Gmailiieer.partial_pull(), it checks # in case a not_sync tag has been added and removes stuff from added lists, and (re-)adds to the deleted list. But the not_sync tag has been removed there, not added, so it's the wrong way around AFAICT.

[gauteh]

Julian Andres Klode writes on juni 2, 2017 9:10:

I believe inside the ```if 'labelsRemoved' in h:``` case of Gmailiieer.partial_pull(), it checks ```# in case a not_sync tag has been added``` and removes stuff from added lists, and (re-)adds to the deleted list. But the not_sync tag has been removed there, not added, so it's the wrong way around AFAICT.

Yeah, I thought so first too - but then I remembered that the labelIds field contains the current labels, not the labels that have been removed (I think!). - gaute

[julian-klode]

I had that issue again (with an email I archived remotely IIRC), and it started working again after I trashed and untrashed the message in question in the gmail web UI.

[gauteh]

I think this should no longer be an issue since TRASH and SPAM messages are synced.