403 forbidden dismissing event notification (Bugzilla Bug 1150571)

[kewisch]

Bugzilla Bug 1150571

Date: 2015-04-02T18:21:45+02:00
From: @dblohm7
Assigned To: nobody

Depends on: bug 1101175
Blocker for: bug 1314185
Last updated: 2019-09-11T13:39:25+02:00

[kewisch]

Comment 10111900

Date: 2015-04-02 18:21:45 +0200
From: @dblohm7

Created attachment 8587479
Log

Thunderbird 31.6.0, GData Provider 1.0.4, Lightning 3.3.3

I'm seeing the same stuff that we saw in bug 1101175.

Attached file: gdata.log (text/plain, 2521 bytes)
Description: Log

[kewisch]

Comment 10114927

Date: 2015-04-03 09:18:31 +0200
From: Stefan Sitter [:ssitter] <ssitter@gmail.com>

Is this the problem mentioned in bug 1137249 comment 1?

[kewisch]

Comment 10118059

Date: 2015-04-04 01:56:28 +0200
From: Jan Peterson <jlp+mozilla@peterson.ath.cx>

Created attachment 8588232
calendar.log

Attached file: calendar.log (text/plain, 2571 bytes)
Description: calendar.log

[kewisch]

Comment 10118061

Date: 2015-04-04 01:57:45 +0200
From: Jan Peterson <jlp+mozilla@peterson.ath.cx>

Comment on attachment 8588232
calendar.log

I am seeing this, too. Posted about it on the google group.
https://groups.google.com/d/topic/provider-for-google-calendar/Y6eefaVeA2Y/discussion

[kewisch]

Comment 10118362

Date: 2015-04-04 05:59:11 +0200
From: @dblohm7

(In reply to Stefan Sitter from comment #1)

Is this the problem mentioned in bug 1137249 comment 1?

Unsure. I didn't start seeing this again until the last week or so.

[kewisch]

Comment 10128949

Date: 2015-04-08 02:56:57 +0200
From: @dblohm7

It looks like I can dismiss events that I created for my own calendar. The ones that I have trouble with are events that I have been invited to by someone else.

[kewisch]

Comment 10177461

Date: 2015-04-20 23:31:22 +0200
From: @dblohm7

(In reply to Aaron Klotz [:aklotz] (please use needinfo) from comment #5)

It looks like I can dismiss events that I created for my own calendar. The
ones that I have trouble with are events that I have been invited to by
someone else.

Actually, the events that I've been having a problem with are for recurring meetings whose events were imported during the Google Calendar migration.

[kewisch]

Comment 12804681

Date: 2017-10-31 10:25:09 +0100
From: @mortenlj

Created attachment 8923748
thunderbird.calendar.log

I seem to have a very similar problem, or possibly two similar problems. There are a lot of bugs about dismissing reminders, but I think my problem is either this one, or the one described in #1271867.

I have just added a new google calendar to Thunderbird, and after connection it showed me reminders for a couple hundred events, all in the past. Dismiss All worked fine for most of them, but three reminders remain, and refuse to go away.

If I try clicking Dismiss, I get a dialog about modification failed, asking me to either "discard and reload", or "submit anyway". Which ever selection I go for, I get the same reminder again, and the same options if I try to discard it again. Similar behavior for the other two reminders.

I've enabled debug logging for the calendar (attached, with some privacy edits), and it looks like one of the events are returning a 404 when I try to dismiss, while the others return 403.

I've tried removing the calendar and adding it again, but the same three event reminders show up again.

The only way to get rid of the reminders dialog is to close it using the X in the corner, and then selecting "submit anyway". The next time a reminder triggers, I still have those three, in addition to the new one. The new one will dismiss fine.

Attached file: thunderbird.calendar.log (text/plain, 20195 bytes)
Description: thunderbird.calendar.log

[kewisch]

Comment 12804690

Date: 2017-10-31 10:28:48 +0100
From: @mortenlj

Forgot to add version info:
Linux fimojoha-w530 4.13.4-gentoo #1 SMP Tue Oct 3 10:31:58 CEST 2017 x86_64 Intel(R) Core(TM) i7-3820QM CPU @ 2.70GHz GenuineIntel GNU/Linux

Mozilla Thunderbird 52.4.0 (64-bit)
Lightning 5.4.4
Provider for Google Calendar 3.3

[kewisch]

Comment 13528724

Date: 2018-08-08 00:37:58 +0200
From: @kriston

I also have three reminders that cannot be dismissed.

Windows 10
Mozilla Thunderbird 60.0 (32-bit)
Lightning 6.2
Provider for Google Calendar 4.4.1

Error in log:

Lightning:[calGoogleCalendar] Modifying item Team Meeting failed:2147500037: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "forbidden",
    "message": "Forbidden"
   }
  ],
  "code": 403,
  "message": "Forbidden"
 }
}
 calGoogleCalendar.js:517

[kewisch]

Comment 13899953

Date: 2019-02-02 22:43:24 +0100
From: Andrei <id.mozilla.8dd0@sent.com>

Same issue with
Thunderbird 60.4.0
Lightning 6.2.4
Provider for Google Calendar 4.4.2

This is a really annoying bug. Is by any chance anyone who can take a look? It's a 4 years old issue...

[kewisch]

Comment 14060676

Date: 2019-04-17 17:47:17 +0200
From: stephen.donahue@equitymultiple.com

Same issue with
Thunderbird 60.5.3
Lightning 6.2.5.3
Provider for Google Calendar 4.4.2

[kewisch]

Comment 14354693

Date: 2019-09-11 13:00:55 +0200
From: @kewisch

*** Bug 1524268 has been marked as a duplicate of this bug. ***

[kewisch]

Comment 14354727

Date: 2019-09-11 13:17:27 +0200
From: @kewisch

*** Bug 1524268 has been marked as a duplicate of this bug. ***

[kewisch]

Comment 14354730

Date: 2019-09-11 13:18:36 +0200
From: @kewisch

Interesting info from bug #1454901:

(eivindt from bug #1454901 comment #11)

I am also now seeing this again, but it's not the same behaviour as in my original report. Let me know if you want me to open a separate bug, or keep it in this one.

The behaviour I'm seeing now seems to be only for recurring events that I have been invited to (i.e. they are not my events).

As other people have noted, there's now a 403 Forbidden response, not the 404 in the original report, and there are no response at all in the user interface (no dialogs of any kind).

I don't know why the provider sends a full calendar json to the googleapis in order to dismiss a notification, but I was able to figure out what field that's causing this problem.

If I manually send the same request to the same url with the same authorization header, I also get a 403 Forbidden. But if I remove the 'recurrence' field:

"recurrence" : [
      "RRULE:FREQ=WEEKLY;BYDAY=FR"
],

and then resend, I get a 200 OK response.

Also interesting to notice is that in the 200 OK response, the recurrence field looks like this:

"recurrence": [
        "RRULE:FREQ=WEEKLY;WKST=MO;BYDAY=FR"
    ],

I.e. the WKST=MO; has been added.

After manually posting this, the reminder is dismissed correctly from Thunderbird as well.

[kewisch]

Comment 14354733

Date: 2019-09-11 13:19:51 +0200
From: @kewisch

*** Bug 1137249 has been marked as a duplicate of this bug. ***

[kewisch]

Comment 14354735

Date: 2019-09-11 13:20:50 +0200
From: @kewisch

*** Bug 1263098 has been marked as a duplicate of this bug. ***

[TheCrazyT]

Guess the only thing that is important for dismissing an event-notification is "x-moz-lastack" inside extended properties.
Calling modifyItem of calGoogleCalendar.js is probably a bad idea just for dismissing.

The google documentation says about PATCH:

Updates an event. This method supports patch semantics. The field values you specify replace the existing values.

So there is no need to send everything.

[TheCrazyT]

Oh and I guess they had problems about that already before the current problem:

gdata-provider/src/components/calGoogleCalendar.js

Lines 491 to 492 in d55f7b4

	// some parts are not writable. Using PATCH ignores anything
	// that isn't allowed.

Well guess what ... thats not true.

[kewisch]

Yep, I need to rewrite this to check the differences between oldItem and newItem in a performant way, and then send only what really changed. Not particularly complicated, but needs to be done :)

[frittentheke]

@kewisch thank you very very much for looking into this. This really not just annoying, but makes Google Calendar borderline unuable in Lightning when you have lots of recurring events (think "Scrum dailies" ... I have this problem absolutely solidly.

Is there a hotfix or proposed change that I could test in my environment?

[kewisch]

I don't have any code written for this. Would you or someone on your team be interested in looking into a fix?

[csnook]

This doesn't happen at all if I mark the calendar with the offending events read-only. Does that also break notifications?

It also doesn't happen for instances of the notification that happen after import, as far as I can tell.

It may be easier to avoid ever hitting the codepath than to deal with screwy PATCH semantics.

[TheCrazyT]

The official documentation about PATCH does not mention that values that keep the same won't be overwritten ,guess that was just a wrong assumption.
But now it is a problem because an access validation fails for some fields.
Dismissing should really just send a PATCH with "x-moz-lastack".

[csnook]

I found a simple (but racy) workaround: remove the calendar, then re-add it and mark it read-only immediately, before it finishes importing any of the offending events. Wait until it has finished importing all the events, sync again, close and restart Thunderbird to make absolutely sure it's done importing, and then mark it writable. The problem hasn't come back yet, though I don't understand it well enough to know if that can be relied upon.

It took a while to get the calendar properties to load while the import was running, but fortunately the offenders are more than a thousand events deep in my calendar, so I had time.

[solomonsunder]

I updated to Thunderbird 68.2.1 and installed the equivalent plugin and have the same problem ie 403 forbidden when I do a dismiss. I updated from Thunderbird 68.0 and 4.4 plugin ie the default Ubuntu 18.04 versions which also had these problems.

Removing calendar, doing read only mentioned by the last commenter does not work for me.

[robertjk]

I found a simple (but racy) workaround: remove the calendar, then re-add it and mark it read-only immediately, before it finishes importing any of the offending events. Wait until it has finished importing all the events, sync again, close and restart Thunderbird to make absolutely sure it's done importing, and then mark it writable. The problem hasn't come back yet, though I don't understand it well enough to know if that can be relied upon.

It took a while to get the calendar properties to load while the import was running, but fortunately the offenders are more than a thousand events deep in my calendar, so I had time.

Tried doing exactly that twice, but it doesn't fix anything for me either.

What works for me now is unchecking Show missed reminders for writable calendars in Preferences -> Calendar -> Reminders.

[boschi]

Also have the same issue.
The only workaround that alleviates the problem for me is opening the calendar in Google's web UI, editing the offending events by deleting the reminders, saving, unsubscribing and re-subscribing to the calendar in Thunderbird, then back at the Google Calendar's web UI, re-enabling the reminders (only for the recurring events)

[d3ngar]

Yeah, I'm not sure what's going on here. This has been an annoying bug for years now (5-years actually). And since I've been moving to Thunderbird more and more, especially now that we are quarantined, it's getting really frustrating.

If I can make a suggestion: add the event to a "black-list" that will never ever crop up again in Thunderbird. Essentially exclude it from the calendar completely.

I will now try to do what @robertjk suggested and report back if this works. It's the third time that I remove the calendar.

[kewisch]

@Joaobranquinho When the work in #224 completes, I can make a release.

[peci1]

This should be fixed on the main branch, therefore I'm closing the issue.

Until #224 is merged, the simple workaround is to delete the event. For me, that's a more acceptable workaround than setting to not remind about missed events...

(I've edited this post as originally I reported I'm still getting 403 when trying PATCH via API, but that was wrong config on my side)

[exteon]

So would there be any chance of backporting the fix to patch the current release? It's such an usability showstopper...

[glide007]

is there a way I can take beta or unreleased version?

[eivindt]

I see yet another mention of this bug in #542, and while I'm sure the eventual release from the main branch will fix this, here's my simple patch that fixes this for me (any maybe me only) on the legacy branch:

diff --git a/src/legacy/modules/gdataCalendar.jsm b/src/legacy/modules/gdataCalendar.jsm
index c0b201d..af7a895 100644
--- a/src/legacy/modules/gdataCalendar.jsm
+++ b/src/legacy/modules/gdataCalendar.jsm
@@ -493,7 +493,8 @@ class calGoogleCalendar extends cal.provider.BaseClass {
         // Updating invitations often causes a forbidden error because
         // some parts are not writable. Using PATCH ignores anything
         // that isn't allowed.
-        if (cal.itip.isInvitation(aNewItem)) {
+        if (cal.itip.isInvitation(aNewItem) || aNewItem.organizer.id != "mailto:" + this.mCalendarName) {
+          console.log(`Changing request type to PATCH, since organizer ${aNewItem.organizer.id} != mailto:${this.mCalendarName}`)
           request.type = request.PATCH;
         }
 

I.e. if the organizer of the event is not the current user, change request type to PATCH. I've been running with this patch for a couple of weeks without issues.

[Joaobranquinho]

@eivindt Can you create a PR for it? Would be greatly appreciated :)

[eivindt]

I'll try to do that tomorrow unless someone explains why my change is a stupid idea. ;)

[peci1]

@eivindt This still doesn't work for me.

I see the Changing request type to PATCH in the log, but then the error again:

[calGoogleCalendar] Modifying item *** failed:2147500037: {
 "error": {
  "errors": [
   {
    "domain": "global",
    "reason": "forbidden",
    "message": "Forbidden"
   }
  ],
  "code": 403,
  "message": "Forbidden"
 }
}

[eivindt]

Yes, unfortunately, still not perfect. I have no idea why yours is failing @peci1 your error message seems to indicate no access to the calendar at all, which isn't the type of error my small patch was trying to fix.
I've also got a stuck reminder now, which seems to be caused by the event never being refreshed. I don't know if my patch somehow is causing this or if it is something else.

[peci1]

I'm not sure if it's relevant here, but I have an open support case with Google regarding the calendar API. This is an excerpt:

this error was probably in misunderstanding of the API docs. I had account A and an event in it. Then I shared A's calendar to B as readonly and wanted to call PATCH changing extendedProperties/private in the B account.

The docs on extended properties say:

Shared properties are visible and editable by all attendees of an event, while private properties are set on one attendee's local "copy" of the event. More concretely, private properties are specific to the calendarId and eventId used in the request, while shared properties will be shown regardless of the calendarId used in the request.
I imagined that even user B has a "local" copy of the event, but that would only be true if B were an invited participant of the event. But that was not the case, B had just a readonly share of the calendar. And since private extended properties are unique to a combination of calendarID-eventID, it makes sense that B cannot change the private properties, because calendarID is the same for both A and B.

Maybe it would help to improve wording of this part of docs on https://developers.google.com/calendar/api/guides/extended-properties to also mention that the "local copy" of the event is not made for accounts that share access to the calendar but are not invited.

[dlech]

The @eivindt patch from #3 (comment) fixed a couple of undismissable events that kept popping back up every few minutes and was annoying me for days. Did a PR ever get made?

[eivindt]

Sorry for the late response. I never did make a PR for this. The main reason is that I still have situations where I cannot dismiss event notifications, and digging into these, trying to find the correct resolution has been difficult.

I'll see if I can create a PR for this situation, and create separate issues for the other situations that I see.

[MoralCode]

here is my experience in case it helps: I was experiencing the same issue (was redirected here from one of the closed issues above) where every time my main google calendar (the one with my email address as its name) would refresh/resync id get an alert with reminders that were entirely from events i had been invited to/attended. dismissing them did nothing and i was able to find this issue by searching for the error messages that appeared in the Tools >Developer tools > error console menu when i clicked to dismiss the individual events.

My workaround
After getting annoyed with this, I went on google calendar directly and noticed that i could make edits to only certain parts of the event (most, like the description, had a yellow background, which i suspect means i cant edit that, which makes sense since i dont own the event). I noticed Google calendar would let me edit/remove the reminders though.

After removing reminders from the affected events (and also unsubscribing/re-adding that calendar in thunderbird), i dont get the annoying reminder popups anymore (i think).

I also suspect (although I dont know this codebase and admittedly havent checked myself) that some of the errors from this extension might be because the extension is assuming that none of the fields are editable in a shared event, when it seems like the reminders one at a minimum is designed to be (and maybe things like color too?)

hope this helps, if only to provide an extra workaround

Edit: whoops i realize this may not be new info and may just be a rehash of existing info in this thread - sorry!!!

[peci1]

and also unsubscribing/re-adding that calendar in thunderbird

This doesn't seem necessary. Restarting TB usually helps me.