No calendar updates in Android 8+

[iloveeclipse]

I'm using forked new agenda app, and observe that the calendar info seem to be updated once in the day. Means, I see new or deleted or changed appointments only next day OR if I remove the widget and add it again.

This seem to be a regression from the last version of the old app.
I have it on Oreo and Pie, Huawei and Nokia, so it seem to be an app issue.

[haakonhh]

I am unfortunately experiencing the same behavior on my Pixel 3.

[yvolk]

I don't think that failure to update calendar events is specific to the latest build. Please see #182 "Using CalDAV - No widget sync. after calendar sync" more than 3 years ago with our discussion on a proper way to know, when a calendar was updated (which Android notification to catch...). It has links to different documentation sources...

Similar #225 "seems the calendar cannot up to date automatically even enable the tool background running" with a suggestion to introduce auto refresh.
#264 "Failed to refresh" says that "The only case that I'm 100% sure about it, is when I'm scrolling to the bottom of the list, then it refresh the list, and I've got updated"

As of now I don't know any better fix / workaround than to introduce periodic updates of calendar events.
Do you know any Open Source app / widget that reliably refreshes itself on calendar event changes? So we could lookup better solution.

[iloveeclipse]

Google own (ugly) Calendar widget refreshes instantly. I don't know if this is open source?
May be there is some new API addedin Android 8 or 9 which allows them to do so?

[yvolk]

I found code for AOSP/Calendar https://android.googlesource.com/platform/packages/apps/Calendar.git and didn't find any clues at a first sight.
Then I made events update test with my android 7 and 8 devices and found out that the Todo Agenda widget is not updated (at least, for a long time) on Android 8 only.
And finally, looking into device logcat, I saw the problem:
`` W/BroadcastQueue: Background execution not allowed: receiving Intent ...
This means that for Android 8+ we need to handle background updates differently. I will do this soon...

@iloveeclipse @haakonhh What Android version has your device?

[iloveeclipse]

Huawei 8 and Nokia 9.

[iloveeclipse]

Quick google search found this:
https://commonsware.com/blog/2017/04/11/android-o-implicit-broadcast-ban.html

Does it make sense here? I'm far away from Android development but it looks like our issue here.

[iloveeclipse]

Also see
https://stackoverflow.com/questions/53578252/widget-issue-broadcastqueue-background-execution-not-allowed-receiving-intent
https://stackoverflow.com/questions/50894567/widget-case-that-doesnt-work-with-oreo-8-1-message-received-w-broadcastqueue

[haakonhh]

@yvolk Google Pixel 3 with Android 9.

[yvolk]

@iloveeclipse @haakonhh I see, so the problem is really with Android 8 / 9 support.
I already made similar fix for AndStatus application, so this fix should not be a problem.

[yvolk]

I looked deeper, read docs ( https://developer.android.com/about/versions/oreo/background#broadcasts ) and see that the "implicit intent" ( action=android.intent.action.PROVIDER_CHANGED ), which our widget is using to update a list of events after any change, is available in Android 8+ in these cases:

1. The widget is "in the foreground" (meaning explicit and constant User notification on this, which doesn't look practical...).
2. The widget self-register to this intent during its work.
   We cannot make the intent "explicit", because it is generated by unknown Calendar app (and this is that app, which sends the implicit (i.e. to all) intent)...

So it looks like in order to update the widget, we have to use some workarounds, e.g.:

1. Using existing Refresh button (I easily fixed it now...). Maybe optionally making it larger...
2. Self-register on the PROVIDER_CHANGED action on the fist update (e.g. after reboot of manual refresh...
3. Refreshing not only on "Refresh button" click, but on any action on the Widget...
4. Periodic scheduled updates. Schedule of such updates may be smart. E.g. normally refresh once in an hour (configurable), and additionally make several frequent updates (e.g. 5 times once in a minute) after any click on the widget (expecting that a User is making some changes in the Calendar app...)

?!

[yvolk]

When I realized that self-registering on the needed intents works, the fix appeared really easy.
In addition to the PROVIDER_CHANGED action, the widget is now subscribing itself to the USER_PRESENT action, meaning that widgets will be updated each time a User unlocks a screen.

As I noticed, the PROVIDER_CHANGED action occurs about 30 seconds after event change in a Calendar.

Please try the fix in the open Beta channel here: https://play.google.com/apps/testing/org.andstatus.todoagenda

[iloveeclipse]

OK, now "Refresh" button properly refreshes, and also widget refreshes on lock/unlock.
This makes the widget at least usable again on Android >= 8.
Great.
... but there is still no widget refresh after adding/changing/removing an event in calendar app.
If you would manage how to refresh the widget after changing events in the calendar apps, it would be perfect.

[iloveeclipse]

Correction: on Huawei (Oreo, custom Calendar app) I see events immediately after creation in the widget.
On Nokia (Pie, Google Calendar app) this doesn't work.

[iloveeclipse]

On both systems, calendar events created on another system appear only after longer time (~5 minutes?), while the internet calendar.google.com shows changes instantly.

Only opening Calendar app and "refresh" from there synchronizes the widget state. It would be also nice if "Refresh" button in the widget would also trigger calendar sync.

[yvolk]

Only opening Calendar app and "refresh" from there synchronizes the widget state. It would be also nice if "Refresh" button in the widget would also trigger calendar sync.

So now we are talking about speeding up syncing/refresh of a Calendar application. I think that this depends on a Calendar app you are using and maybe that Calendar app has some "sync button widget" for this?

[iloveeclipse]

No, I mean that after I'm creating an event, I see this in the widget only after explicit refresh or after locking/unlocking, but the main problem is fixed now. Great!

[yvolk]

@iloveeclipse After adding a new event, could you wait for a while for the widget's update, don't letting your device to go to sleep, and see, when the widget is updated.
I just tested again on Android 8 device: after changing existing event, the widget is being automatically updated after 30 seconds.

[iloveeclipse]

OK, it seems it takes a bit longer on Nokia, somewhat about one minute. Anyway, release that patch! It is great!

[yvolk]

I added corresponding answer to StackOverflow on this: https://stackoverflow.com/a/54273840/297710

[yvolk]

Recently I measured time between a change in a Google Calendar and corresponding change in the ToDo Agenda Widget:
It looks like the Google's calendar recent syncing changes increased the time, when our widget receives the updates (the same for any other app that listens to updates).
Concrete period, when a change is available in the widget, depends on the change itself, as I see: whether this is a new or updated event...

Today on my Android 10 phone with the latest Google calendar:

• A new event is visible automatically within a minute
• Changes to the same event sometimes are shown within the same time, sometimes later...

It looks like the delay depends on the events sync time between my devices. And my device's "Calendar provider" service (component, provided by Android system) receives any updates only when the syncing over the Internet is finished, not immediately after the change was done on the same device.