Content Provider returning null when trying to insert new calendars

[ArnyminerZ]

Currently synchronization is not working, the system is not allowing to create the calendars in the system. Logs:

I/icsx5: Manual sync, ignoring network condition
I/icsx5: Synchronizing (forceReSync=false,onlyMigrate=false)
D/icsx5: Creating local calendar from subscription #1
E/WM-WorkerWrapper: Work [ id=f2eb9c5e-1f1a-4ebd-95bc-72d284ac7859, tags={ at.bitfire.icsdroid.SyncWorker } ] failed because it threw an exception/error
    java.util.concurrent.ExecutionException: java.lang.Exception: Couldn't create calendar: provider returned null
        at androidx.work.impl.utils.futures.AbstractFuture.getDoneValue(AbstractFuture.java:516)
        at androidx.work.impl.utils.futures.AbstractFuture.get(AbstractFuture.java:475)
        at androidx.work.impl.WorkerWrapper$2.run(WorkerWrapper.java:317)
        at androidx.work.impl.utils.SerialExecutorImpl$Task.run(SerialExecutorImpl.java:96)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1137)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:637)
        at java.lang.Thread.run(Thread.java:1012)
     Caused by: java.lang.Exception: Couldn't create calendar: provider returned null
        at at.bitfire.ical4android.AndroidCalendar$Companion.create(AndroidCalendar.kt:63)
        at at.bitfire.icsdroid.SyncWorker.updateLocalCalendars(SyncWorker.kt:186)
        at at.bitfire.icsdroid.SyncWorker.doWork(SyncWorker.kt:112)
        at androidx.work.CoroutineWorker$startWork$1.invokeSuspend(CoroutineWorker.kt:68)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664)
I/WM-WorkerWrapper: Worker result FAILURE for Work [ id=f2eb9c5e-1f1a-4ebd-95bc-72d284ac7859, tags={ at.bitfire.icsdroid.SyncWorker } ]

[rfc2822]

Strange, it works here on my real device as well as in the Android 12 emulator…

[ArnyminerZ]

The issue comes from the ID, there seems to be an issue with new credentials. If I manually change the id of the calendar from 1 to 1000, for example, synchronization runs flawlessly.

[rfc2822]

Do you have steps to reproduce so that I can give a try?

[ArnyminerZ]

I'm not sure how to reproduce, started popping up after doing a fresh install of the app. There must be an issue with the Android's db. Maybe removing the ICS account from the system settings, and then uninstalling and reinstalling the app makes it pop up. I've opened #130, which creates the subscription without ID, and then updates the Room DB with the ID generated by the system. This fixed the issue for me, but maybe it's a bit dirty...

[rfc2822]

Can't reproduce it with a fresh install:

1. Install latest dev version from Android Studio (Android 12 emulator)
2. Add https://www.calendarlabs.com/ical-calendar/ics/76/US_Holidays.ics subscription
3. Grant calendar permission in the snackbar
4. Swipe down to refresh

→ Calendar is refreshed

[Brog33]

I'm happy to do some testing to report back. Let me know when I can download a new version to try (I am btw using Android 11).

[ArnyminerZ]

@Brog33 can you go to system settings > Accounts, and remove the ICSx⁵ account? Then wipe the app data of ICSx⁵, add the subscription back, and try to synchronize

[ArnyminerZ]

I also can't reproduce on a clean emulator...

[Brog33]

I noticed that ICSx5 provider in the android account settings does not have calendar sync enabled by default, so I need to manually set the permission (this seem to be separate from the calendar grant to the app itself).

Okay. App+account wiped & subscription added back.

When I manually sync (swipe down) the spinner flashes in an erratic manner until I navigate out from the app and back in. Then the spinner has stopped and it says "not synchronized yet" (Reproduced this 3 times now).

23-02-27-12-04-48.mp4

[ArnyminerZ]

I'm not able to reproduce on other devices anymore, but it's happening on my personal phone. There's definetely an issue with the ID, with Logcat I have found:

Error inserting allowedReminders=0,1,2 sync_events=1 allowedAttendeeTypes=0,2,1,3 _id=1 calendar_color=-4831587 allowedAvailability=0,1 account_name=Subscripcions a calendaris account_type=at.bitfire.icsdroid calendar_displayName=UPV - Examens calendar_access_level=200 into table  Calendars
android.database.sqlite.SQLiteConstraintException: UNIQUE constraint failed: Calendars._id (code 1555 SQLITE_CONSTRAINT_PRIMARYKEY)
     at android.database.sqlite.SQLiteConnection.nativeExecuteForLastInsertedRowId(Native Method)
     at android.database.sqlite.SQLiteConnection.executeForLastInsertedRowId(SQLiteConnection.java:961)
     at android.database.sqlite.SQLiteSession.executeForLastInsertedRowId(SQLiteSession.java:790)
     at android.database.sqlite.SQLiteStatement.executeInsert(SQLiteStatement.java:89)
     at android.database.DatabaseUtils$InsertHelper.insertInternal(DatabaseUtils.java:1306)
     at android.database.DatabaseUtils$InsertHelper.insert(DatabaseUtils.java:1431)
     at com.android.providers.calendar.CalendarDatabaseHelper.calendarsInsert(CalendarDatabaseHelper.java:264)
     at com.android.providers.calendar.CalendarProvider2.insertInTransactionInner(CalendarProvider2.java:2496)
     at com.android.providers.calendar.CalendarProvider2.insertInTransaction(CalendarProvider2.java:2340)
     at com.android.providers.calendar.SQLiteContentProvider.insert(SQLiteContentProvider.java:100)
     at com.android.providers.calendar.CalendarProvider2.insert(CalendarProvider2.java:2313)
     at android.content.ContentProvider.insert(ContentProvider.java:1748)
     at android.content.ContentProvider$Transport.insert(ContentProvider.java:343)
     at android.content.ContentProviderNative.onTransact(ContentProviderNative.java:168)
     at android.os.Binder.execTransactInternal(Binder.java:1302)
     at android.os.Binder.execTransact(Binder.java:1265)

[ArnyminerZ]

I can only think of #130 to fix it. Uninstall and reinstall doesn't fix it.

[rfc2822]

Maybe the calendar ID belongs to another account/app, so that ICSx⁵ doesn't see it, but it's of course anyway not possible to create the calendar with that ID?

So steps would be:

1. Create a calendar with _id=10 that belongs to another app/account
2. Create an ICSx⁵ subscription with id=10
3. Sync ICSx⁵ → it will try to create calendar with LocalCalendar._id = subscription.id = 10
4. This will fail becaues there's already a system calendar with ID 10

Is that possible?

[ArnyminerZ]

Yup, that's the issue

[ArnyminerZ]

I was thinking that maybe the issue is that Google Calendar is the first calendar registered in the system, and that's why ID 1 is in use. That would also explain why I was having issues with the first emulator but not after wiping. I had my Google account loaded on the emulator, but after wiping it got logged out.

[rfc2822]

Hm so what would you suggest?

[ArnyminerZ]

The "safest" option would be:

1. Create the subscription in Room with automatic ID.
2. When synchronizing, when creating the calendar, remove the ID from the subscription.
3. Once the calendar has been created, fetch the assigned ID from the response URI of the insert function.
4. Remove the subscription with the old ID, and add the new one, with the same data, just different IDs.

It's a bit dirty, but I think it's the safest way to make sure there are no conflicts, just let the system assign IDs.

[rfc2822]

It's a bit dirty, but I think it's the safest way to make sure there are no conflicts, just let the system assign IDs.

Yes, the calendar ID should be independent of the subscription ID. So the question is: do we want to put the calendar ID into the Subscriptions table or do we want to put the subscription ID into the Calendar, for instance into a sync column?

I think it's better to put the subscription ID into the calendar because the calendar is dependent of the subscription (managed by the subscription) and not the other way.

Doesn't sound dirty to me.

[ArnyminerZ]

Then, do you suggest adding an extra column to the subscriptions table? Or doing this?

https://github.com/bitfireAT/icsx5/pull/130/files#diff-c9bb24faf03b5cafac55846a10a18d2a2c9178cc506103cb4695b0a64d8cdcef

I think a new column is a bit overkill

[rfc2822]

Then, do you suggest adding an extra column to the subscriptions table? Or doing this?

https://github.com/bitfireAT/icsx5/pull/130/files#diff-c9bb24faf03b5cafac55846a10a18d2a2c9178cc506103cb4695b0a64d8cdcef

That would again create an implied correlation between the Subscription ID and the calendar ID. Maybe PRIMARY KEY doesn't work as expected in one special case, then this would fail, too. I'd make the correlation explicit.

We should have either

• an extra "(system) calendar id" colum in our Subscriptions table, or
• use one of the CAL_SYNC* columns to save the subscription ID

When deciding this we have to take into account that system calendars may be not available anymore (for instance when a backup is restored). When for instance have in our db:

subscription 1:   displayName = someCalendar, local calendar id=23

and then the system is reset, and other apps create 23 calendars, the assignment won't be correct anymore.

If we would save the subscription ID in a CAL_SYNC column, the advantage would be that the ID would only be here when the calendar is actually available. (Of course, it could become incorrect through mechanisms like "Reset ICSx⁵ storage", which would reset our DB but keep the system calendars.)

So we have to deal with those cases somehow.

[ArnyminerZ]

I think we should add the calendar id to the subscription, and not to the calendar through a CAL_SYNC column.

In any case, if the data of the app is wiped, but not the calendar, the next synchronization of ICSx⁵ would add the subscription back just like in the migration process. If the other thing occurs, simply the calendar will be created again.

I think that using the CAL_SYNC column will only complicate things, and also if you want to migrate to Room as our source-of-trust, we shouldn't trust data to the calendar provider that is not on the database.

[rfc2822]

I think that using the CAL_SYNC column will only complicate things, and also if you want to migrate to Room as our source-of-trust, we shouldn't trust data to the calendar provider that is not on the database.

Yeah, probably… let's try with the calendar id in the subscription table!

[ArnyminerZ]

@rfc2822 Take a look at #133.