Added option to skip objects the remote fails to provide

[hvanderheide]

To deal with a broken CalDAV implementation (and all-around terrible product) SmarterMail, I needed to be able to skip remote objects that are listed but can't be queried. It even fails to render them in its own calendar webinterface. It seems to fail on certain special characters that can be introduced by external calendar invites. There is little chance it will be fixed upstream, so this seemed the best solution.

Does this feature make sense, as does its implementation?

[WhyNotHugo]

Don't you need to somehow pass this data to this function?

I guess it should be a setting for the storage, is that what you had a mind?

If so, that would be an acceptable change.

[hvanderheide]

Hi @WhyNotHugo, yes correct. I have this in my config:
[storage example]
...
ignore_missing_href = true

I'm not sure about the name of the option. It probably won't be obvious what it does for someone that hasn't read the code

[dilyanpalauzov]

Some CalDAV servers announce URLs with @ , but want %40 in the request (and vice-versa). You might want to percent-encode/decode the characters to find out what the server wants.

[NickHu]

I've encountered this problem with Google Calendar's CalDAV implementation too. I believe that the steps to reproduce it are as follows:

1. import some 'bad' .ics file not from Google Calendar;
2. vdirsyncer will choke on trying to download the .ics file corresponding to the new Google Calendar event - indeed, the API endpoint itself returns 404.

Debug log:

error: Unknown error occurred for calendar_gcal/<redacted>: /caldav/v2/<redacted>/events/%2Bd7b2a65b-4c9e-455a-a3b1-89d26a83f26a@tickets.gwr.com.ics
error: Use `-vdebug` to see the full traceback.
debug:   File "/nix/store/16aybf8fy7n176hhlb38j9ynqcivkwy8-python3.10-vdirsyncer-0.19.0/lib/python3.10/site-packages/vdirsyncer/cli/tasks.py", line 72, in sync_collection
debug:     await sync.sync(
debug:   File "/nix/store/16aybf8fy7n176hhlb38j9ynqcivkwy8-python3.10-vdirsyncer-0.19.0/lib/python3.10/site-packages/vdirsyncer/sync/__init__.py", line 144, in sync
debug:     a_nonempty = await a_info.prepare_new_status()
debug:   File "/nix/store/16aybf8fy7n176hhlb38j9ynqcivkwy8-python3.10-vdirsyncer-0.19.0/lib/python3.10/site-packages/vdirsyncer/sync/__init__.py", line 62, in prepare_new_status
debug:     async for href, item, etag in self.storage.get_multi(prefetch):
debug:   File "/nix/store/16aybf8fy7n176hhlb38j9ynqcivkwy8-python3.10-vdirsyncer-0.19.0/lib/python3.10/site-packages/vdirsyncer/storage/dav.py", line 555, in get_multi
debug:     raise exceptions.NotFoundError(href)

I don't know what ICS files are considered 'bad', but perhaps it has something to do with the %2B in there - perhaps it needs to be not URL encoded to resolve properly. Another hypothesis is that the UID field of the ICS containing a / causes it to be bad.

The contents of this PR are a workaround.

[WhyNotHugo]

A "bad" ICS file would be very useful to reproduce this.

[vwegert]

@hvanderheide In your opinion, would this patch fix #1095 as well or is that a different issue?

[WhyNotHugo]

I don't really know how to test this patch, but it does make sense IMHO.

Have you tested it reliably?

[WhyNotHugo]

Knowing how to reproduce the underlying issue would help test this.

[vwegert]

Have you tested it reliably?

I have not tested this exact patch, but the modification outlined in #1095 which is similar in function, but not configurable. I've had this patch in place since Dec 10th and have had no issues related to this since.

Knowing how to reproduce the underlying issue would help test this.

I completely agree - but at least in my case, I have no idea how the Google backend got into the state I'm seeing.

[hvanderheide]

This PR is already quite old and for me personally obsolete as I moved to a different job. I'm happy to close it if it has little to no value.

@vwegert I agree that it looks like a similar use-case.