Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

An "unsafe character" (+) seems to be derailing the sync process when syncing with Google. #865

Open
xplreitr opened this issue Feb 2, 2021 · 4 comments

Comments

@xplreitr
Copy link

xplreitr commented Feb 2, 2021

Before you submit bug reports: https://vdirsyncer.pimutils.org/en/stable/contributing.html

Things to include in your bugreport:

  • Your vdirsyncer version
    0.16.8
  • Your Python version
    Python 3.7.9
  • Your operating system
    alpine docker container on ubuntu
  • Your config file
[general]
status_path = "/calendardata/.vdirsyncer/status/"

[pair calendar_test]
a = "my_calendar_local"
b = "my_calendar_remote"
collections = ["from a", "from b"]

[storage my_calendar_local]
type = "filesystem"
path = "/calendardata/test_calendar/"
fileext= ".ics"

[storage my_calendar_remote]
type = "google_calendar"
token_file = "/calendardata/.google_auth/"
client_id = "supersecretclientid"
client_secret = "supersercretsecret"
  • Use vdirsyncer -vdebug for debug output. The output is sensitive, but
    please attach at least the last few lines before the error (if applicable),
    censored as necessary. This is almost always the most useful information.

There was a debugging note right before the error regarding an "unsafe character" which is the + symbol translated into %2b. I included snippets of the debug output surrounding the offending character. Ellipses show gaps (that I would think are fairly irrelevant) cut out.

debug: Using 1 maximal workers.
Syncing calendar_test/calendar_name
debug: PROPFIND https://apidata.googleusercontent.com/caldav/v2/calendar_name/events/
debug: {'User-Agent': 'randomnumbers.apps.googleusercontent.com', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n            <D:propfind xmlns:D="DAV:">\n                <D:prop>\n                    <D:resourcetype/>\n                    <D:getcontenttype/>\n                    <D:getetag/>\n                </D:prop>\n            </D:propfind>\n            '
debug: Sending request...
debug: 207
debug: {'Date': 'Mon, 01 Feb 2021 23:05:42 GMT', 'Pragma': 'no-cache', 'Content-Type': 'text/xml; charset=UTF-8', 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'Vary': 'Origin, X-Origin, Referer', 'Content-Encoding': 'gzip', 'Server': 'ESF', 'X-XSS-Protection': '0', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Alt-Svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'}
...
debug: b'<?xml version="1.0" encoding="UTF-8"?>\n<D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">\n ... ... <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n  <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0686402-108582152.ics</D:href>\n  <D:propstat>\n   <D:status>HTTP/1.1 200 OK</D:status>\n   <D:prop>\n    <D:getcontenttype>text/calendar; component=vevent</D:getcontenttype>\n    <D:getetag>"63518914751"</D:getetag>\n    <D:resourcetype/>\n   </D:prop>\n  </D:propstat>\n </D:response>\n ... ...D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n  <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0680272-245422177.ics</D:href>\n  <D:propstat>\n   <D:status>HTTP/1.1 200 OK</D:status>\n   <D:prop>\n    <D:getcontenttype>text/calendar; component=vevent</D:getcontenttype>\n    <D:getetag>"63531356273"</D:getetag>\n    <D:resourcetype/>\n   </D:prop>\n  </D:propstat>\n </D:response>\n ... ... /D:multistatus>\n'
...
debug: Normalized URL from '/caldav/v2/calendar_name/events/m6icp53ietetp7l1gqe15540lk%40google.com.ics' to '/caldav/v2/calendar_name/events/m6icp53ietetp7l1gqe15540lk@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/6pftiq1k4nm3dt9866thuo3br0%40google.com.ics' to '/caldav/v2/calendar_name/events/6pftiq1k4nm3dt9866thuo3br0@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0686402-108582152.ics' to '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/248h42p0kesa2tvhsrj4c04n3k%40google.com.ics' to '/caldav/v2/calendar_name/events/248h42p0kesa2tvhsrj4c04n3k@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/6pj3gcj36dgm4b9n6phm2b9k74s3ebb275im6bb46hj68chh64s66p9ic8%40google.com.ics' to '/caldav/v2/calendar_name/events/6pj3gcj36dgm4b9n6phm2b9k74s3ebb275im6bb46hj68chh64s66p9ic8@google.com.ics'
...
debug: Normalized URL from '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E029A129CFACD0017700001706C698DFBD5A36141BDE96A1B1058C02B.ics' to '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E029A129CFACD0017700001706C698DFBD5A36141BDE96A1B1058C02B.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/tooubg46aa4rfimr0up83370h8%40google.com.ics' to '/caldav/v2/calendar_name/events/tooubg46aa4rfimr0up83370h8@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0680272-245422177.ics' to '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E07E35C50B92D4017700001708697308B07EEDA4F95AE7E466CF804AD.ics' to '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E07E35C50B92D4017700001708697308B07EEDA4F95AE7E466CF804AD.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/cdh3ae3464p3abb66hhj4b9k64p30b9pc9j3cb9h6so66d35c9i38p9kco%40google.com.ics' to '/caldav/v2/calendar_name/events/cdh3ae3464p3abb66hhj4b9k64p30b9pc9j3cb9h6so66d35c9i38p9kco@google.com.ics'
...
debug: Already normalized: '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E0087001013269558E0D201770000170B92F3FD560EB8F48B07E8F4C3FCA120E.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/apaqr9f9rpo4pppefe9h9cp1ks@google.com.ics'
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/ik7bg09ub8cjs2p1i3nkdk3114@google.com.ics'
...
debug: Already normalized: '/caldav/v2/calendar_name/events/9s46nk3pb4nho69g592b5fd65g@google.com.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/3ffviaqu42ev1e2lj1proijedc@google.com.ics'
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/5608r6i8nq19msdp1anr100s3v@google.com.ics'
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n        <C:calendar-multiget xmlns:D="DAV:"\n            xmlns:C="urn:ietf:params:xml:ns:caldav">\n            <D:prop>\n                <D:getetag/>\n                <C:calendar-data/>\n            </D:prop>\n            <D:href>/caldav/v2/calendar_name/events/or57be33sferu9juai0vn13u20@google.com.ics</D:href>... ...\n<D:href>/caldav/v2/calendar_name/events/9s46nk3pb4nho69g592b5fd65g@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/3ffviaqu42ev1e2lj1proijedc@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/5608r6i8nq19msdp1anr100s3v@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/0aln4lncn4oml6n60ubesepo24@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/b4i7bp6ialrv59cv1cga2mgqao@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/02nmc16u3nvrbcu7drajeut5a9@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/C38E9895-D6BA-4C30-9FFF-270FF248EA7A.ics</D:href> ... ...       </C:calendar-multiget>'
...
debug: Sending request...
debug: 207
debug: {'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Content-Type': 'text/xml; charset=UTF-8', 'Date': 'Mon, 01 Feb 2021 23:06:00 GMT', 'Expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'Pragma': 'no-cache', 'Vary': 'Origin, X-Origin, Referer', 'Content-Encoding': 'gzip', 'Server': 'ESF', 'X-XSS-Protection': '0', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Alt-Svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'}
...
debug: b'<?xml version="1.0" encoding="UTF-8"?>\n<D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">\n ... ...<D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n  <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics</D:href>\n  <D:status>HTTP/1.1 404 Not Found</D:status>\n </D:response>\n <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n  <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics</D:href>\n  <D:status>HTTP/1.1 404 Not Found</D:status>\n </D:response>\n</D:multistatus>\n'
...
debug: Already normalized: '/caldav/v2/calendar_name/events/uec002tktnj5if16b4lvb7tsa8@google.com.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/tlp35ob9j4rl3fi8l2o4vjbp0g@google.com.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/ts4p2oqivvt4i2abj7qhsanhuk@google.com.ics'
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics'
debug: Skipping '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics', properties are missing.
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics'
debug: Skipping '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics', properties are missing.
error: Unknown error occurred for calendar_test/calendar_name: /caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics
error: Use `-vdebug` to see the full traceback.
debug:   File "/usr/local/lib/python3.7/site-packages/vdirsyncer/cli/tasks.py", line 67, in sync_collection
debug:     partial_sync=pair.partial_sync
debug:   File "/usr/local/lib/python3.7/site-packages/vdirsyncer/sync/__init__.py", line 135, in sync
debug:     b_nonempty = b_info.prepare_new_status()
debug:   File "/usr/local/lib/python3.7/site-packages/vdirsyncer/sync/__init__.py", line 58, in prepare_new_status
debug:     if prefetch else ()):
debug:   File "/usr/local/lib/python3.7/site-packages/vdirsyncer/storage/dav.py", line 503, in get_multi
debug:     raise exceptions.NotFoundError(href)
error: 1 out of 12 tasks failed.
~                                                                                                                                                                                                                

Reading the output, it seems the href cant be found. Is that because the href is using the encoded character? Any ideas on how to resolve the issue?

@Accacio
Copy link

Accacio commented Apr 9, 2021

I have exactly the same error with WEBEX meetings

@jvasile
Copy link

jvasile commented Apr 14, 2021

I have this exact error with WEBEX meetings as well.

In #491 an "unsafe character" error was fixed by removing it from the reserved char list. I tried that here, removing the + from line 29 of dav.py, but now I get a new error:

debug: Already normalized: '/caldav/v2/james@opentechstrategies.com/events/WEBEX-MEETING%2BCENTER-6.012920952-1184426117.ics'
debug: Skipping '/caldav/v2/james@opentechstrategies.com/events/WEBEX-MEETING%2BCENTER-6.012920952-1184426117.ics', properties are missing.
error: Unknown error occurred for personal_calendar/james@opentechstrategies.com: /caldav/v2/james@opentechstrategies.com/events/WEBEX-MEETING%2BCENTER-6.012920952-1184426117.ics
error: Use `-vdebug` to see the full traceback.
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/cli/tasks.py", line 68, in sync_collection
debug:     sync.sync(
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/sync/__init__.py", line 138, in sync
debug:     b_nonempty = b_info.prepare_new_status()
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/sync/__init__.py", line 60, in prepare_new_status
debug:     for href, item, etag in (self.storage.get_multi(prefetch)
debug:   File "/home/james/src/3p/vdirsyncer/vdirsyncer/storage/dav.py", line 507, in get_multi
debug:     raise exceptions.NotFoundError(href)
error: 1 out of 2 tasks failed.

@dchersey
Copy link

dchersey commented Oct 5, 2021

I have this too ... fortunately I don't have too many webex meetings.
I worked around it by finding the webex meeting invitation, deleting it from google calendar and re-creating without the invitation. Probably could have just edited it too.

It was pretty impossible to find the offending calendar entry by searching, but fortunately I noticed my calendars were not syncing within a few days of accepting a webex meeting, so it it was easy to find the entry.

It would be nice to see a fix that works since this is the only thing that derails my sync.

@WhyNotHugo
Copy link
Member

WhyNotHugo commented Oct 5, 2021 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants