vdirsyncer fails to sync due to double URL encoding

[sboehringer]

• Your vdirsyncer version
  vdirsyncer, version 0.11.3
• If applicable, which server software (and which version) you're using
  DavMail 4.7.2-2427
• Your Python version
  python --version: Python 2.7.12
  python3 --version: 4.7.2-2427
• Your operating system
  OpenSuse 13.2
• Your config file

[general]
# A folder where vdirsyncer can store some metadata about each pair.
status_path = ~/.local/share/vdirsyncer/status/

# CALDAV
[pair my_calendar]
a = my_calendar_local
b = my_calendar_remote
collections = ["from a", "from b"]

# Calendars also have a color property
metadata = ["displayname", "color"]

[storage my_calendar_local]
type = filesystem
path = ~/.local/share/khal/calendars/private
fileext = .ics

[storage my_calendar_remote]
type = caldav
url = http://localhost:10801/users/s.boehringer@lumc.nl/calendar
username = ****
password = *****

• Use vdirsyncer -vdebug for debug output.

...
error: Unknown error occured for my_calendar/calendar: /users/S.Boehringer@lumc.nl/calendar/AAMkADFhMDg1YzYwLTgwMDItNDJiYi1hNmY2LTgxZDI1MDcwNmIzMgBGAAAAAAAutjjQvZGzQZDR7unScTwfBwA4R1qAXwlGQ7K94p6FsS4bAAAA66iHAAD2R2asQXn-SImONQz5BBbsAABHFlNrAAA%3D.EML
error: Use `-vdebug` to see the full traceback.
debug:   File "/usr/lib/python3.4/site-packages/vdirsyncer/cli/tasks.py", line 66, in sync_collection
debug:     force_delete=force_delete
debug:   File "/usr/lib/python3.4/site-packages/vdirsyncer/sync.py", line 216, in sync
debug:     b_info.prepare_idents()
debug:   File "/usr/lib/python3.4/site-packages/vdirsyncer/sync.py", line 124, in prepare_idents
debug:     if prefetch else ()):
debug:   File "/usr/lib/python3.4/site-packages/vdirsyncer/storage/dav.py", line 452, in get_multi
debug:     raise exceptions.NotFoundError(href)
error: 1 out of 2 tasks failed.

The failing URL is partly encoded (trailing %3D). Replacing this with '=' in cadaver prints a proper caldav event.

[untitaker]

Further above there should be a "Normalized URL from" or "Already normalized". What does it look like?

[sboehringer]

This would be the line:

debug: Unsafe character: '%3D'
debug: Already normalized: '/users/S.Boehringer@lumc.nl/calendar/AAMkADFhMDg1YzYwLTgwMDItNDJiYi1hNmY2LTgxZDI1MDcwNmIzMgBGAAAAAAAutjjQvZGzQZDR7unScTwfBwA4R1qAXwlGQ7K94p6FsS4bAAAA66iHAAD2R2asQXn-SImONQz5BBbsAAAAAbI1AAA%3D.EML'

[untitaker]

This should be fixed in master. If you want, you could try:

virtualenv -p python3 fooenv
. fooenv/bin/activate
pip install git+https://github.com/untitaker/vdirsyncer

Then you have a working development version on ./fooenv/bin/vdirsyncer.

[untitaker]

0.12.1 is out, please reopen if this bug persists.

[sboehringer]

Unfortunately, the error persists in version 0.12.1 in identical form.

DavMail is the CalDav server which is a Microsoft Exchange adaptor. As cadaver has a similar problem with showing the events with html-quoted characters, I am wondering whether is is really a DavMail bug. However, I would have a good idea as to how to proceed at the moment.

[untitaker]

Is the forn absolutely identical? In particular the debugging lines about URL normalization (as asked above) interest me.

[sboehringer]

debug: Already normalized: '/users/S.Boehringer@lumc.nl/calendar/AAMkADFhMDg1YzYwLTgwMDItNDJiYi1hNmY2LTgxZDI1MDcwNmIzMgBGAAAAAAAutjjQvZGzQZDR7unScTwfBwA4R1qAXwlGQ7K94p6FsS4bAAAA66iHAAD2R2asQXn-SImONQz5BBbsAABHFlNRAAA%3D.EML'
warning: Skipping identical href: '/users/S.Boehringer@lumc.nl/calendar/AAMkADFhMDg1YzYwLTgwMDItNDJiYi1hNmY2LTgxZDI1MDcwNmIzMgBGAAAAAAAutjjQvZGzQZDR7unScTwfBwA4R1qAXwlGQ7K94p6FsS4bAAAA66iHAAD2R2asQXn-SImONQz5BBbsAABHFlNRAAA%3D.EML'

The already normalized line occurs two times in the debug output. The "@" character is unquoted, the "=" is quoted.
Interestingly, the debug logs of DavMail do not show an access attempt to the event above. No errors are shown at all. This is different from cadaver that triggers errors in the DavMail server. Finally, Thunderbird works fine. Taken together, I would rather tend to assume an independent bug in vdirsync.

[sboehringer]

Here is the davmail log from DavMail as given for a Thunderbird transaction (it is not the same event as I could not identify it). The EML-resources seem to be recurrent events. Thunderbird has quoted the "@" also.

Date:           Thu Aug 25 17:04:21 CEST 2016 (1472137461367)
Thread:     CaldavConnection-55185
Message #:  2
Level:      DEBUG
NDC:            
Category:   davmail
Message:        < PUT /users/s.boehringer%40lumc.nl/calendar/AAMkADFhMDg1YzYwLTgwMDItNDJiYi1hNmY2LTgxZDI1MDcwNmIzMgBGAAAAAAAutjjQvZGzQZDR7unScTwfBwA4R1qAXwlGQ7K94p6FsS4bAAAA66iHAAD2R2asQXn-SImONQz5BBbsAAAkN0HIAAA%3D.EML HTTP/1.1
Location:   davmail.ui.tray.DavGatewayTray.displayMessage(DavGatewayTray.java:96)
Thrown:

[untitaker]

Could you try adding return x after this line in your installation?