vdirsyncer fails on a DAV server with authentication (radicale 0.9-1) #140

legrostdg · 2014-11-28T16:12:49Z

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1)

$ .local/bin/vdirsyncer --version
vdirsyncer, version 0.3.1
$ .local/bin/vdirsyncer --verbosity debug sync
debug: Using 0 maximal workers.

Traceback (most recent call last):
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/cli.py", line 164, in storage_instance_from_config
    return cls(**config)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 488, in __init__
    super(CaldavStorage, self).__init__(**kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 264, in __init__
    dav_header=self.dav_header)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 172, in __init__
    'auth': prepare_auth(auth, username, password),
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/http.py", line 27, in prepare_auth
    import requests_toolbelt
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/__init__.py", line 21, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/__init__.py", line 16, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/encoder.py", line 12, in <module>
ImportError: cannot import name iter_field_objects

Traceback (most recent call last):
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/cli.py", line 164, in storage_instance_from_config
    return cls(**config)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 488, in __init__
    super(CaldavStorage, self).__init__(**kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 264, in __init__
    dav_header=self.dav_header)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 172, in __init__
    'auth': prepare_auth(auth, username, password),
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/http.py", line 27, in prepare_auth
    import requests_toolbelt
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/__init__.py", line 21, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/__init__.py", line 16, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/encoder.py", line 12, in <module>
ImportError: cannot import name iter_field_objects
critical: Failed to initialize my_calendars_remote.
critical: Failed to initialize my_calendars_remote.

cadaver works:

$ cadaver https://dav.example.com
Authentication required for Radicale - Password Required on server `dav.example.com':
Username: user
Password: 
dav:/> ls
Listing collection `/': succeeded.
Coll:   user                                  0  Jan  1  1970

The text was updated successfully, but these errors were encountered:

untitaker · 2014-11-28T16:17:31Z

How did you install vdirsyncer?

legrostdg · 2014-11-28T16:22:07Z

pip install --user --force-reinstall git+https://github.com/untitaker/vdirsyncer.git

untitaker · 2014-11-28T16:54:24Z

How did you install vdirsyncer before you reported #138?

legrostdg · 2014-11-28T17:07:11Z

By cloning khal git repo, and running:

 python setup.py --install --prefix $HOME

untitaker · 2014-11-28T17:12:23Z

Alright, please try:

pip uninstall requests_toolbelt
pip install --user requests_toolbelt

legrostdg · 2014-11-28T17:15:41Z

no change :(

untitaker · 2014-11-28T17:19:53Z

Could you try removing the build directory? I wonder why that is showing up in the traceback.

legrostdg · 2014-11-28T17:34:59Z

OK. There is a problem in requests_toolbelt or in my version of requests (2.4.3-5).

$ .local/bin/vdirsyncer sync
...
 File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/multipart/encoder.py", line 12, in <module>
    from requests.packages.urllib3.filepost import iter_field_objects
ImportError: cannot import name iter_field_objects

$ python
Python 2.7.8 (default, Oct 18 2014, 12:50:18) 
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from requests.packages.urllib3.filepost import iter_field_objects
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name iter_field_objects

untitaker · 2014-11-28T17:37:37Z

Are you on Ubuntu or Debian? If yes, you'll have to take a look at https://vdirsyncer.readthedocs.org/en/latest/problems.html#error-messages
Try: pip install --user --upgrade requests requests_toolbelt

legrostdg · 2014-11-28T17:44:21Z

Debian (sid, so python-requests is 2.4.3-5). I don't have the issues listed there https://vdirsyncer.readthedocs.org/en/latest/problems.html#error-messages
$ pip install --user --upgrade requests requests_toolbelt
Requirement already up-to-date: requests in /usr/lib/python2.7/dist-packages
Requirement already up-to-date: requests-toolbelt in ./.local/lib/python2.7/site-packages
Cleaning up...

$ .local/bin/vdirsyncer --verbosity debug sync
debug: Using 0 maximal workers.

Traceback (most recent call last):
File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/cli.py", line 164, in storage_instance_from_config
return cls(**config)
File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 488, in init
super(CaldavStorage, self).init(**kwargs)
File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 264, in init
dav_header=self.dav_header)
File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 172, in init
'auth': prepare_auth(auth, username, password),
File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/http.py", line 27, in prepare_auth
import requests_toolbelt
File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/init.py", line 21, in
from .multipart import (
File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/multipart/init.py", line 16, in
from .encoder import MultipartEncoder, MultipartEncoderMonitor
File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/multipart/encoder.py", line 12, in
from requests.packages.urllib3.filepost import iter_field_objects
ImportError: cannot import name iter_field_objects
critical: Failed to initialize my_calendars_remote.

legrostdg · 2014-11-28T17:49:27Z

But I don't understand why I can't import iter_field_objects from requests.packages.urllib3.filepost. The funtion is defined in /usr/lib/python2.7/dist-packages/requests/packages/urllib3/filepost.py...

untitaker · 2014-11-28T17:49:28Z

I honestly think you do have that problem, but with a different error message... can you do:

$ python
Python 3.4.2 (default, Oct  8 2014, 13:44:52)
[GCC 4.9.1 20140903 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests.packages.urllib3.filepost
>>>

untitaker · 2014-11-28T17:50:41Z

Maybe you have a different version installed as user... take a look at /home/user/.local/lib/python2.7/, and try pip uninstall requests.

legrostdg · 2014-11-28T17:54:05Z

The system version of requests is the only one I have installed.

That's weird:

$ python
Python 2.7.8 (default, Oct 18 2014, 12:50:18)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests.packages.urllib3.filepost
>>> requests.packages.urllib3.filepost.iter_field_objects
<function iter_field_objects at 0x7f9759d4f8c0>
>>>> from requests.packages.urllib3.filepost import iter_field_objects
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name iter_field_objects

legrostdg · 2014-11-28T18:04:22Z

OK, I found the bug: on Debian, /usr/lib/python2.7/dist-packages/requests/packages/urllib3 is a symlink to ../../urllib3, which (I don't know why) don't behave the same as the original urllib3...

$ ls -l /usr/lib/python2.7/dist-packages/requests/packages/urllib3
lrwxrwxrwx. 1 root root 13 Oct 21 10:25 /usr/lib/python2.7/dist-packages/requests/packages/urllib3 -> ../../urllib3

python interpreter:

>>> from urllib3.filepost import iter_field_objects
>>> iter_field_objects
<function iter_field_objects at 0x7f82cfbf0848>
>>> from requests.packages.urllib3.filepost import iter_field_objects
>>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name iter_field_objects

untitaker · 2014-11-28T18:06:42Z

Eww... thanks for finding this. You still might want to take the approach described in the linked doc page and create a virtualenv with all your dependencies.

legrostdg · 2014-11-28T18:13:36Z

I just reported a bug on Debian:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771349

untitaker · 2014-11-28T18:15:00Z

Awesome, thanks!

untitaker · 2014-11-28T18:15:27Z

I suppose the symlinks were a reaction to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753578, but I am not even sure if the Debian maintainers are at fault. A symlink should work fine.

mrjovanovic · 2015-01-04T21:43:39Z

I just wanted to confirm that I have the same issue, @legrostdg did you come up with a workaround? I'm on debian jessie. I haven't been able to install vdirsyncer in a virtualenv using the instructions provided here, so that doesn't work for me unfortunately. :(

python requests is version 2.4.3-4 per your recommendation. :(

untitaker · 2015-01-05T11:17:37Z

@mrjovanovic Which problems did you run into with the virtualenv?

untitaker · 2015-01-05T11:37:55Z

Also, I just realized you can bypass this problem by setting the auth parameter of CalDAV storage etc to either basic or digest.

mrjovanovic · 2015-01-05T15:09:44Z

@untitaker It was actually just an issue of missing dependencies. I haven't been in dependency hell in a decade, so this was interesting. A bunch of lib*-dev are required on a default jessie install. Now it works fine with the virtualenv, although perhaps it should be added as a note in the guide?

I tried your suggestion, and I'm getting:

Detected change in config file, discovering collections for bob_calendar
Discovering collections for pair bob_calendar
Syncing bob_calendar
Unhandled exception occured while syncing bob_calendar.
Traceback (most recent call last):
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/cli/tasks.py", line 77, in sync_collection
force_delete=force_delete
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/sync.py", line 130, in sync
b_idents = _prepare_idents(storage_b, storage_a, b_href_to_status)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/sync.py", line 67, in _prepare_idents
for href, etag in storage.list():
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 654, in list
headers=headers)
 File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 260, in request
return utils.request(method, url, session=self._session, **more)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/utils/__init__.py", line 220, in request
r = func(method, url, **kwargs)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/requests/sessions.py", line 461, in request
resp = self.send(prep, **send_kwargs)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/requests/adapters.py",  line 431, in send
raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

The SSL cert is self-signed, but installed as a system-wide cert. I also point to it in the config as verify = /etc/ssl/certs/certificatename.crt Maybe I should open a new issue with this?

untitaker · 2015-01-05T17:55:52Z

@mrjovanovic See #98. Maybe you have more luck with verify_fingerprint?

legrostdg · 2015-02-04T12:48:54Z

@mrjovanovic I'm still waiting for a fix in Debian https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771349 :). The fix in requests is quite new https://github.com/kennethreitz/requests/pull/2375 So for now, I'm waiting for a new release of requests.

legrostdg closed this as completed Nov 28, 2014

untitaker added a commit that referenced this issue Nov 28, 2014

Update debian problems in response to #140

af3b081

untitaker added a commit that referenced this issue Jan 5, 2015

Add new workaround for #140 (debian bs)

d4e3b99

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1) #140

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1) #140

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

untitaker commented Nov 28, 2014

mrjovanovic commented Jan 4, 2015

untitaker commented Jan 5, 2015

untitaker commented Jan 5, 2015

mrjovanovic commented Jan 5, 2015

untitaker commented Jan 5, 2015

legrostdg commented Feb 4, 2015

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1) #140

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1) #140

Comments

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

legrostdg commented Nov 28, 2014

untitaker commented Nov 28, 2014

untitaker commented Nov 28, 2014

mrjovanovic commented Jan 4, 2015

untitaker commented Jan 5, 2015

untitaker commented Jan 5, 2015

mrjovanovic commented Jan 5, 2015

untitaker commented Jan 5, 2015

legrostdg commented Feb 4, 2015