Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1) #140

Closed
legrostdg opened this issue Nov 28, 2014 · 25 comments
Closed

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1) #140

legrostdg opened this issue Nov 28, 2014 · 25 comments

Comments

@legrostdg
Copy link

@legrostdg legrostdg commented Nov 28, 2014

vdirsyncer fails on a DAV server with authentication (radicale 0.9-1)

$ .local/bin/vdirsyncer --version
vdirsyncer, version 0.3.1
$ .local/bin/vdirsyncer --verbosity debug sync
debug: Using 0 maximal workers.

Traceback (most recent call last):
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/cli.py", line 164, in storage_instance_from_config
    return cls(**config)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 488, in __init__
    super(CaldavStorage, self).__init__(**kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 264, in __init__
    dav_header=self.dav_header)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 172, in __init__
    'auth': prepare_auth(auth, username, password),
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/http.py", line 27, in prepare_auth
    import requests_toolbelt
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/__init__.py", line 21, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/__init__.py", line 16, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/encoder.py", line 12, in <module>
ImportError: cannot import name iter_field_objects

Traceback (most recent call last):
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/cli.py", line 164, in storage_instance_from_config
    return cls(**config)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 488, in __init__
    super(CaldavStorage, self).__init__(**kwargs)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 264, in __init__
    dav_header=self.dav_header)
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 172, in __init__
    'auth': prepare_auth(auth, username, password),
  File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/http.py", line 27, in prepare_auth
    import requests_toolbelt
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/__init__.py", line 21, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/__init__.py", line 16, in <module>
  File "build/bdist.linux-x86_64/egg/requests_toolbelt/multipart/encoder.py", line 12, in <module>
ImportError: cannot import name iter_field_objects
critical: Failed to initialize my_calendars_remote.
critical: Failed to initialize my_calendars_remote.

cadaver works:

$ cadaver https://dav.example.com
Authentication required for Radicale - Password Required on server `dav.example.com':
Username: user
Password: 
dav:/> ls
Listing collection `/': succeeded.
Coll:   user                                  0  Jan  1  1970
@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

How did you install vdirsyncer?

@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

pip install --user --force-reinstall git+https://github.com/untitaker/vdirsyncer.git
@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

How did you install vdirsyncer before you reported #138?

@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

By cloning khal git repo, and running:

 python setup.py --install --prefix $HOME
@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

Alright, please try:

pip uninstall requests_toolbelt
pip install --user requests_toolbelt
@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

no change :(

@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

Could you try removing the build directory? I wonder why that is showing up in the traceback.

@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

OK. There is a problem in requests_toolbelt or in my version of requests (2.4.3-5).

$ .local/bin/vdirsyncer sync
...
 File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/multipart/encoder.py", line 12, in <module>
    from requests.packages.urllib3.filepost import iter_field_objects
ImportError: cannot import name iter_field_objects

$ python
Python 2.7.8 (default, Oct 18 2014, 12:50:18) 
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from requests.packages.urllib3.filepost import iter_field_objects
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name iter_field_objects
@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

  1. Are you on Ubuntu or Debian? If yes, you'll have to take a look at https://vdirsyncer.readthedocs.org/en/latest/problems.html#error-messages
  2. Try: pip install --user --upgrade requests requests_toolbelt
@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

  1. Debian (sid, so python-requests is 2.4.3-5). I don't have the issues listed there https://vdirsyncer.readthedocs.org/en/latest/problems.html#error-messages

  2. $ pip install --user --upgrade requests requests_toolbelt
    Requirement already up-to-date: requests in /usr/lib/python2.7/dist-packages
    Requirement already up-to-date: requests-toolbelt in ./.local/lib/python2.7/site-packages
    Cleaning up...

    $ .local/bin/vdirsyncer --verbosity debug sync
    debug: Using 0 maximal workers.

    Traceback (most recent call last):
    File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/cli.py", line 164, in storage_instance_from_config
    return cls(**config)
    File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 488, in init
    super(CaldavStorage, self).init(**kwargs)
    File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 264, in init
    dav_header=self.dav_header)
    File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 172, in init
    'auth': prepare_auth(auth, username, password),
    File "/home/user/.local/lib/python2.7/site-packages/vdirsyncer/storage/http.py", line 27, in prepare_auth
    import requests_toolbelt
    File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/init.py", line 21, in
    from .multipart import (
    File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/multipart/init.py", line 16, in
    from .encoder import MultipartEncoder, MultipartEncoderMonitor
    File "/home/user/.local/lib/python2.7/site-packages/requests_toolbelt/multipart/encoder.py", line 12, in
    from requests.packages.urllib3.filepost import iter_field_objects
    ImportError: cannot import name iter_field_objects
    critical: Failed to initialize my_calendars_remote.

@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

But I don't understand why I can't import iter_field_objects from requests.packages.urllib3.filepost. The funtion is defined in /usr/lib/python2.7/dist-packages/requests/packages/urllib3/filepost.py...

@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

I honestly think you do have that problem, but with a different error message... can you do:

$ python
Python 3.4.2 (default, Oct  8 2014, 13:44:52)
[GCC 4.9.1 20140903 (prerelease)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests.packages.urllib3.filepost
>>>
@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

Maybe you have a different version installed as user... take a look at /home/user/.local/lib/python2.7/, and try pip uninstall requests.

@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

The system version of requests is the only one I have installed.

That's weird:

$ python
Python 2.7.8 (default, Oct 18 2014, 12:50:18)
[GCC 4.9.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests.packages.urllib3.filepost
>>> requests.packages.urllib3.filepost.iter_field_objects
<function iter_field_objects at 0x7f9759d4f8c0>
>>>> from requests.packages.urllib3.filepost import iter_field_objects
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name iter_field_objects
@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

OK, I found the bug: on Debian, /usr/lib/python2.7/dist-packages/requests/packages/urllib3 is a symlink to ../../urllib3, which (I don't know why) don't behave the same as the original urllib3...

$ ls -l /usr/lib/python2.7/dist-packages/requests/packages/urllib3
lrwxrwxrwx. 1 root root 13 Oct 21 10:25 /usr/lib/python2.7/dist-packages/requests/packages/urllib3 -> ../../urllib3

python interpreter:

>>> from urllib3.filepost import iter_field_objects
>>> iter_field_objects
<function iter_field_objects at 0x7f82cfbf0848>
>>> from requests.packages.urllib3.filepost import iter_field_objects
>>> Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name iter_field_objects
@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

Eww... thanks for finding this. You still might want to take the approach described in the linked doc page and create a virtualenv with all your dependencies.

@legrostdg
Copy link
Author

@legrostdg legrostdg commented Nov 28, 2014

@legrostdg legrostdg closed this Nov 28, 2014
@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

Awesome, thanks!

@untitaker
Copy link
Member

@untitaker untitaker commented Nov 28, 2014

I suppose the symlinks were a reaction to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=753578, but I am not even sure if the Debian maintainers are at fault. A symlink should work fine.

untitaker added a commit that referenced this issue Nov 28, 2014
@mrjovanovic
Copy link

@mrjovanovic mrjovanovic commented Jan 4, 2015

I just wanted to confirm that I have the same issue, @legrostdg did you come up with a workaround? I'm on debian jessie. I haven't been able to install vdirsyncer in a virtualenv using the instructions provided here, so that doesn't work for me unfortunately. :(

python requests is version 2.4.3-4 per your recommendation. :(

@untitaker
Copy link
Member

@untitaker untitaker commented Jan 5, 2015

@mrjovanovic Which problems did you run into with the virtualenv?

@untitaker
Copy link
Member

@untitaker untitaker commented Jan 5, 2015

Also, I just realized you can bypass this problem by setting the auth parameter of CalDAV storage etc to either basic or digest.

untitaker added a commit that referenced this issue Jan 5, 2015
@mrjovanovic
Copy link

@mrjovanovic mrjovanovic commented Jan 5, 2015

@untitaker It was actually just an issue of missing dependencies. I haven't been in dependency hell in a decade, so this was interesting. A bunch of lib*-dev are required on a default jessie install. Now it works fine with the virtualenv, although perhaps it should be added as a note in the guide?

I tried your suggestion, and I'm getting:

Detected change in config file, discovering collections for bob_calendar
Discovering collections for pair bob_calendar
Syncing bob_calendar
Unhandled exception occured while syncing bob_calendar.
Traceback (most recent call last):
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/cli/tasks.py", line 77, in sync_collection
force_delete=force_delete
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/sync.py", line 130, in sync
b_idents = _prepare_idents(storage_b, storage_a, b_href_to_status)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/sync.py", line 67, in _prepare_idents
for href, etag in storage.list():
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 654, in list
headers=headers)
 File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/storage/dav.py", line 260, in request
return utils.request(method, url, session=self._session, **more)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/vdirsyncer/utils/__init__.py", line 220, in request
r = func(method, url, **kwargs)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/requests/sessions.py", line 461, in request
resp = self.send(prep, **send_kwargs)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
  File "/home/username/src/vdirsyncer_env/local/lib/python2.7/site-packages/requests/adapters.py",  line 431, in send
raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)

The SSL cert is self-signed, but installed as a system-wide cert. I also point to it in the config as verify = /etc/ssl/certs/certificatename.crt Maybe I should open a new issue with this?

@untitaker
Copy link
Member

@untitaker untitaker commented Jan 5, 2015

@mrjovanovic See #98. Maybe you have more luck with verify_fingerprint?

@legrostdg
Copy link
Author

@legrostdg legrostdg commented Feb 4, 2015

@mrjovanovic I'm still waiting for a fix in Debian https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771349 :). The fix in requests is quite new psf/requests#2375 So for now, I'm waiting for a new release of requests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.