Failing to fetch events from personal Google/Google Apps Calendars

[jjmcdn]

I'm not convinced this is an issue at all and not just pilot error, but this is what I'm seeing.

I've configured vdirsyncer thus:

[general]
status_path = /home/jjm/.vdirsyncer/status

[pair gmail_calendar]
a = gmail_local
b = gmail_remote
collections = [ "from b" ]

[storage gmail_local]
type = filesystem
path = /home/jjm/.calendars/gmail/
fileext = .ics

[storage gmail_remote]
type = caldav
url = https://www.google.com/calendar/dav/
username = joe.macdonald@gmail.com
password = "*************************"
read_only = true

My password is enclosed in quotes because it contains a # mark. vdirsyncer discover seems to operate successfully and when I run vdirsyncer sync I see a bunch of stuff gets populated into /home/jjm/.calendars/gmail/events but it turns out that nothing in there is actually out of my personal calendar, they're just the public holidays calendar (eg. New Year's Day, Martin Luther King Day, etc.).

Looking at the instructions here: https://support.google.com/calendar/answer/99358 I thought perhaps I should change the url to be url = https://www.google.com/calendar/dav/joe.macdonald@gmail.com/user/ but that results in an error:

Discovering collections for pair gmail_calendar

Traceback (most recent call last):
  File "build/bdist.linux-x86_64/egg/vdirsyncer/cli/utils.py", line 218, in _discover_from_config
    discovered = list(cls.discover(**config))
  File "build/bdist.linux-x86_64/egg/vdirsyncer/storage/dav.py", line 303, in discover
    for c in d.discover():
  File "build/bdist.linux-x86_64/egg/vdirsyncer/storage/dav.py", line 166, in discover
    for x in self.find_collections():
  File "build/bdist.linux-x86_64/egg/vdirsyncer/storage/dav.py", line 144, in find_collections
    url = self.find_home()
  File "build/bdist.linux-x86_64/egg/vdirsyncer/storage/dav.py", line 128, in find_home
    url = self.find_principal()
  File "build/bdist.linux-x86_64/egg/vdirsyncer/storage/dav.py", line 123, in find_principal
    raise InvalidXMLResponse()
InvalidXMLResponse

This is all from a vdirsyncer I installed from git:

vdirsyncer master$ git show HEAD
commit 4e8ae1e15f6f8bbe08fa6dc2386057efbc2abc26
Merge: 6ab6d1d 4adebe6
Author: Markus Unterwaditzer <markus@unterwaditzer.net>
Date:   Thu May 7 22:32:59 2015 +0200

    Merge pull request #200 from untitaker/handle-empty-items

    Skip empty items

Current as of today and python2.7. I had previously installed directly via pip but that used python3.4 on my system (an up-to-date Debian 8 / Jessie) and failed in the same ways, so I tried using an older version of python also installed on my system. I did that with this command:

vdirsyncer master$ sudo python2.7 ./setup.py install

which ensures I'm using python2.7:

vdirsyncer master$  head -2 `which vdirsyncer`
#!/usr/bin/python2.7
# EASY-INSTALL-ENTRY-SCRIPT: 'vdirsyncer==0.4.4','console_scripts','vdirsyncer'

Does any of this make any sense? I looked for a similar issue in the list of open issues and didn't see one, but it's entirely possible I've just missed it, since I'm not really sure what I want to be searching for.

Oh, maybe it is worth mentioning, the first time I ran vdirsyncer discover it displayed this:

Discovering collections for pair gmail_calendar
error: No collection events found for storage gmail_local.
Should vdirsyncer attempt to create it? [y/N]: y
Saved for gmail_calendar: collections = ["events"]

Which I thought made since, since I had only create the top level .calendars/ directory, nothing underneath it.

I've placed the debug output from my latest vdirsyncer sync run in a gist at https://gist.github.com/joeythesaint/6156d85a4e9d881020dd

[untitaker]

Not sure how my previous comment didn't get posted completely.

• As you can see, Google support is untested -- IIRC the settings in the docs are taken from the official support pages
• Vdirsyncer authenticates lazily -- could you try setting auth = basic?
• Do you encounter any problems with other DAV clients?

[jjmcdn]

Thanks for the quick response. I know the google support is untested and I really appreciate your taking time to look at this anyway.

I set auth = basic in the gmail_remote section of the config above, so now it looks like:

[storage gmail_remote]
type = caldav
auth = basic
url = https://www.google.com/calendar/dav/
username = joe.macdonald@gmail.com
password = "*************************"
read_only = true

but it didn't seem to make any difference. This is the first DAV client I've tried using, though I did try out gcalcli and it is showing the events.

I was doing a bit more searching around myself and I found this: https://developers.google.com/google-apps/calendar/caldav/v2/guide specifically:

The old endpoint https://www.google.com/calendar/dav is deprecated and no longer supported; use it at your own risk. We recommend you transition to the new endpoint format described above.

which is the URL described in the page linked from the docs, so it may be just that the old API is failing me and Google expects everyone to migrate to the OAuth API. :-/

[untitaker]

It is quite possible that Google is intentionally limiting the DAV API this way to force users and devs to OAuth (see also #8, personally I don't really care about Google support).

What you can do at this point is using vdirsyncer's HTTP storage to access the "Private Addresses" in iCal format:

[storage gmail_calendar_one_remote]
type = http
url = ...  # "Private URL" to calendar one

[storage gmail_calendar_two_remote]
type = http
url = ...  # "Private URL" to calendar two

[pair gmail_calendar_one]
a = gmail_calendar_one_remote
b = gmail_calendar_one_local

[pair gmail_calendar_two]
a = gmail_calendar_two_remote
b = gmail_calendar_two_local

[storage gmail_calendar_one_local]
type = filesystem
path = ~/.calendars/one

[storage gmail_calendar_two_local]
type = filesystem
path = ~/.calendars/two

This is insanely repetitive, but it probably works.

[untitaker]

FWIW I think you'll get a better user experience if you use e.g. gcalcli and other software that is specifically built around Google's infrastructure.

[jjmcdn]

I tried the ical/http option and that works for reading, which is all I need right now anyway. Thanks!

The main reason I'd like to have vdirsyncer working is because it also works really well right now for me with my work (exchange) calendar and I'm willing to trade some functionality on the local update / push side for the convenience of using a single tool.

Again, thanks for the assistance.

[untitaker]

You're welcome!