fix proper support for expandable calendar search

[tobixen]

ref tobixen/calendar-cli#52

• one user reported that he was unable to fetch events where recurrances are within the time range. Make sure this works, and make options to explicitly request this from the server
• also, currently the server does not expand recurring events into multiple objects in a date search. We need this feature for good agendas in calendar-cli.

[tobixen]

7.8.3 is an example, and the formal definition is in section 9.6.5.

If I understand it right, the expand attribute should cause the server to expand a recurring vobject (i.e. "meeting every Wednesday at 12:00") into single vobjects ("meeting this Wednesday, meeting next Wednesday, ....). Apparently the expand element is passed in the query (ref #40), so it's a bit weird that it doesn't work with calendar-cli.

Ref 7.8.1 and 9.9 recurring vobjects should be returned as long as any recurrences overlaps the query time. Hence the calendar-cli bug reported that the agenda did not contain recurrences seems to be a server-side error.

Ref 7.6, 9.6.6 and 7.8.2 a calendar query can specify the limit-recurrence-set parameter to override the behaviour above - but limit-recurrence-set is not used in the caldav library presently.

Things to do:

• Refactor the calendar query logic a bit to make it more flexible wrg of filtering and attributes sent (I think one of the major purposes of the library is to abstract away all XML interaction, so users of public methods should neither have to deal with XML nor any of the classes under caldav/elements ... at the other hand, I would also like to offer the full flexibility of the query language ... this is a bit tricky).
• I think I will procrastinate support for the limit-recurrence-set attribute, but support to choose between expanded and non-expanded reports should be explicitly supported
• We need test code that tests both expanded reports and non-expanded reports.
• Ref Ensure compatibility with Xandikos #40, it may be needed with a test config flag to disable expanded searches from servers that don't support expanded searches

[tobixen]

I think the work done suffices for release 0.6.0; the date_search now have a parameter "expand". Default to true'ish for backward-compatibility.

[tobixen]

Apparently I need to revisit this one.

There is some test code, but it's not very "hard". I just wrote a comment that different servers behave differently, so the test code accepts a single non-expanded event as long as the RRULE given - but that's not really "expanded", is it?

Section-references above is (of course) referring to RFC4791

Example query: https://tools.ietf.org/html/rfc4791#section-7.8.3

When using the "expand"-flag to the date_search, the query created by the caldav library looks quite the same as the one given, except the namespace attribute looks a bit misplaced. Could that be relevant? I should check that up.

[tobixen]

Now the XML sent from the caldav library looks quite the same as the example in the RFC. Here is the body (manually added some whitespace for clarity):

<?xml version='1.0' encoding='utf-8'?>
<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav" xmlns:I="http://apple.com/ns/ical/">
  <D:prop>
    <C:calendar-data>
      <C:expand start="20081101T160000Z" end="20091103T160000Z"/>
    </C:calendar-data>
  </D:prop>
  <C:filter>
    <C:comp-filter name="VCALENDAR">
        <C:comp-filter name="VEVENT">
          <C:time-range start="20081101T160000Z" end="20091103T160000Z"/>
        </C:comp-filter>
      </C:comp-filter>
   </C:filter>
</C:calendar-query>

Looks quite the same as the example to me.

According to the example in the RFC, the result should be one vcalendar item with two events, both having a recurrence-id, none of them having rrule set. From radicale I get this (no matter if expand is set or not):

<?xml version='1.0' encoding='utf-8'?>
<multistatus xmlns="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav"><response><href>/user1/pythoncaldav-test/19970901T130000Z-123403%40example.com.ics</href><propstat><prop><C:calendar-data>BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Example Corp.//CalDAV Client//EN
BEGIN:VEVENT
UID:19970901T130000Z-123403@example.com
DTSTART;VALUE=DATE:19971102
CATEGORIES:ANNIVERSARY,PERSONAL,SPECIAL OCCASION
CLASS:CONFIDENTIAL
DTSTAMP:19970901T130000Z
RRULE:FREQ=YEARLY
SUMMARY:Our Blissful Anniversary
TRANSP:TRANSPARENT
END:VEVENT
END:VCALENDAR

[tobixen]

At least, for radicale it seems to be a reported issue on this: Kozea/Radicale#662

[tobixen]

The tests have been sharpened, and run towards calendar servers that properly supports the expand-attribute.

As per the assertions in the test itself, the caldav library returns the data it gets from the server - and that's one vCalendar object with multiple vEvents within. I think this is the correct behavior, as the caldav library mostly should be focusing on the caldav communication, and not so much on the icalendar logic. At the other hand it's also a bit confusing as the "Event"-class in the caldav library will always be wrapping a vcalendar object, and in this case the vcalendar object contains multiple events.

I guess it would be more intuitive to get a list of events rather than one event. I've even expected that myself in the calendar-cli utility. This explains #86

I will close this issue, and follow up further in #86