EXDATE support not working properly

[wichmannpas]

The EXDATE support (from the most recent PyPi version) does not seem to be working properly.

I have for example an event like the following where at least some of the EXDATE values are not honoured:

BEGIN:VCALENDAR
PRODID:+//IDN bitfire.at//DAVx5/2.6.1.1-gplay ical4j/2.2.6
VERSION:2.0
BEGIN:VTIMEZONE
TZID:Europe/Berlin
TZURL:http://tzurl.org/zoneinfo/Europe/Berlin
X-LIC-LOCATION:Europe/Berlin
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19810329T020000
RRULE:FREQ=YEARLY;BYMONTH=3;BYDAY=-1SU
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19961027T030000
RRULE:FREQ=YEARLY;BYMONTH=10;BYDAY=-1SU
END:STANDARD
BEGIN:STANDARD
TZOFFSETFROM:+005328
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:18930401T000000
RDATE:18930401T000000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19160430T230000
RDATE:19160430T230000
RDATE:19170416T020000
RDATE:19180415T020000
RDATE:19400401T020000
RDATE:19430329T020000
RDATE:19440403T020000
RDATE:19450402T020000
RDATE:19460414T020000
RDATE:19470406T030000
RDATE:19480418T020000
RDATE:19490410T020000
RDATE:19800406T020000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19161001T010000
RDATE:19161001T010000
RDATE:19170917T030000
RDATE:19180916T030000
RDATE:19421102T030000
RDATE:19431004T030000
RDATE:19441002T030000
RDATE:19451118T030000
RDATE:19461007T030000
RDATE:19471005T030000
RDATE:19481003T030000
RDATE:19491002T030000
RDATE:19800928T030000
RDATE:19810927T030000
RDATE:19820926T030000
RDATE:19830925T030000
RDATE:19840930T030000
RDATE:19850929T030000
RDATE:19860928T030000
RDATE:19870927T030000
RDATE:19880925T030000
RDATE:19890924T030000
RDATE:19900930T030000
RDATE:19910929T030000
RDATE:19920927T030000
RDATE:19930926T030000
RDATE:19940925T030000
RDATE:19950924T030000
END:STANDARD
BEGIN:DAYLIGHT
TZOFFSETFROM:+0200
TZOFFSETTO:+0300
TZNAME:CEMT
DTSTART:19450524T010000
RDATE:19450524T010000
RDATE:19470511T020000
END:DAYLIGHT
BEGIN:DAYLIGHT
TZOFFSETFROM:+0300
TZOFFSETTO:+0200
TZNAME:CEST
DTSTART:19450924T030000
RDATE:19450924T030000
RDATE:19470629T030000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:+0100
TZOFFSETTO:+0100
TZNAME:CET
DTSTART:19460101T000000
RDATE:19460101T000000
RDATE:19800101T000000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTAMP:20191219T182547Z
UID:f0f31ddb-6918-46af-a5a1-0a7254fbce71
SEQUENCE:11
SUMMARY:Test
LOCATION:Example
DTSTART;TZID=Europe/Berlin:20191015T161500
DURATION:PT1H30M
RRULE:FREQ=WEEKLY;UNTIL=20200204T151459Z;BYDAY=TU;WKST=SU
EXDATE:20191015T141500Z,20191022T141500Z,20191105T151500Z,20191119T151500Z,
 20191126T151500Z,20191203T151500Z,20191217T151500Z,20191224T151500Z,201912
 31T151500Z
CLASS:PUBLIC
STATUS:CONFIRMED
CREATED:20191013T184131Z
X-MOZ-GENERATION:10
END:VEVENT
END:VCALENDAR

[niccokunzmann]

@wichmannpas Thanks for reporting this. I see the following workflow:

• add ICS example file https://github.com/niccokunzmann/python-recurring-ical-events/tree/master/test/calendars
• add a test, see the example:
  

  python-recurring-ical-events/test/test_issue_15.py

  Line 21 in f4a90b2

  def test_sequence_is_not_present(calendars):

• implement the test
• fix the test

You and anyone is very welcome to take on these tasks. I will not do it today, rather when I like to go into a nice development flow.

[niccokunzmann]

This is the test result:

E AssertionError: BEGIN:VEVENT
E SUMMARY:Test
E DTSTART;TZID=Europe/Berlin;VALUE=DATE-TIME:20191203T161500
E DTEND;TZID=Europe/Berlin;VALUE=DATE-TIME:20191203T174500
E DURATION:PT1H30M
E DTSTAMP:20191219T182547Z
E UID:f0f31ddb-6918-46af-a5a1-0a7254fbce71
E SEQUENCE:11
E CLASS:PUBLIC
E CREATED:20191013T184131Z
E LOCATION:Example
E STATUS:CONFIRMED
E X-MOZ-GENERATION:10
E END:VEVENT
E should not occur at 20191203T151500Z.

It seems, the date of the recurring event is automatically transformed into a time zone. This leads to the exdate not matching. Compare:

• 20191203T151500Z (EXDATE)
• 20191203T161500 (DTSTART)

@wichmannpas, I can confirm the behavior is a bug.

[niccokunzmann]

A this point I wonder if this is a bug in this module. It could very well be that

1. I did not understand properly how the exdate works if it uses different timezones.
   Exdate here is UTC where the dates generated from DTSTART are Europe/Berlin time.
2. This is undefined behavior.
3. The Davx5 software does something unintended.

I will check with the specification. Since Exdates can only be dates, I see no problem with adding the special behavior of EXDATES differing in time zone.

Note for implementation:

• EXDATE also overwrites RDATE values. So, if we make a special case for EXDATE, what would it look like for RDATE?
• Specification: https://www.kanzaki.com/docs/ical/exdate.html

I should contact Davx5 and see if this is intentional or not.

[niccokunzmann]

Posted: https://forums.bitfire.at/topic/2234/differences-in-exdate-calculations-how-to-achieve-compatibility It seems their interpretation is broader but very well in scope of the specification. From my intuition, I would say, this library needs to be changed.

Possible places:

python-recurring-ical-events/recurring_ical_events.py

Line 172 in 8921025

def make_all_dates_comparable(self):

python-recurring-ical-events/recurring_ical_events.py

Line 124 in 8921025

self.exdates.append(exdate.dt)

[niccokunzmann]

This should now be fixed by #25

[niccokunzmann]

This fix is included in v0.1.14b

[wichmannpas]

Thanks for your work and for fixing this! I am currently having rather tight time constraints, so I was not yet able to look into this myself.