Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

some events are not displayed in the calendar when offline mode is active #4660

Closed
30 of 42 tasks
ganthern opened this issue Oct 11, 2022 · 1 comment · Fixed by #4681
Closed
30 of 42 tasks

some events are not displayed in the calendar when offline mode is active #4660

ganthern opened this issue Oct 11, 2022 · 1 comment · Fixed by #4681
Labels
android issues that concern the android client but not all clients bug broken functionality, usability problems, unexpected errors desktop Desktop client related issues state:done meets our definition of done state:tested We tested it and are about to release it topic: calendar topic:offline issues with the offline database of the native clients
Milestone
3.103.3

Comments

@ganthern
Copy link
Contributor

ganthern commented Oct 11, 2022 •
edited by charlag

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. create a premium account
  2. log in with the desktop client without saving credentials
  3. import the events below (that should repeat every year) into any calendar
  4. see that the events are displayed
  5. log out, log back in with the "save credentials" checkbox activated
  6. see that the events are only displayed in the original location (the occurrence of this year is missing)

Expected behavior
events should be displayed no matter for which date they are originally defined

Additional context
One ical file for reproduction:

BEGIN:VCALENDAR
VERSION:2.0
BEGIN:VEVENT
DTSTART;VALUE=DATE:19300510
DTEND;VALUE=DATE:19300511
DTSTAMP:20221008T104036Z
UID:7BD7F21E-0000-0000-0000-000000000000
SEQUENCE:0
SUMMARY:Geburtstag von XY
RRULE:FREQ=YEARLY;INTERVAL=1
END:VEVENT
BEGIN:VEVENT
DTSTART;TZID=Europe/Berlin:19650426T230000
DTEND;TZID=Europe/Berlin:19650427T230000
DTSTAMP:20221008T104036Z
UID:040000008200E00074C5B7101A82E0080000000020640A4E4135C90100000000000000001000000024D9E952CED6F049BB93C0728CCDA669
SEQUENCE:0
SUMMARY:Privater Termin
RRULE:FREQ=YEARLY;INTERVAL=1
END:VEVENT
END:VCALENDAR

Test notes:

  • Open calendar event creation dialog
  • select an end-date which is the same as start-date and click save
  • dialog should inform that this is not possible, because start is not before end
  • select an end-date which is before the start-date and click save
  • dialog should inform that this is not possible, because start is not before end
  • Create a new Calendar and import the attached .txt file
  • a dialog pops up stating that 2 elements already exist
  • press Cancel and no events should show up in october 22
  • import the same .txt file again
  • press ok for the first dialog informing about duplicate elements
  • a second dialog pops up informing that two events are earlier than 1970
  • press cancel and no events should show up in october 22
  • import the .txt file again and click ok on both popups
  • events should appear in october 22
  • import the .txt file again
  • a dialog informing that there are 10 duplicate items should show up -> click ok
  • a dialog informing that 2 events are before 1970 shows up -> click ok
  • no new events are imported to october 22

Regressions:

  • Open calendar event creation dialog
  • type in a start-date which is before 1970 and click the end-date field
  • start-date should change automatically to the current date
  • type in a start-date which is not an actual date, for example some string or numbers and click out of the field
  • field should be set to a valid date
  • type in an end-date which is not an actual date, for example some string or numbers and click out of the field
  • field should be set to a valid date

Migration:

broken.txt
@ganthern ganthern added bug broken functionality, usability problems, unexpected errors desktop Desktop client related issues android issues that concern the android client but not all clients topic: calendar topic:offline issues with the offline database of the native clients labels Oct 11, 2022
nokhub added a commit that referenced this issue Oct 13, 2022
@nokhub
fix dates from before 1970 being truncated during offline serialization
546195e 
we use CBOR blobs to encode our entities for storage in the offline DB.
This requires to tag items with type information, which was done
incorrectly for dates: we always set the type to uint, which is wrong
when storing the result of Date.getTime (ms since Jan 1 1970) for dates
before 1970 - these are negative. CBORG truncated these to 0, which
results in wrong start and end times after deserialization and failure
to display the events.

fix #4660
@ganthern
Copy link
Contributor Author

ganthern commented Oct 17, 2022 •
edited by nokhub

We have a bug in the serialization of negative (pre-1970) dates, which would be zeroed out when deserializing.

this leads to all manner of broken events like equal start and end time.

we need to fix it thusly:

  • fix serialization bug
  • prevent import of pre-1970 dates for now
  • write migration to re-download broken events to the offline cache.

nokhub added a commit that referenced this issue Oct 18, 2022
@nokhub
prevent creation of some calendar event edge cases
45e8b6f 
we opted not to handle pre-1970 dates in the calendar and prevent them
from being created through the ui, along some other validation.

this commit consolidates these checks into a single location and applies
them to events created through the UI and through imports.

#4660
nokhub added a commit that referenced this issue Oct 18, 2022
@nokhub
repair existing broken events in offline dbs
a4a3c99 
we have some cases of technically illegal events that got imported into
some offline DBs. they were serialized in a lossy way and can be
detected by checking for zeroed out start- and end times.

this migration deletes the event lists from the offline db that contain
such events, they will then be re-downloaded when the calendar is
populated in the cache postlogin action.

#4660
nokhub added a commit that referenced this issue Oct 19, 2022
@nokhub
repair existing broken events in offline dbs
f136d59 
we have some cases of technically illegal events that got imported into
some offline DBs. they were serialized in a lossy way and can be
detected by checking for zeroed out start- and end times.

this migration deletes the event lists from the offline db that contain
such events, they will then be re-downloaded when the calendar is
populated in the cache postlogin action.

we're leaving events belonging to these lists in the cache, they will be
overwritten while redownloading because of the use of INSERT OR REPLACE
and the way we have set up our primary keys.

#4660
@nokhub nokhub mentioned this issue Oct 19, 2022
Merged
@charlag charlag assigned charlag and unassigned ganthern and nokhub Oct 24, 2022
@charlag charlag added this to the Release 3.103.x milestone Oct 27, 2022
charlag pushed a commit that referenced this issue Oct 27, 2022
@nokhub @charlag
fix dates from before 1970 being truncated during offline serialization
fe8a8d9 
we use CBOR blobs to encode our entities for storage in the offline DB.
This requires to tag items with type information, which was done
incorrectly for dates: we always set the type to uint, which is wrong
when storing the result of Date.getTime (ms since Jan 1 1970) for dates
before 1970 - these are negative. CBORG truncated these to 0, which
results in wrong start and end times after deserialization and failure
to display the events.

fix #4660
charlag pushed a commit that referenced this issue Oct 27, 2022
@nokhub @charlag
prevent creation of some calendar event edge cases
fe240cb 
we opted not to handle pre-1970 dates in the calendar and prevent them
from being created through the ui, along some other validation.

this commit consolidates these checks into a single location and applies
them to events created through the UI and through imports.

#4660
charlag pushed a commit that referenced this issue Oct 27, 2022
@nokhub @charlag
repair existing broken events in offline dbs
c3ea057 
we have some cases of technically illegal events that got imported into
some offline DBs. they were serialized in a lossy way and can be
detected by checking for zeroed out start- and end times.

this migration deletes the event lists from the offline db that contain
such events, they will then be re-downloaded when the calendar is
populated in the cache postlogin action.

we're leaving events belonging to these lists in the cache, they will be
overwritten while redownloading because of the use of INSERT OR REPLACE
and the way we have set up our primary keys.

#4660
@charlag charlag removed their assignment Oct 27, 2022
@nif-tutao nif-tutao self-assigned this Nov 1, 2022
@charlag charlag self-assigned this Nov 2, 2022
@charlag charlag added the state:tested We tested it and are about to release it label Nov 2, 2022
@charlag charlag removed their assignment Nov 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
android issues that concern the android client but not all clients bug broken functionality, usability problems, unexpected errors desktop Desktop client related issues state:done meets our definition of done state:tested We tested it and are about to release it topic: calendar topic:offline issues with the offline database of the native clients
Projects
None yet
Milestone
3.103.3
Development

Successfully merging a pull request may close this issue.

Fix date serialization 4660 tutao/tutanota
4 participants
@charlag @ganthern @nif-tutao @nokhub