Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tasks become unsorted - task list sort order is not preserved (tasksOrder errors) #7645

Open
Alys opened this issue Jun 11, 2016 · 20 comments

Comments

Projects
None yet
8 participants
@Alys
Copy link
Contributor

commented Jun 11, 2016

Sometimes when you sort your To-Dos, some or all of them will become unsorted, either immediately after you have sorted them, or later when you add new To-Dos.

The workaround is to post to the Report a Bug guild to say that your To-Do sort order is not being saved. An admin will then cleanse some data in your account which will fix the problem. You cannot fix this problem yourself. In particular, deleting completed To-Dos will NOT help (it used to help for a previous version of this bug, but it won't make any difference now).

It is likely that this bug occurs only during periods of heavy site use, or if you mark several To-Dos as completed very quickly, or if you mark one as completed at the same time as taking other actions on the site very quickly. To minimise the chance of this bug happening (and certain other bugs and sync errors):

  • on the website, each time you take an action, wait for the blue sync line at the top of the site to disappear before you take another action
  • on the mobile app, each time you mark a task as complete, don't take any other actions until you've seen the notification at the bottom of the screen that tells you how much XP and GP you earned

Technical Details


EDIT by @paglias: the best place to fix this is at cron. It can also happen that some tasks ids are missing from user.tasksOrder, this should be fixed too.


When you mark a To-Do as completed, Habitica has to modify the To-Do's record and also remove the To-Do's ID from a list that stores the sort order of your incomplete To-Dos (tasksOrder.todos). If Habitica is overloaded at the time, it might not do that second action. Thus the sort-order list can contain IDs for completed To-Dos, which interfere when you try to sort your incomplete To-Dos.

This was found when PaperCrystals (52e31d82-e731-4713-81cb-6ef8c5d5d9bc) reported problems with sorting To-Dos in the Report a Bug guild. PaperCrystals's tasksOrder.todos array contained the IDs of five To-Dos that had been completed. Their completion dates were all within 64 seconds, indicating that there might have been problems with the user document being updated multiple times in a short period:

"dateCompleted" : ISODate("2016-05-28T19:48:28.021Z")
"dateCompleted" : ISODate("2016-05-28T19:48:46.384Z")
"dateCompleted" : ISODate("2016-05-28T19:49:00.962Z")
"dateCompleted" : ISODate("2016-05-28T19:49:23.729Z")
"dateCompleted" : ISODate("2016-05-28T19:49:32.054Z")

I was able to replicate the bug by inserting fake task IDs into my own tasksOrder.todos array. To-Do sorting then became impossible: I could move a To-Do to a new position, but as soon as I clicked sync, it would jump to a different position.

To find out how widespread this problem might be, I am searching for completed To-Dos in the tasksOrder.todos array for all players who logged in in the past three months (since 2016-03-15). It's going to take a while for that to finish, but initial results indicate that maybe 2% of the IDs in the tasksOrder.todos arrays are for completed To-Dos.

@Alys

This comment has been minimized.

Copy link
Contributor Author

commented Jun 11, 2016

I got bored and stopped the search, but I was getting a pretty consistent result: Out of the To-Dos IDs appearing in the user's tasksOrder.todos array, about 2 to 3 % are actually for completed To-Dos, so we need to clean up those arrays. We could either do a stand-alone migration script that we run now, and then re-run it in future, however often seems necessary, or build something automatic that runs regularly, maybe as part of cron.

I guess which we do depends on how easily this problem can be stopped. If it's happening just because the code times out and does finish executing properly, it might be really hard to stop it.

@paglias @crookedneighbor What do you think?

@crookedneighbor

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2016

Should be trivial to purge them at cron.

@paglias

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2016

Yeah fixing them at cron is the best option

@paglias

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2016

I'm closing this in favor of #7790

@paglias paglias closed this Sep 3, 2016

@Alys

This comment has been minimized.

Copy link
Contributor Author

commented Sep 3, 2016

I'm reopening this because it'ssthe most useful one for players looking for a sorting bug, and because it describes what mingt be a cause of the bug and the suggested fix.I'll close the other one.

@Alys Alys reopened this Sep 3, 2016

@paglias

This comment has been minimized.

Copy link
Contributor

commented Sep 3, 2016

Okay, anyone working on this: the best place to fix this is at cron

@Alys Alys changed the title To-Dos become unsorted - ToDo list sort order is not preserved tasks become unsorted - last list sort order is not preserved Oct 3, 2016

@Alys Alys changed the title tasks become unsorted - last list sort order is not preserved tasks become unsorted - last list sort order is not preserved (tasksOrder errors) Jan 10, 2017

@Alys Alys changed the title tasks become unsorted - last list sort order is not preserved (tasksOrder errors) tasks become unsorted - task list sort order is not preserved (tasksOrder errors) Feb 21, 2017

@TNychka

This comment has been minimized.

Copy link
Contributor

commented Apr 12, 2017

Still need help on this one? I'm willing to take a look at it.

@Alys

This comment has been minimized.

Copy link
Contributor Author

commented Apr 12, 2017

@TNychka Thank you, it would be great if you could work on this!

The most pressing fix is to have a function that runs during every cron to fix taskOrder discrepancies, and that's currently our only requirement for this issue.

I personally would also like to see a feature in future that gives the user a button to click to run that function at any time (because any player who keeps their tasks in specific orders will not want to wait until cron for a sort problem to be fixed) and/or I'd like to see error checking added to any code that updates tasksOrder to prevent discrepancies from ever occurring. However those are things that we can make new issues for if it's deemed necessary after the cron fix is in place.

The function should ensure that the tasksOrder arrays contain the ID of every task owned by the user (including tasks from challenges and group plans that the user has joined), except that the IDs of completed To-Dos must not appear in tasksOrder.todos.

The function should also ensure that the tasksOrder arrays do not contain any IDs for deleted tasks (i.e., if tasksOrder contains an ID that doesn't match any existing task, remove that ID).

The function should also ensure that the tasksOrder arrays do not contain any duplicate IDs (i.e., if an ID appears in tasksOrder two or more times, remove all but one).

@TNychka

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2017

So, I'm very close to fixing this, there is just one issue I'm encountering you can probably answer pretty quickly.
Currently it appears that I've fixed the reordering bug, it will now not include tasks that have been completed when a user drags and drops so it won't "randomize" the task locations anymore.
But, obviously I would like to fix the issue of having the completed tasks in there in the first place, I'm doing it in the cron function of libs/cron.js as suggested, but when the cron function is called the modifications to the user.todos do not persist to the database. What is the correct way to modify the user object in this case? I've encountered this issue before. I'll play around with this a little more to see what I can do with it too.

the code I currently have

  _.remove(user.tasksOrder.todos, task => {
    return task.completed;
 });
@paglias

This comment has been minimized.

Copy link
Contributor

commented Aug 7, 2017

@TNychka user.todos doesn't exist on the server, it's only used in the client version. On the server you have user.tasksOrder.{taskType} which is an array of sorted tasks ids divided by task type and tasksOrder.todos should already include only non completed todos. The real todos are stored in the tasks collection inside the database

@Alys

This comment has been minimized.

Copy link
Contributor Author

commented Aug 7, 2017

One aspect of this bug is that tasksOrder.todos sometimes does include the IDs of completed To-Dos - I see that often in reports of this bug. It can also contain the IDs of deleted To-Dos, sometimes does not have the IDs of incomplete To-Dos, and occasionally contains duplicated IDs. The same applies to the other tasks types, although To-Dos are the most common culprit.

@MathWhiz

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2017

@TNychka Are you still working on this?

@TNychka

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2017

@MathWhiz yes, the task is finished and the PR is in progress here #8928

@MathWhiz

This comment has been minimized.

Copy link
Contributor

commented Oct 20, 2017

@MathWhiz

This comment has been minimized.

Copy link
Contributor

commented Dec 13, 2017

Has this been fixed? @paglias

@TheHollidayInn

This comment has been minimized.

Copy link
Collaborator

commented Dec 13, 2017

I believe this has been fixed by some of my recent updates. @Alys Are we still getting reports? It is possible I missed some.

@paglias

This comment has been minimized.

Copy link
Contributor

commented Dec 13, 2017

Let's reopen if we get more reports

@paglias paglias closed this Dec 13, 2017

@Alys Alys reopened this Dec 13, 2017

@Alys

This comment has been minimized.

Copy link
Contributor Author

commented Dec 13, 2017

Yes, we still get reports, and the fix in the cron code won't work when the player is resting in the inn (#9627 (comment)). :)

@TheHollidayInn From memory, the code I saw wouldn't cover every case that causes this, but I might be misremembering. Are you able to find your PR or commits for it easily? I'd like to link to them from here, but don't worry about it if they're too hard to track down.

@beffymaroo

This comment has been minimized.

Copy link

commented Apr 17, 2018

@TheHollidayInn @Alys Is this still in process?

@Alys

This comment has been minimized.

Copy link
Contributor Author

commented Apr 17, 2018

The bug still happens. I don't think there's a fix currently in progress.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.