Getting the same training puzzles repeatedly, no score updates #813

bilts · 2018-04-19T02:33:30Z

This problem persists in a severe way, beyond logging out, clearing data, and reinstalling.

The app shows me the wrong score and it doesn't update as I complete puzzles. It sends me through the same puzzles in the same order, so I don't get new puzzles and my results rarely appear on the website.

Every once in a while, I'll see some scores reflected or get a new puzzle, but I can't see a rhyme or reason to it.

Puzzles work fine on the website.

l3d00m · 2018-04-22T10:25:16Z

This was reported on reddit too, thought I maybe should leave it here

https://redd.it/8doluf

l3d00m · 2018-04-23T10:09:05Z

And this was reported in lichess' forums as well, some examples:

https://lichess.org/forum/lichess-feedback/bug-report-puzzle-score-inconsistent-between-android-app-and-website-v550

https://lichess.org/forum/lichess-feedback/bug-mobile-app-android-puzzle

https://lichess.org/forum/lichess-feedback/tactics-bug-when-trying-to-retry-this-puzzle-ios

https://lichess.org/forum/lichess-feedback/lichess-app-puzzle-bug-with-dodgy-internet-connection

veloce · 2018-04-23T13:25:39Z

Ok. I know this issue has been reported several times, but I have not been able to reproduce it so far. It may be a client (app) issue but it may be also a server issue. The offline caching functionality adds complexity.

Now, you must send at least the app version (some bugs have been fixed on latest version) and precise steps to reproduce the problem, or I can't do anything about it.

l3d00m · 2018-04-23T13:39:18Z

Sure thing, I'm on app version 5.6.1 and android version 7 with a S7.

Steps:

Open the profile screen, look at the rating there (eg 1400)
Open the puzzle screen, check the current rating there via the overflow menu too (in this example it's 1400 as well)
Play a puzzle (I won one in my example)
Look at the rating via overflow menu again (It updated there this time, sometimes it does not. I now have a rating of 1450)
Look at the rating in the profile (it's still stuck at 1400, although it's updated on lichess web. If I restart the app after killing it, it still shows 1400).

My example was executed while having a full internet connection.

This is just one example though, in other cases the rating in the overflow menu does not update or in very rare cases I have duplicated puzzles after a while. For this I can't get you any steps to reproduce though as of now

bilts · 2018-04-23T16:31:42Z

I see the same as @l3d00m (score stuck at 1492 now, with no puzzles I complete listed). No amount of data clearing or reinstallation fixes it, which seems to indicate a client-server problem rather than a caching problem.

I'm currently on v5.6.1. Username is bilts if that helps in any way. Android 8.1.0

@veloce I'm also a developer with experience in both web and Android. I'm not in a situation where I can commit a ton of time right now, but I can get my dev environment set up and help with inserting / reporting diagnostics if you can point me at useful areas in the codebase to log or alter.

bilts · 2018-04-23T16:41:23Z

I should also mention that I've noticed glitchiness for a while, but when I first noticed this particular set of problems being an issue, I thought the website may have changed to not allow people to solve the same problem twice. My hypothesis was that I was being shown repeats of problems I had solved before, so solving them wasn't affecting my score, so I wasn't able to get new problems.

That said, I should reiterate what I said in my initial report: This all works fine for me on the website.

l3d00m · 2018-04-24T13:37:32Z

Ok, I could reproduce duplicated puzzles too. Same app version as above.

Play a puzzle or two
Stay in the puzzle screen when a new puzzle was loaded and you haven't played it
Exit the app from puzzle screen and force close it (Android makes it easier with Developer Options -> Don't keep apps in background)
Enter the app again and load puzzles
One of the first puzzles from step 1 is loaded again. The next puzzle is usually a new one though

I'm not sure if it can be always reproduced, I'll play around a bit further. This happenened while I had full internet connection

bilts · 2018-04-24T18:46:36Z

I compiled the app and looked at network logs when logged in and have some more details here. Surprisingly, I see the same issue running the app on my computer as I see when running on my phone.

First, my rating is 1443 as reported on the website and through the API, but 1492 in the app (don't want to hear it... it had been over 2000 and I got super lazy and stopped being able to see when it was tanking in the app)

Interestingly, the graphs on the website and in the app roughly agree and show that the 1443 number is from several games ago:

Here are relevant bits from loading a puzzle and playing it (pretty printed for sanity reasons):

https://lichess.org/training/65447/load

{
  "game": {
    "id": "rF8uN7Js",
    "perf": {"icon": ")", "name": "Blitz"},
    "rated": true,
    "players": [
      {"userId": "piet1pegr", "name": "piet1pegr (2044)", "color": "white"},
      {"userId": "yonzon", "name": "yonzon (1799)", "color": "black"}
    ],
    "treeParts": {
      "ply": 30,
      "fen": "r4rk1/pp3ppp/1n6/2qp4/6b1/2PQ1N2/P3BPPP/R3R1K1 w - - 8 16",
      "id": "]A",
      "uci": "c8g4",
      "san": "Bg4"
    },
    "clock": "5+0"
  },
  "puzzle": {
    "id": 65447,
    "rating": 1366,
    "attempts": 20074,
    "fen": "r1b2rk1/pp3ppp/1n6/2qp4/8/2PQ1N2/P3BPPP/R3R1K1 b - - 7 15",
    "color": "white",
    "initialPly": 30,
    "gameId": "rF8uN7Js",
    "lines": {"f3g5": {"g7g6": {"e2g4": {"c5d6": "win"}}}},
    "vote": 74,
    "branch": {
      "ply": 31,
      "fen": "r4rk1/pp3ppp/1n6/2qp2N1/6b1/2PQ4/P3BPPP/R3R1K1 b - - 1 16",
      "id": "8I",
      "uci": "f3g5",
      "san": "Ng5",
      "children": [
        {
          "ply": 32,
          "fen": "r4rk1/pp3p1p/1n4p1/2qp2N1/6b1/2PQ4/P3BPPP/R3R1K1 w - - 0 17",
          "id": "YQ",
          "uci": "g7g6",
          "san": "g6",
          "children": [
            {
              "ply": 33,
              "fen": "r4rk1/pp3p1p/1n4p1/2qp2N1/6B1/2PQ4/P4PPP/R3R1K1 b - - 0 17",
              "id": "/A",
              "uci": "e2g4",
              "san": "Bxg4",
              "children": []
            }
          ]
        }
      ]
    },
    "enabled": true
  },
  "mode": "play",
  "user": {
    "rating": 1443,
    "recent": [
      [64880, -3, 1425],
      [64886, 12, 1422],
      [64889, 15, 1434],
      [64892, -6, 1449],
      [65011, 5, 1542],
      [65020, 12, 1547],
      [65024, -10, 1559],
      [65028, -8, 1549],
      [65031, 8, 1541],
      [65032, -8, 1549],
      [65033, 7, 1541],
      [65035, 12, 1548],
      [65038, -15, 1560],
      [65258, -6, 1510],
      [65259, -12, 1504]
    ]
  }
}

https://lichess.org/training/65447/round2

Request URL: https://lichess.org/training/65447/round2
Request Method: POST
Status Code: 200 OK
Remote Address: 37.187.205.99:443
Referrer Policy: no-referrer-when-downgrade

Body: {"win":1}

{
  "user": {
    "rating": 1443,
    "recent": [
      [64880, -3, 1425],
      [64886, 12, 1422],
      [64889, 15, 1434],
      [64892, -6, 1449],
      [65011, 5, 1542],
      [65020, 12, 1547],
      [65024, -10, 1559],
      [65028, -8, 1549],
      [65031, 8, 1541],
      [65032, -8, 1549],
      [65033, 7, 1541],
      [65035, 12, 1548],
      [65038, -15, 1560],
      [65258, -6, 1510],
      [65259, -12, 1504]
    ]
  },
  "round": {"ratingDiff": 0, "win": true},
  "voted": null
}

Notice that it is registering the win, but awarding no points and thus isn't showing up in the score history.

It could very well be that I've played this particular puzzle before (I don't remember it) and the site is refusing to give points for replaying an app. That would be a clean explanation, but I can't say for sure. As I have said before, though, I've noticed a lot of duplicate puzzles appearing.

If the puzzles selected are directed by the server, it would seem that this is a lichess.org issue.

*edit ... I'm waffling on that last statement, because as I mentioned before, this all works flawlessly when I play on the lichess website. I'm only affected in the app.

ghost · 2018-04-25T06:54:12Z

What do you want in the way of reproduction steps? I pretty much only play puzzles on Lichess so this problem is 100% reproducable every day, for months.

Perhaps as a starting point add some logging to the app to dump my puzzle score/history there locally and compare against what's on the server. It's as if there are additional puzzle history locally which makes the score differ.

Oddly, if I use two mobile devices they both stay in sync and have the same (wrong) score compared to the server. I just solved a few puzzles this morning; the server is correct but my score hasn't changed on my phone.

I'm a developer and I can spend time on this issue. What do you want me to do?

bilts · 2018-04-25T20:19:57Z

Poking around a bit in the Chrome console, I've noticed several concrete bugs that would explain some of the problems:

If I play in the app, the batch API feeds me a new puzzle after I solve the first, which gets queued up. If I then play online, I'm given the same puzzle fed by the batch API. Since the one given to the app is still queued, I'll see it ~50 puzzles later, when I probably won't remember it.
I've yet to see the API return a ratingDiff of anything other than 0.
The API returns my current, correct rating (as listed in the website) from the latest call to /training/batch?nb=1 in the user object, but this is not the score in the menu. Something is failing to propagate.
The first time I solve a puzzle after refreshing the page, if I make an incorrect move, it gets reported to the site using the /training/____/round2 API. If I continue to play, it'll force me to play the same puzzle again and use the /training/batch API.

And one that's unrelated but should probably be fixed nonetheless:

Offline games are a queue that's 50 long. It's possible for ratings to drift a lot over the course of 50 puzzles, which means there could be significant lag in difficulty adjustment. I think I've actually noticed this but have attributed it to other things.

l3d00m · 2018-04-30T16:09:35Z

Something new on my end: The duplicate puzzles happen on lichess' site as well. I finished this puzzle (https://lichess.org/training/68431) first in the app, then I got it again at lichess.org (after finishing rating + 0).

I did a few other puzzles and exited lichess.org. After returning back I got the puzzle again, exactly what happened in the app too. So it might be a lichess web problem too. I'll open an issue there.

l3d00m · 2018-05-04T06:47:44Z

As described in lichess-org/lila#4317, I could reproduce the other reported bug on lichess web as well. It seems like a general lichess bug and not one only in the app.

bilts · 2018-05-04T12:29:49Z

After dealing with it for over a month. I'm no longer seeing this in the app. Don't know what changed, maybe something server side.

veloce · 2018-05-04T12:38:25Z

Something changed server side indeed. I also made a couple of edge cases fixes on app side too. Closing this then.

natancox · 2018-09-09T17:28:03Z

Still is happening. Even after reinstall of Android app. This issue should be reopened.

fwn · 2019-03-08T16:03:08Z

This issue should be reopened as it still persists in 6.1.1.

This bug can be completely avoided by using 5.3.1.

Bug can ONLY be reproduced if

first the app version is greater 5.3.1
and second batches of puzzles are solved on at least two clients alternately. (e.g.: The app and the web client or two apps on different devices.)

If clients are alternated - let's say first app (works!), then web (works!), then app (bug!) - puzzles get out of sync.

All puzzles played on the web will need to be replayed in the app without the account getting any rating for it.

Bug symptoms disappear after

either replaying all previously solved puzzles in the app alone
or clearing app data on the device.

We can provide documentation for this asynchronous puzzle behavior as described if needed, as we already tested several devices and app versions to narrow down the problem.

We would love to provide any additional information to get this bug resolved, as it is an otherwise awesome, beautiful app!

edit regarding version 6.1.2:

While the underlying sync issue does not appear to be fixed in this release, the new "refresh puzzles button" serves as a handy additional method to make the bug symptoms disappear. That's much better than clearing app data!

Emobe · 2019-07-18T14:22:42Z

This still happens...

fmmarzoa · 2020-01-20T11:38:26Z

This is still happening.

If I remove the app data in Android, it will start showing new puzzles but as soon as I get back to the game after leaving, it'll start repeating the same old ones always from the same puzzle.

gety9 · 2020-01-25T22:32:15Z

Still happening on iOS. App version 6.3.0

l3d00m · 2020-01-25T23:02:06Z

@veloce Could you reopen this? There are very regular reports about this on lichess forums too, see this search:

https://lichess.org/forum/search?text=puzzle+app

Edit: After searching some new issues I saw that this was also acknowledged in #876, so nvm

vpatov · 2020-01-27T05:48:38Z

I am also experiencing the same issues described here. It has made the puzzles experience bad enough for me that I do them much less :(

igroglaz · 2020-01-30T22:25:46Z

The same puzzles: https://lichess.org/training/61982 and https://lichess.org/training/61987

feafarot · 2020-08-27T19:20:38Z

~~The issue is still present on the latest app~~
Nvm. Found sync puzzles button after searching through releases, and it works perfectly.

l3d00m mentioned this issue Apr 22, 2018

Training puzzle #818

Closed

bilts mentioned this issue Apr 24, 2018

GET /training/batch returns incorrect user rating lichess-org/lila#4061

Closed

This was referenced Apr 30, 2018

Endless puzzle loop [android] lichess-org/lila#4194

Closed

Duplicate puzzles and wrong puzzle ratings on lichess web lichess-org/lila#4317

Closed

veloce closed this as completed May 4, 2018

veloce added this to the 6.0.0 milestone May 4, 2018

niklasf mentioned this issue May 7, 2018

Mobile (iPhone) training ratings not updating lichess-org/lila#4303

Closed

wmhopkins mentioned this issue Aug 27, 2018

Incorrect puzzle history lichess-org/lila#4579

Closed

rwuwon mentioned this issue Oct 15, 2018

Puzzle chart and ranking is not updated #906

Closed

DarthFlappy mentioned this issue Jul 23, 2019

Puzzle issue #1008

Closed

This was referenced Jan 25, 2020

BUG - Training remove points #1104

Open

Chess puzzle problem with new latest update #1039

Closed

gety9 mentioned this issue Jan 26, 2020

Strange issue with Chess Problems changing smartphone #1111

Open

ahmadmysra mentioned this issue Jul 16, 2020

Always the same tactical exercises #1308

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Getting the same training puzzles repeatedly, no score updates #813

Getting the same training puzzles repeatedly, no score updates #813

bilts commented Apr 19, 2018

l3d00m commented Apr 22, 2018 •

edited

Loading

l3d00m commented Apr 23, 2018 •

edited

Loading

veloce commented Apr 23, 2018

l3d00m commented Apr 23, 2018 •

edited

Loading

bilts commented Apr 23, 2018

bilts commented Apr 23, 2018

l3d00m commented Apr 24, 2018 •

edited

Loading

bilts commented Apr 24, 2018 •

edited

Loading

ghost commented Apr 25, 2018 •

edited by ghost

Loading

bilts commented Apr 25, 2018

l3d00m commented Apr 30, 2018 •

edited by ornicar

Loading

l3d00m commented May 4, 2018 •

edited

Loading

bilts commented May 4, 2018

veloce commented May 4, 2018 •

edited

Loading

natancox commented Sep 9, 2018

fwn commented Mar 8, 2019 •

edited

Loading

Emobe commented Jul 18, 2019

fmmarzoa commented Jan 20, 2020

gety9 commented Jan 25, 2020

l3d00m commented Jan 25, 2020 •

edited

Loading

vpatov commented Jan 27, 2020

igroglaz commented Jan 30, 2020

feafarot commented Aug 27, 2020 •

edited

Loading

Getting the same training puzzles repeatedly, no score updates #813

Getting the same training puzzles repeatedly, no score updates #813

Comments

bilts commented Apr 19, 2018

l3d00m commented Apr 22, 2018 • edited Loading

l3d00m commented Apr 23, 2018 • edited Loading

veloce commented Apr 23, 2018

l3d00m commented Apr 23, 2018 • edited Loading

bilts commented Apr 23, 2018

bilts commented Apr 23, 2018

l3d00m commented Apr 24, 2018 • edited Loading

bilts commented Apr 24, 2018 • edited Loading

ghost commented Apr 25, 2018 • edited by ghost Loading

bilts commented Apr 25, 2018

l3d00m commented Apr 30, 2018 • edited by ornicar Loading

l3d00m commented May 4, 2018 • edited Loading

bilts commented May 4, 2018

veloce commented May 4, 2018 • edited Loading

natancox commented Sep 9, 2018

fwn commented Mar 8, 2019 • edited Loading

Emobe commented Jul 18, 2019

fmmarzoa commented Jan 20, 2020

gety9 commented Jan 25, 2020

l3d00m commented Jan 25, 2020 • edited Loading

vpatov commented Jan 27, 2020

igroglaz commented Jan 30, 2020

feafarot commented Aug 27, 2020 • edited Loading

l3d00m commented Apr 22, 2018 •

edited

Loading

l3d00m commented Apr 23, 2018 •

edited

Loading

l3d00m commented Apr 23, 2018 •

edited

Loading

l3d00m commented Apr 24, 2018 •

edited

Loading

bilts commented Apr 24, 2018 •

edited

Loading

ghost commented Apr 25, 2018 •

edited by ghost

Loading

l3d00m commented Apr 30, 2018 •

edited by ornicar

Loading

l3d00m commented May 4, 2018 •

edited

Loading

veloce commented May 4, 2018 •

edited

Loading

fwn commented Mar 8, 2019 •

edited

Loading

l3d00m commented Jan 25, 2020 •

edited

Loading

feafarot commented Aug 27, 2020 •

edited

Loading