[BUG] Parameters caused long interval seems wrong

[L-M-Sherlock]

          I ran the optimizer again with the new revlog_start_date setting of "2020-01-01", and got this result: 1.9755, 2.285, 2.495, -0.1222, -2.2217, 0.0009, 2.0, -0.0532, 1.5, 2.5552, -0.0921, 0.5222, 1.3403

I'm still using the fsrs4anki-helper.zip you linked, and maybe I need to update that, but after rescheduling with the new w values, I still have 9878 cards with intervals of 10 years or more, with the most egregious intervals shown here:

Originally posted by @mantixero in open-spaced-repetition/fsrs4anki-helper#49 (comment)

[L-M-Sherlock]

@mantixero I fixed the optimizer for your case. Could you use it to generate new parameters and reschedule your collection with them?

https://colab.research.google.com/github/open-spaced-repetition/fsrs4anki/blob/Fix/ignore-incomplete-revlogs-in-optimizer/fsrs4anki_optimizer.ipynb

[mantixero]

Thank you for trying a fix to the optimizer. I'm trying to run it, but I get the following error:

[user1823]

Exactly when do you see this error message? Just after clicking the link? I don't see this message.

[mantixero]

It occurs after clicking the link, then clicking on the button that says "Open in Colab". Maybe the link is already in Colab, and there is no need to click that button?

[user1823]

Yeah, I think that you don't need to click that button.

[L-M-Sherlock]

"Open in Colab" only works for release versions.

[mantixero]

Okay, thank you. The optimizer is running now.

By the way, I was reading through the usage instructions for FSRS4Anki, and I noticed the line The requestRetention of FSRS4Anki is equivalent to the interval modifier. Maybe this was part of the problem for me? I was using an interval modifier of 0.82 with Anki's v2 scheduler, but I used a requestRetention value of 0.93 with FSRS, because I thought it meant to target 93% retention. Should I set requestRetention to 0.82 in order to make intervals only 82% as long as they would otherwise be, or will the optimizer take that into consideration given that my review history is based on intervals that are 18% shorter than normal v2 intervals?

[L-M-Sherlock]

The optimizer calculate the true retention before training. So the interval modifier doesn't affect the optimization process.

[user1823]

By saying that the requestRetention of FSRS4Anki is equivalent to the interval modifier, @L-M-Sherlock meant to say that the purpose of adjusting the requestRetention in FSRS is same as the purpose of adjusting the interval modifier in Anki.

In Anki, you adjust the interval modifier to control your retention rate indirectly.
In FSRS, the requestRetention provides you with a direct way to adjust the retention rate.

In your case, you should set the requestRetention to 0.93 to achieve a 93% retention rate.

[mantixero]

I see. If @L-M-Sherlock agrees with the above characterization of what he meant, I will leave requestRetention at 0.93.

[user1823]

Also, as pointed out by @L-M-Sherlock in his above comment, the value of requestRetention doesn't affect the w values produced by the optimizer. However, the requestRetention does affect the intervals produced by the helper and the scheduler.

[mantixero]

Thank you for the clarification, although I did realize that requestRetention is only taken into account via the custom scheduling box under Anki's v3 deck options, not as a variable when running the optimizer.

[L-M-Sherlock]

Here is the formula:
Interval Modifier = log(requestRetention) / log(90%)

[user1823]

@mantixero, I am waiting for the results. Were our efforts successful?

[mantixero]

using a revlog_start_date of 2006-10-05, the optimizer finished with w values of the following:
[3.3685, 5.0, 4.7342, -1.4386, -2.2743, 0.0152, 1.9768, -0.1614, 1.2641, 2.0268, -0.1922, 0.6378, 0.2776]

previously, with revlog_start_date set to 2020-01-01, I had gotten these results: [1.9755, 2.285, 2.495, -0.1222, -2.2217, 0.0009, 2.0, -0.0532, 1.5, 2.5552, -0.0921, 0.5222, 1.3403]

After rescheduling my cards, my reviews due in my main deck went from 2741 (after reverting to v2 intervals yesterday) to 2403. Keep in mind, before reverting to v2 intervals yesterday, I had only 90 reviews due in my main deck, so the 2000+ new reviews probably reflect the reviews I should have been doing these past 5 months had my w values been more realistic. This seems to be a good direction, and the slightly reduced reviews compared to v2 are what I would expect if FSRS is reducing the unnecessary reviews of the v2 scheduler. I'll look into some of the intervals via the browser, but so far, I feel more confident with these results.

[mantixero]

Just to clarify, the interval underlined in red is FSRS's rescheduled interval, and the one underlined in blue is Anki's v2 scheduler interval, correct?

[user1823]

Just to clarify, the interval underlined in red is FSRS's rescheduled interval, and the one underlined in blue is Anki's v2 scheduler interval, correct?

Yes

[mantixero]

Thank you for confirming. Looking at some of the cards from issue#54, one of the mature cards in question went from an interval of 5.6 years to the much more realistic 1.98 years, and another just out of the Learning stage went from 4.9 months to a more reasonable sounding 2.33 months:

As far as I can tell, outside of the 4 bugged cards with 26+year intervals, everything else is looking pretty good! I'll keep using FSRS and see what my retention rate turns out like once I get my neglected reviews under control.

[L-M-Sherlock]

the 4 bugged cards with 26+year intervals

What about these cards? What are their original intervals?

[mantixero]

Here are the cards' info. I've already set a new due date for each of them, as you can see in their latest "Manual" state. I don't know if it's a bug, but there is no custom data field for these, nor the 岡 card I showed you yesterday, nor many other cards in that deck. Is there a reason some cards wouldn't get a custom data field, and is there a way to search for all such cards in the browser that lack custom data?

Here's another example of a card (罰) that doesn't have custom data, although it has a believable interval:

[L-M-Sherlock]

Oh, these cards seems to be reseted.

[L-M-Sherlock]

Oh, these cards seems to be reseted.

[mantixero]

The difference seems to be that if a card has been returned to the Learn state without a 1 rating, it doesn't have custom data. If it returned to the Learn state with a 1 rating, it will have custom data. Does that mean that FSRS4Anki Helper can't modify the intervals of these cards, and they will have v2 intervals?

[mantixero]

Oh, these cards seems to be reseted.

Yes, I reset them about an hour ago after running the optimizer and rescheduling them with FSRS4Anki Helper to no effect.

[mantixero]

That's not true, because I ran the helper first, then saw that their intervals were still the same, so I reset them via set new date, but the fact is that NONE of the cards that are missing custom data are getting rescheduled, not just these 4. Like I said above, it seems that ANY card that ever went back to the Learn state without a 1 rating is being ignored by the Helper.

Edit: I still had Anki open, so I mashed Ctrl + Z (Undo) just now until all 4 cards went back to their prior status (Review, not Manual).

[user1823]

That's not true

Yeah, I am sorry. I didn't see that the first review of these cards is not of learn type. The helper is designed to skip such cards.

But, still I would advise you to restore from a backup just before you reset these cards. (Or undo the reset using Ctrl + Z, if possible)
This is because the manual rescheduling would interfere with our ability to fix the issue in the future.

[mantixero]

Yes, I undid the reset using Ctrl + Z. Now I just need the Helper to not skip such cards, because there are MANY more like this that just happen to have more believable intervals.

[user1823]

@L-M-Sherlock

Now, we need to change this line
if i == 0 and (revlog.review_kind not in (0, 2)) and not has_again(revlogs):
to
if i == 0 and (revlog.review_kind not in (0, 2)) and not (has_again(revlogs) or has_manual_reset(revlogs)):
and define the function has_manual_reset.

By manual reset, I mean manual reset without the manual rating in the revlog (open-spaced-repetition/fsrs4anki-helper#51).

[mantixero]

the first review of these cards is not of learn type. The helper is designed to skip such cards.

Here, you can see that this card (for the kanji 邪) doesn't have a first review of the Learn type, but it still has custom data. I really think it has more to do with it returning to Relearn via a 1 rating that keeps it from being skipped by the Helper:

[user1823]

I really think it has more to do with it returning to Relearn via a 1 rating that keeps it from being ignored.

Yes, it is the case. But, the first thing that is checked is whether the first review type is learn or not.

This rule of the helper was developed on my request. So, I know how it works.

[L-M-Sherlock]

Now, we need to change this line
if i == 0 and (revlog.review_kind not in (0, 2)) and not has_again(revlogs):
to
if i == 0 and (revlog.review_kind not in (0, 2)) and not (has_again(revlogs) or has_manual_reset(revlogs)):
and define the function has_manual_reset.

I will implement it tomorrow.

[mantixero]

Thank you both for staying with me on this issue. I look forward to all my cards having custom data and truly optimized intervals based on my remembering/forgetting history.

[user1823]

By manual reset, I mean manual reset without the manual rating in the revlog (open-spaced-repetition/fsrs4anki-helper#51).

I thought about it again and realised that it is better to include both manual reset with revlog and without revlog.

The reason is that it doesn't actually matter whether the manual entry is there in the revlogs or not. If the card was reset, it should not be withheld because the first review in the revlog was not of learn or relearn type.