Can this be used as a passive review scheduler?

[c1-g]

In Supermemo, there're 4 types of element: Items, topics, concepts, and tasks. I want to develop an incremental reading app so I'm mainly interested in how to schedule the topics properly. If you don't know what a topic is, here is the description from this article:

topics are pages of new information to learn. They are reviewed passively, i.e. they do not provide a stimulus, do not require a response, and do not expect any feedback from you.

So it's just a passage of text but somehow Supermemo knows when to show it to you.

Also, from the same article:

topics are presented in always increasing intervals. Each new interval equals the old interval multiplied by a constant called A-Factor

This sounds similar to the much older SM-2 algorithm but it doesn't need user's grade to determine the interval.

So the question is: Is there a way for ebisu to function like this? Schedule this type of card that have 0 feedback?

[fasiha]

Hmm, interesting question! I'm not familiar with what theory is being invoked by Supermemo for these topics elements, but after thinking about it for a few minutes, I can't think of anything in the Ebisu modeling framework analogous to topics.

However, as you may have guessed, there's absolutely nothing stopping you from just calling ebisu.updateRecall() with a pretend "success" review: this will grow the half-life just like the second excerpt suggests. (I know this isn't theoretically satisfying but I hope you understand—the core idea in Ebisu is that quizzes are binomial random variables. If you don't have a quiz… I'm afraid this might be the best we can do?)

Let me know if this answers your question or if I've misunderstood!

[c1-g]

Yeah, I think I'll go the pretend review route.

I've misunderstood how the algorithm works. From #42, you wrote that this algorithm doesn't actually schedule anything but just spit out the probability of recall and the developer decides the cutoff for review correct? So I guess I would do something like show the user a topic that have low recall probability?

Also, In Supermemo, each topic can be assigned a "priority" value by the user. There is some info about it here.

During learning, on a given day, elements with highest priorities are processed first.

Can this be incorporated to the algorithm? I saw this paragraph on the README:

Quiz apps that allow a students to indicate initial familiarity (or lack thereof) with a flashcard should modify the initial half-life t.

So I guess high priority topics will have a low initial half-life while low priority topics will a higher value so they don't get reviewed too soon right?

[fasiha]

From #42, you wrote that this algorithm doesn't actually schedule anything but just spit out the probability of recall and the developer decides the cutoff for review correct? So I guess I would do something like show the user a topic that have low recall probability?

Right on. I pick the flashcard in the lowest predicted recall (or something similar; often I like to find the 5% of cards with the lowest recall probability and pick a random card from that set, etc.). Others use modelToPercentileDecay to do the Anki/Supermemo interval scheduling thing.

So I guess high priority topics will have a low initial half-life while low priority topics will a higher value so they don't get reviewed too soon right?

Wow, I am totally out of date on Supermemo and didn't know about its priority system. Hmm! Your proposal could work, but it's not a perfect replacement 😕. Here's why I say that: if you learn two new things that are equally new to you and therefore equally hard to remember, Ebisu would like both flashcards to start with the same model (same numbers, same predicted probability of recall at the starting halflife). If one of those facts is more important than the other, and you give the less important one a higher initial halflife, sure, in practice you will review the less important one less frequently, but you would be underreviewing in some sense (since our assumption was that both were equally hard to remember when you first learned them)? But it might be fine!

This is the kind of functionality that I think might need to live in the app level (one level above Ebisu), because I can't right now think of anything that I can add to the Ebisu model to mimic Supermemo's priority idea. Because, again, Ebisu's core idea is that quizzes are binomial/binary random variables whose underlying probability is Ebbinghaus' exponential decay. And I don't know how to add priority to that model (where some probabilities are more important than others). But I'll keep thinking about it!

So here's another proposal: maybe in the app level, you can assign flashcards to priorities, and then apply some kind of heuristic whereby the predicted recalls for higher-priority cards are reweighted to have lower probability of recall, and therefore get selected sooner…? That way, Ebisu's memory model stays coherent (you don't lie to Ebisu by claiming a low priority card has a higher halflife) but you do differentiate between cards that have the same predictRecall.

As a very rough suggestion, if there were N priorities (I'd try to restrict N<5 in the app?), where priority=1 is the normal and priority=5 is highest, you might try applyPriority = lambda rawProbabilityRecall, priority: rawProbabilityRecall ** priority and see if you like that behavior? (Here, rawProbabilityRecall is the output of ebisu.predictRecal().)

Edit I simplified the example above from my original comment

What do you think? Let me know if something in the above doesn't make sense. The difference between your proposal and mine may be hard to see and maybe I'm overstating its importance, so thanks for bearing with me 😅!

[fasiha]

Minor suggestion: I think a slightly better applyPriority might be

def applyPriority(rawProbabilityRecall: float, priority: int|float) -> float:
  return rawProbabilityRecall / priority

I think what I suggested in my previous comment, rawProbabilityRecall ** priority is too aggressive: that's likely going to keep asking you priority 5 questions over and over again 😅.

Both are just very rough suggestions, from me thinking about this question for only a few minutes, feel free to think outside the box here! Happy to give feedback on any schemes you might think of.

[zxl777]

This problem can be solved at the application level.
Just set a tag for each row.
The user can select the content he wants to review first by a special tag and the lowest recall.

[fasiha]

Closing this but please feel free to reopen or add more comments!

[hobodrifterdavid]

Hello. Today I added some additional info in Language Reactor saved item list, to show the current halflife.

I did some reviews, and immeadiatly noticed an issue. The word review feature (PhrasePump) uses passive reviews heavily (2 passive reviews for every explicit test), using a new example sentence for the word most of the time.

When a word is reviewed in a session, I do an update on the ebisu model for the word once (the user actually sees three example sentences for a word), ebisu.updateRecall() is called with a pretend "success".

The problem: Say I didn't study for one month. An item is shown to me that has halflife 0.5 days, way overdue. ebisu.updateRecall() is called with "success", ebisu say, wow, you're awesome, you remembered this word after one month, I'm going to bump it right up to halflife 8.3 days! But, I probably didn't really remember it.

We're on ebisu 2.0. As a fix on our end, I'm thinking to cap tnow to the items halflife (0.5 days) when doing passive exposure, as if the item was reviewed on-schedule. This results in halflife climbing to only 0.63 days, which seems better.

Any thoughts? :) I'm sending you an email btw.

[fasiha]

Hi @hobodrifterdavid thanks for the note, always happy to hear from Language Reactor!

Say I didn't study for one month. An item is shown to me that has halflife 0.5 days, way overdue. ebisu.updateRecall() is called with "success", ebisu say, wow, you're awesome, you remembered this word after one month, I'm going to bump it right up to halflife 8.3 days! But, I probably didn't really remember it

In this situation, I would not call updateRecall with a fake success, I would just overwrite the "last quiz time" column in your database to the current timestamp 😝 this has the same effect as a totally uninformative quiz (in v2.1, passing in successes=0.5, q0=0.5), i.e., the memory was refreshed right now but we got no information about how strong or weak it was so we just keep the old half-life of 0.5 days.

Does that make sense?

[hobodrifterdavid]

Perfect sense, thanks. I added a special case for "EXPOSURE" type cards. I'll try get around that email soon. :)