-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Algorithm to select next best item for review (Ebisu for dummies?) #57
Comments
I like the idea to stop testing old facts depending on its probability of recall. This could help calculate the number of facts you still have to test today, and not have to review indefinitely. In regards to your last question, I think that facts can be forgotten in different speeds, so even if recall probability of fact A is lower than the recall probability of fact B, on a future time this might not be the case because of the different speeds of forgetting. One fact might be "weaker" and will decay more rapidly than the other. If I remember correctly, Fasiha was working on something to address this change in speed of forgetting/learning, which he was calling acceleration. I might be forgetting exactly what he wrote, but I think it was about multiplying the half-life by some factor after a recall. |
Oh, so I see since this forgetting is not linear, the only way is to define what is best to review in every moment is to call
I'm lazy, and don't like the word "have to". :) I would prefer to have an app that I could open at any time for 5 minutes (waiting in line, sitting on a toilet, etc..), and it will just give the next best thing to review. Something like Facebook algorithm, but actually useful. So not for daily reviews, but for reviews multiple times per day, on random moments. |
Thank you @bunyk for considering Ebisu for building your app, that has always been my dream with Ebisu—to make it easy to create your perfect quiz app 😄
So first I'll say this: my goal is to make a library that anybody can use, without needing a Master's degree in statistics 😅! The mathematical details in the README are mostly for folks who want to improve the algorithm—you don't need to understand this to use the Python/JavaScript/etc. library in your app. However, if you're really very very motivated to learn this, I think an introductory course on probability and Bayesian inference should be all you need to check my derivation of the posterior.
I'm sorry that unfortunately we don't have a good way to teach/learn mathematics the way we have excellent resources for programming 😢.
This is what I do. (I also like to gently randomize the flashcard to review, so instead of the card with the absolute lowest
Some people do this. I don't really like apps that do this though, because some days I don't want to review (just learn), other days I don't want to learn (just review), so I prefer apps that let me pick whether to learn or to review. That also simplifies the app so I don't need a numerical threshold—the threshold for switching from reviewing to learning is my boredom 😃.
As @MNastri mentioned (thank you for weighing in!), because cards decay at differing rates, you have to be quite careful doing this. But like they alluded to, I am working on v3 of the algorithm that changes a lot of how the statistics works (see #43 for the details). This will have a much simpler I was hoping to publish v3 six months ago but got very busy. I'm hoping to release it in the next few months 🙏.
I also love this!!! I have apps that rerun Please feel free to post any more questions or comments if any of the above is unclear. Thanks for writing! |
I'm developing a language learning app for iOS/Android, and both my users and I want to have an SRS system to really master the new vocabulary that's being added. I discovered your Ebisu. Intuition tells me that this algorithm is alive and well, despite your intimidating mathematical arguments 😄 I am going to embed the python program directly into my server, and I am also looking forward to the 3.0 update. Encourage you! Good Job! |
Closing this for now, please feel free to reopen or add further questions/comments! |
I'm finding https://fasiha.github.io/ebisu/ very fascinating, thanks for creating it. I'm reading your documentation every time the idea to build my own Memrise/Duolingo clone struck me again, but one obstacle is that it's a little bit math heavy. So I decided to ask for some explanations here. (Also, if you could recommend some prerequisite statistics books or online courses that would help with better understanding math behind all this - I'll be very grateful)
What I understood is that for each fact to learn, you store parameters, that allow computing what is the probability of forgetting (or 1 - recall probability) that fact at a particular moment of time and confidence in that probability. This probability of recall is computed by
ebisu.predictRecall()
.And then when fact is tested,
ebisu.updateRecall
updates parameters to take into account the results of that test on future recall probabilities.Question is, given a set of facts, how do I select one that needs to be tested in the current moment of time? Call
predictRecall
for each, and select ones with the lovest recall? And then have some threshold, like "if recall probability is > 80%", learn new facts instead now, as reviewing that ones will be not so efficient?Could facts be sorted just by some value calculated when the model parameters are updated, so you should more efficiently select them, for example from SQL DB? Because I imagine that if fact A had lower recall probability today than fact B, then probably the same should be true tomorrow, right?
The text was updated successfully, but these errors were encountered: