Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature request] Add support for the custom study feature recently implemented into Anki (2.1.55) #12620

Closed
Arbitrate3280 opened this issue Oct 11, 2022 · 18 comments

Comments

@Arbitrate3280
Copy link

Arbitrate3280 commented Oct 11, 2022

Hi! I'm wondering if the latest alphas support the CustomData feature recently implemented in Anki.

It's needed to experiment with a new custom scheduler going around lately.

ankitects/anki#2039
ankitects/anki#2040
https://forums.ankiweb.net/t/some-problems-in-implementing-a-state-of-the-art-srs-scheduler-on-anki/22705/5
https://www.reddit.com/r/Anki/comments/x9jjcx/implement_a_new_spaced_repetition_algorithm_based/inpm44b/

@welcome
Copy link

welcome bot commented Oct 11, 2022

Hello! 👋 Thanks for logging this issue. Please remember we are all volunteers here, so some patience may be required before we can get to the issue. Also remember that the fastest way to get resolution on an issue is to propose a change directly, https://github.com/ankidroid/Anki-Android/wiki/Contributing

@dae
Copy link
Contributor

dae commented Oct 11, 2022

The whole custom study feature is not supported yet.

@Arbitrate3280
Copy link
Author

I see, I guess I'll leave this open as a feature request then, thank you!!

@Arbitrate3280 Arbitrate3280 changed the title [Question] Does Ankidroid supports the CustomData feature present on the latest Anki 2.1.55? [Feature request] Add support for the custom study feature recently implemented into Anki (2.1.55) Oct 11, 2022
@BrayanDSO
Copy link
Member

BrayanDSO commented Oct 11, 2022

If I got it right, Anki has a shiny new way to customize the scheduler via JavaScript (https://faqs.ankiweb.net/the-2021-scheduler.html#add-ons-and-custom-scheduling), which works on the v3 scheduler and can be configured on the new deck options screen (#12175). Looks very cool.

With the new backend and deck options screen, is there anything more to change in order to implement this, @dae?

@dae
Copy link
Contributor

dae commented Oct 12, 2022

It will require:

  • loading review_extras.js from the desktop code into the webview
  • switching from the 'card' property in AbstractSched to instead storing the current card state and next states inside the reviewer
  • evaluating the provided javascript after the question is shown
  • displaying the answer buttons based on the stored/mutated state, instead of via the method in AbstractSched
  • answering the card via the stored/mutated state, instead of via the method in AbstractSched

See V3CardInfo in reviewer.py and the code in mediasrv.py for more. The API has slightly changed in 2.1.55, so probably not worth looking into this until that releases.

@github-actions
Copy link
Contributor

Hello 👋, this issue has been opened for more than 2 months with no activity on it. If the issue is still here, please keep in mind that we need community support and help to fix it! Just comment something like still searching for solutions and if you found one, please open a pull request! You have 7 days until this gets closed automatically

@github-actions github-actions bot added the Stale label Dec 11, 2022
@Arbitrate3280
Copy link
Author

go away bot

@L-M-Sherlock
Copy link

Two months on, Anki for PC 2.1.55 stable has been released and I wonder if this feature will be supported in AnkiDroid 2.16?

@galantra
Copy link

open-spaced-repetition/fsrs4anki#107 (comment)

So, I can tell you. customData is a field specific to scheduler v3. We don’t support it. And sadly, we don’t even have an idea of when we’ll actually support it, because Android Play store does not allow us to update the app until we make some complex changes.

Upon reading this, I was not sure whether

  1. there is a technical obstacle in Play Store that prevents the use of customData, or
  2. there are more pressing issues than customData (, which are related to the Play Store).

If the technical obstacle is the case, perhaps customData could already be included in Github and/or F-Droid (pre-)releases.

I have searched through the issues. Probably this is meant? #9259

@BrayanDSO
Copy link
Member

BrayanDSO commented Dec 23, 2022

In a nutshell,

@Arthur-Milchior
Copy link
Member

@L-M-Sherlock I don’t know how much you can infer from Brayan’s statement regarding your specific question.
No, it quite probably won’t be available in 2.16. That is not a priority. The only thing that is needed for 2.16 is the storage permission. Once we have it and are confident in it, we release it.
All contributors are volunteers, and one can decide that it is their priority. But as far as I know, nobody decided to start working on it. But I’d be surprised if it is deemed stable enough that we are confident to release it outside of the advanced settings.

@mikehardy
Copy link
Member

Not disagreeing with anything from Brayan or Arthur - on the contrary their statements are all correct - just chiming in to say I would welcome this as a contribution and all of our infrastructure work to support 2.1.55 desktop release + to support javascript programming APIs etc etc is all aimed at making things like this possible. But it will need someone with motivation to adopt it as a feature they will work on (as Arthur mentioned - likely behind an advanced toggle at first, or something similar that makes sure it won't destabilize the non-custom-scheduling code paths)

@doctorman
Copy link

Not disagreeing with anything from Brayan or Arthur - on the contrary their statements are all correct - just chiming in to say I would welcome this as a contribution and all of our infrastructure work to support 2.1.55 desktop release + to support javascript programming APIs etc etc is all aimed at making things like this possible. But it will need someone with motivation to adopt it as a feature they will work on (as Arthur mentioned - likely behind an advanced toggle at first, or something similar that makes sure it won't destabilize the non-custom-scheduling code paths)

Do you recomend enabling the v3 scheduling on desktop or does it cause issue with the current play store version or even with the alpha builds? as a beta option which alpha is the most stable? the latest one?

@Ceynou
Copy link

Ceynou commented Dec 31, 2022

Not sure if I understood correctly but if customData is ever supported it would be nice to have it start running before rendering the card so the template can depend on customData value, I have this variable and the card display one thing or the other depending on its value

globalThis.intervalTest = states.current.normal.review.elapsedDays || 0;

It might already be working correctly but the template is displaying both f1 and f2

<script>
  qFade = 0; //make it so the card appears immediately instead of fading in
	f1 = document.getElementById("field1");
	f2 = document.getElementById("field2");
	if (globalThis.intervalTest < 10){
		f1.style.display = "None";
	} else {
		f2.style.display = "None";
	}
</script>

<div id="field1">
thing one
</div>

<div id="field2">
thing two
</div>

@L-M-Sherlock
Copy link

L-M-Sherlock commented Jul 27, 2023

Congratulations for the release of 2.16! I notice that the change log said that "We include anki 2.1.61 now, we’re fully up to date!" Does it mean CustomData feature has been implemented in 2.16?

https://ankidroid.org/docs/changelog.html#_version_2_16_2_20230726

@user1823
Copy link
Contributor

Does it mean CustomData feature has been implemented in 2.16?

No. The custom scheduling feature is not implemented yet. I think that they plan to release it with 2.17.

"We include anki 2.1.61 now, we’re fully up to date!"

In my opinion, this part of the changelog is misleading. There are several features where AnkiDroid lags behind Anki.

dae added a commit to ankitects/Anki-Android that referenced this issue Aug 8, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 8, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 8, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 19, 2023
mikehardy pushed a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
mikehardy pushed a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
mikehardy pushed a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 24, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 26, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 26, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 26, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 27, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 27, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 27, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 27, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 27, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 27, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 28, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 28, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 28, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 28, 2023
Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 28, 2023
dae added a commit to ankitects/Anki-Android that referenced this issue Aug 28, 2023
n1snt pushed a commit to n1snt/Anki-Android that referenced this issue Sep 7, 2023
* Update target API to 23

* Update to 2.1.66, and switch to new schema by default

+ Temporarily disable unused resource warnings

* Drop legacy syncing code

* Drop legacy import/export code

* Drop legacy tag handling

* Drop legacy config handling

* Drop legacy database check

* Stub out unused v1 scheduler code

Can't be fully removed, as we still need to be able to open a v1 collection
so the user can upgrade.

moveVersions test has been removed, as backend code is used for moving
outside of unit tests.

* Drop legacy deck handling

* Drop legacy search code

* Drop legacy notetype code

This breaks the card template editor, so I've temporarily disabled it
in the GUI. Getting it working again will require switching to the new
template rendering code in the new backend.

Also breaks an "empty cards" action in CardContentProvider,
as I was not sure what it was trying to accomplish.

* Drop legacy media code

This removes oakkitten's custom media checking code, as keeping it
would require keeping a bunch of the duplicated Kotlin logic. His comment
about cancellation is incorrect: the call can be cancelled with backend.
setWantsAbort() on a background thread. And since the app enforces filenames
are normalized when they're added anyway, I don't think avoiding automatic
normalization is worth the duplicated code.

* Drop legacy Collection/DB code

* Drop most references to defaultLegacySchema

* Remove legacy schema tests from CI

CI won't work yet, because backend version needs updating

* Fix import CSV call

* Remove some unused symbols from anki module

* Drop majority of old stats code

* Remove unused symbols from libanki

* Remove some broken Android tests, and fix one

checkIfStudyOptionsIsDisplayedOnTablet() is also consistently failing,
but was doing so before I started on these changes.

* Move config methods into col.config

* Simplify config API

- Use kotlinx.serialization so that arbitrary typed objects can be
(de)serialized
- There is a single generic get(), that always returns an optional
value, which will be null if the key is missing, is null, or is not the
correct type. There will always be collections that have invalid data,
so the calling code always needs to be prepared to substitute a reasonable
default in such cases.
- Expose the typed getBool() method from the backend.

* Remove unused legacy importer

* Drop jackson

* Models -> Notetypes

* Remove some more unused symbols

* Remove generic type aliases

* Rename Model -> Notetype

I had to leave ModelTest.kt's filename alone, as changing it to Notetype.kt
reorders the unit tests, and causes about 4 tests to start flaking.

* Get card template editor preview working again

* Enable undo on edits; fix test hang

* Use backend for bulk suspend/undo

By default, undoableOp() notifies all screens listening for opExecuted,
so the refreshing happens automatically, and the manual refresh code
at the end of the old routine is no longer required. It is possible to
bypass this when you want to manually update the UI, but this is probably
not worth attempting until the card browser is either switched over to
a recycler view, or reimplemented as a web component.

The LongArrays have been switched to simple lists, as Google's protobuf
code expects Iterable<Long>

* Use backend for marking/undo, and change deck/undo

* Migrate reviewer actions/undo to backend

* Migrate remainder of undo code to backend, apart from v2 undo

Couple of notes:

- The legacy v2 undo no longer returns the undone card, and instead
rebuilds the queue. In some circumstances (eg a learning card has become
due), this can result in a different card being shown after undoing.
This is not ideal, but v2 does not have long to live at this point.
- The reset/reposition tests were reusing the old card cache values,
which have to be reloaded after an operation.
- A test in AbstractSched had to be moved to ReviewerTest so it can have
access to an activity.
- UndoTest has been removed, as it mainly focused on removed code.

* GetCard() -> getNextCardAndRedraw()

* render_output -> renderOutput()

* Drop AsyncTask

Probably the hardest part of AnkiDroid's code base to follow; very
glad to see the end of it.

Closes ankidroid#7108

* Make note addition undoable

* Make bulk tag update undoable

* Remove remaining explicit transaction handling

The backend automatically wraps backend actions in a transaction,
and removing the explicit calls will allow us to drop the redundant
mutex that rsdroid acquires.

* Remove remaining unused classes/methods/properties

* Remove FunctionalInterfaces

* Switch to backend answer comparison

* Remove unused locking code + legacy storage code

* Switch ContentProviderTest to new schema, and fix bug

* Remove legacy Deck Options settings

* fix: remove minSdkVersion < 23 code

* Use short time in snackbars

People complained about snackbars interrupting their reviews before

* fix: don't duplicate undo label

It were showing a message like `Undo Undo Add Note`

* Remove "Advanced statistics"

* Remove legacy_schema local property

* Leave zip validation up to the backend

The backend takes care of validating the zip file and ensuring files aren't
written outside of the media folder.

I've expanded the mime match to include the zip mime, on the assumption
that's what the mime type is being changed to on some devices. If that
proves to be insufficient, a much simpler approach would be to look
for 50 4B 03 04 at the start of the file to determine if it's a zip file
or not.

This leaves only Mani's add-on code using commons-compression, so you
could potentially switch to a simpler tgz-specific library in the future
if that is easier.

* Disable two flaky tests

They started to flaky reliably after the legacy deck options test
was removed.

* Improve logging of HTTP requests

* Remove ModelBrowser

Replaced by ManageNotetypes

* Remove legacy CardInfo

replaced by com.ichi2.anki.pages.CardInfo

* Change comments at AdvancedSettingsFragment

* test: check if prefs analytics list don't have extra elements

* ContextCompat.getColor -> getColor

that compat method is for implementations in API < 23

* Remove more unused files

* refactor: move SECONDS_PER_DAY to a separate util file

in order to be able to remove Stats.kt later

* refactor: remove Stats and OverviewStatsBuilder

* Access sched.card directly in tests

The helper was adding a 500ms sleep on every fetch, which slowed the
scheduler tests down considerably.

* Make v3 scheduler the default; drop support for v2

The v3 scheduler was originally released in 2021, and we've been waiting
for a stable AnkiDroid release to support it before we could switch users
over to it. Now that 2.16 is out, we can finally push v3 out across the
ecosystem.

While we could theoretically make v3 the default without removing v2,
here are the reasons why I think we're better off switching to v3 only:

- AnkiWeb's review interface will likely switch over to v3-only in the
coming weeks, and AnkiMobile will likely drop v2 around the same time
2.17 comes out.
- v2 and v3 differ in a few ways that makes maintaining the two separate
paths more complicated: things like the different undo paths, counts not
including the current card, and the way the v3 scheduler supports custom
scheduler js. 2.17 is a chance to clean up a lot of old cruft in the
code base, and the old scheduling code is part of that.
- v2 and v3 are compatible with each other, and don't require a full sync
to change, so it doesn't break syncing with old clients (though depending
on settings, due counts may differ, which needs explanation)

I had to disable a couple of tests for this that we'll probably want
to restore in some form:

- corruptVersion16CollectionShowsDatabaseError() needs an update, and
it may be time to rip out the old CollectionHelper colIsOpen(), getColSafe()
and so on, migrating code that uses it over to withCol() instead. The
'collection inaccessible' dialog also needs a rethink - perhaps the various
exceptions could be handled in launchCatchingTask instead.
- testUndoResetsCardCountsToCorrectValue() is failing because initLayout()
creates a gesture listener, which fails with an error about the looper
not being initialized. I am not sure what's going on there - one option
would be to move the test to androidTest.

* Drop v2 sched file; rename files

* Drop AbstractSched and legacy undo code/queue code

* Migrate BaseSched into Scheduler

* More unused symbol removal

* Remove DeckTreeNode

* Remove unused processChildren()

* Remove shouldDisplayCounts(), as counts always available

* Remove manual hashCode()/compareTo() implementations

* Drop AbstractDeckTreeNode

* Turn DeckNode into a wrapper for DeckTreeNode

* Drop TreeNode and simplify filtering

* Remove `New card position` global preference

Overridden by per-deck configuration

Closes ankidroid#12319

* fix: remove chess.css from card_template_html

* Store current queue state in reviewer

Prerequisite for solving ankidroid#12620

Also dropped answerButtons(), as it's always 4

* Use local HTTP server for serving flashcard content

Prerequisite for solving ankidroid#12620

* Implement support for custom JS scheduling

Enables FSRS and closes ankidroid#12620

* Remove more unused code

* Remove separate Deck(Config)V16 objects and more unused code

* Move some NotetypeJson methods into its file

* Finish tidying Decks.kt

* Drop redundant col property on Collection

* Remove some usages of CollectionGetter

* Rename col->getColUnsafe to encourage migration

* Remove Reviewer.sched

* Handle deck tree not initialized in empty collection

* refactor: move syncStatus() to Collection

to avoid direct calls to the backend

* fix: do not show error when updating the menu if there is no internet

Reproduction steps:
1. have the collection synced
2. disable the device internet connection
3. Restart the app

* Remove unused resources and re-enable lint

* Restore sync cancel strings

* Defer loading webpage until server has initialized

* Ignore translations without a base entry

ankidroid#14171 (comment)

* Remove the SortOrder deprecation

* Remove the apparently-unused Backup.kt

* Remove last remaining use of Collection's context field

* Stop passing Context to backend

The library loading is now AnkiDroid's responsibility, so the backend
does not require access to an Android context.

Also remove the unused context field from Collection.

* Use backend for rendering next time labels

* Use backend for rendering finished message

The custom study & unbury descriptions are slightly different, but
should suffice until AnkiDroid can start using the Svelte finished
screen.

* Run SchedulerTest without Robolectric

Halves the run time. To do this, I implemented a new JvmTest class
and copied some of the helpers in RobolectricTest into it.

* Remove unused Kotlin implementation of template parsing

* Move Util code dealing with resources out of libanki

* Migrate remaining libanki tests to raw JVM

+ Remove some commented-out tests

Total savings are a drop from about 3m40s to 2m40s, with the scheduler
test class dropping its runtime by about half.

* Lazy-initialize col in tests

* Restore 'experimental' string

* Add comments to update-localizations.py

* Restore AcraAnalyticsInteraction.kt

---------

Co-authored-by: Brayan Oliveira <69634269+brayandso@users.noreply.github.com>
@xificurC
Copy link

The commit closing this says "enables FSRS". I'm lost in the details here.

  • this issue is about a customdata feature, which IIUC allows using custom SRS schedulers in anki
  • FSRS is a new open source SRS algorithm
  • there are FSRS anki addons one can use on desktop

If one wants to test the FSRS algorithm in AnkiDroid, where do we stand on this? Or is FSRS included in the latest release?

@Arbitrate3280
Copy link
Author

It has been introduced in this commit: #14171

No new release has been made yet that contains this change. The next time a release is made, it will have it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Development

No branches or pull requests