Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add deck and note options. #12

Open
wants to merge 15 commits into
base: main
Choose a base branch
from
Open

Add deck and note options. #12

Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
28766a3
Add deck options via new OptionsGroup class.
holocronweaver Aug 17, 2017
eab7246
Add per-note options and a deck creation time.
holocronweaver Aug 18, 2017
685d962
Ensure deck creation time is an integer.
holocronweaver Aug 18, 2017
ebacc3c
Fix a few typos and a minor interface inconsistency.
holocronweaver Aug 18, 2017
cd261da
Rename note.level -> stage to reduce ambiguity.
holocronweaver Aug 18, 2017
6eb44fa
Minor coding style improvement.
holocronweaver Aug 19, 2017
96606ec
Document deck and note options.
holocronweaver Aug 24, 2017
0fa1870
Internally document note and deck options further.
holocronweaver Aug 24, 2017
a5a8d00
Fix a couple field name typos in apgk_col.
holocronweaver Aug 24, 2017
4d0449a
Update tests to create OptionGroups.
holocronweaver Aug 24, 2017
61b18b5
Add Card.status to allow suspending and burying cards.
holocronweaver Aug 28, 2017
bda7097
Move note options to CardOptions and try to match database field names.
holocronweaver Sep 1, 2017
8286fbc
Merge 'upstream/master' into options.
holocronweaver Sep 1, 2017
0a13191
Remove CardOptions and set options directly on Card or via Note.
holocronweaver Sep 2, 2017
d9fd190
Avoid relative commits in __init__.py
holocronweaver Sep 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
29 changes: 29 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,35 @@ interface. Anki also is happier if you avoid having two notes with the same `sor
necessary. By default, the `sort_field` is the first field, but you can change it by passing `sort_field=` to `Note()`
or implementing `sort_field` as a property in a subclass (similar to `guid`).

## Deck and Note Options
`Deck`s and `Note`s have options, many of which influence the Anki SRS algorithm. These can change when and how often cards are due, how leech cards are handled, how cards transition between SRS stages, and more.

Deck options are primarily provided through the `OptionGroup` which is closely modeled after the Anki deck options window. In addition, the `Deck` has `description` and `creation_time` attributes. Note that Anki defines card due dates relative to deck creation time.

``` python
options = genanki.OptionsGroup()
options.autoplay_audio = False
options.new_cards_per_day = 10
options.max_reviews_per_day = 200
options.review_bury_related_cards = False
options.interval_modifier = 0.85
my_deck = genanki.Deck(
2059400110,
'Country Capitals',
options=options)
my_deck.description = r'The capitals of the 100 most populous countries. \nCreated on {}.'.format(deck.creation_time.date().isoformat())
```

`Note` options typically vary significantly across notes and are thus set directly in the note.

``` python
my_note.stage = 1 # SRS learning stage: 0 = new, 1 = learning, 2 = review.
my_note.interval = 20 # Days between next review and the one following.
```

Decks and notes have more options which are documented in `genanki/__init__.py`. For full details on their meaning and relation to the Anki SRS algorithm, please see the [official Anki documentation](https://apps.ankiweb.net/docs/manual.html#what-spaced-repetition-algorithm-does-anki-use).


## YAML for Templates (and Fields)
You can create your template definitions in the YAML format and pass them as a `str` to `Model()`. You can also do this
for fields.
143 changes: 121 additions & 22 deletions genanki/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
from cached_property import cached_property
from copy import copy
import json
from datetime import datetime
import hashlib
import json
import os
import pystache
import sqlite3
Expand Down Expand Up @@ -155,25 +156,31 @@ class Card:
def __init__(self, ord_):
self.ord = ord_

def write_to_db(self, cursor, now_ts, deck_id, note_id):
self.interval = 0

def write_to_db(self, cursor, now_ts, deck_id, note_id,
stage, queue, due, interval, ease, reps_til_grad):
cursor.execute('INSERT INTO cards VALUES(null,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?);', (
note_id, # nid
deck_id, # did
self.ord, # ord
now_ts, # mod
-1, # usn
0, # type (=0 for non-Cloze)
0, # queue
0, # due
0, # ivl
0, # factor
0, # reps
0, # lapses
0, # left
0, # odue
0, # odid
0, # flags
"", # data
note_id, # nid - note ID
deck_id, # did - deck ID
self.ord, # ord - which card template it corresponds to
now_ts, # mod - modification time as seconds since Unix epoch
-1, # usn - value of -1 indicates need to push to server
stage, # type - 0=new, 1=learning, 2=review
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be called type. In general, we should always use the same variable naming that Anki does. If Anki's naming is confusing (which it often is), detailed comments will help make the field's purpose clear to the user.

Ditto for all the other properties here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean just within this function, or in the user-facing API as well?
If the latter, I think of this as 'spreading the disease', where one unintuitive interface gives rise to another, but I can understand wanting a close correspondence for debugging. I doubt many users will be familiar with the Anki DB column names, so I don't believe it will make the lib easier to learn. So...I'll make the change, but I'll feel dirty, lol.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I remember now why I didn't use type for a variable name: it's a Python reserved keyword. Perhaps we should make an exception in this case?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's better to closely match Anki's naming. I want genanki to be a lower-level library that more-or-less directly maps to the way Anki represents data. Plus, the user will probably have to end up learning the Anki names anyway, for example if they're digging into the collections.anki2 SQLite file to debug something. Abstractions are leaky more often than people expect.

I's OK to use type as the name. If you need to access the real type() function, you can import the builtins module and call builtins.type. (Previously I also had an instance of this, calling a param ord_ instead of ord. But I changed it because it's silly to convolute your external API for the convenience of internal code, and I wasn't using ord() anyway).

queue, # queue - same as type, but
# -1=suspended, -2=user buried, -3=sched buried
due, # due - new: unused
# learning: due time as integer seconds since Unix epoch
# review: integer days relative to deck creation
interval, # ivl - positive days, negative seconds
ease, # factor - integer ease factor used by SRS, 2500 = 250%
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think all these properties should be attributes of the Card object, not the Note object, since you can set them per-card.

So the user would do something like this:

my_note = Note(...)
...
my_note.cards[0].queue = -1

If you have a use case for setting attributes on all cards, you can add a helper function like this:

def set_attributes_on_call_cards(self, **kwargs):
    for card in self.cards:
        for k, v in kwargs.items():
             setattr(card, k, v)

Note: you can skip adding ease, ivl, etc as keyword arguments to Card.__init__. Unlike Note and Deck, the user generally doesn't directly initialize Card instances; instead, they're implicitly initialized when the user creates the Note. So it doesn't make sense to set Card attributes via keyword arguments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okydoke! I actually planned for and forgot to do this, heh.

0, # reps - number of reviews
0, # lapses - # times card went from "answered correctly" to "answered incorrectly"
reps_til_grad, # left - reps left until graduation
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto here. This should be called left. If you want to, you can add a @property that aliases reps_til_grad to left. But the main name of the property should be left.

0, # odue - only used when card is in filtered deck
0, # odid - only used when card is in filtered deck
0, # flags - currently unused
"", # data - currently unused
))


Expand All @@ -189,6 +196,29 @@ def __init__(self, model=None, fields=None, sort_field=None, tags=None, guid=Non
# guid was defined as a property
pass

## Options ##
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per above comment, please move this out of the Note class and into the Card class.


"""SRS learning stage.
0 = new, 1 = learning, 2 = review."""
self.stage = 0
"""SRS queue status modifiers.
0 = normal, 1 = suspended, 2 = user buried, 3 = scheduler buried"""
self.status = 0
"""Behavior depends on learning stage of note.
new: unused.
learning: due time as integer seconds since Unix epoch.
review: integer days relative to deck creation timestamp."""
self.due = 0
"""Time between next review and the one following.
Positive values are in days, negative in seconds."""
self.interval = 0
"""Integer 'ease' factor used by SRS algorithm.
Example: 2500 = 250%."""
self.ease = 0
"""Repititions remaining until graduation from the learning stage.
Unused during other SRS stages."""
self.reps_til_grad = 0

@property
def sort_field(self):
return self._sort_field or self.fields[0]
Expand Down Expand Up @@ -233,8 +263,11 @@ def write_to_db(self, cursor, now_ts, deck_id):
))

note_id = cursor.lastrowid
queue = -self.status if self.status else self.stage
for card in self.cards:
card.write_to_db(cursor, now_ts, deck_id, note_id)
card.write_to_db(cursor, now_ts, deck_id, note_id,
self.stage, queue, self.due, self.interval,
self.ease, self.reps_til_grad)

def _format_fields(self):
return '\x1f'.join(self.fields)
Expand All @@ -243,12 +276,67 @@ def _format_tags(self):
return ' ' + ' '.join(self.tags) + ' '


class OptionsGroup:
def __init__(self, options_id=None, name=None):
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to follow the pattern of allowing all arguments to be set via keyword arguments to __init__.

So you should write something like

def __init__(
    self,
    options_id=None,
    options_group_name=None,
    max_time_per_answer=60,
    ...
    misc=0):
  self.options_id = options_id
  self.options_group_name = options_group_name
  ...

I realize this is extremely verbose, so you could also go this alternate route:

class OptionsGroup:
  ATTRIBUTES_WITH_DEFAULTS = {
    'options_id': None,
    'options_group_name': None,
    'max_time_per_answer': 60,
    ...
  }

  def __init__(self, **kwargs):
    for attr, default in self.ATTRIBUTES_WITH_DEFAULTS.items():
      setattr(self, attr, kwargs.get(attr, default))

Either style is fine. However, the first style is advantageous in that it's easier for tools like IDEs to introspect. And it's pretty easy to automatically generate the code so it ends up being about the same amount of work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do!

self.options_id = options_id
self.options_group_name = name
# Organized according to options window tabs in Anki.
## General ##
self.max_time_per_answer = 60 # minutes
self.show_timer = False
self.autoplay_audio = True
self.replay_audio_for_answer = True
## New Cards ##
self.new_steps = [1, 10] # list of minute intervals per learning stage
self.order = 1 # option selected in dropdown (0 = first, 1 = second)
self.new_cards_per_day = 20 # days
self.graduating_interval = 1 # days
self.easy_interval = 4 # days
self.starting_ease = 2500 # 2500 = 250%
self.bury_related_new_cards = True
## Reviews ##
self.max_reviews_per_day = 100
self.easy_bonus = 1.3
self.interval_modifier = 1.0
self.max_interval = 36500 # days
self.bury_related_review_cards = True
## Lapses ##
self.lapse_steps = [10]
self.leech_interval_multiplier = 0
self.lapse_min_interval = 1
self.leech_threshold = 8
self.leech_action = 0

# Used for adding arbitrary options via JSON string. Useful for
# addons.
self.misc = ''

def validate(self):
if self.misc and self.misc[-1] != ',':
self.misc += ','

def _format_fields(self):
self.validate()
fields = {}
for key, value in self.__dict__.items():
if key.startswith('__') or callable(key):
continue
if type(value) is bool:
fields[key] = str(value).lower()
else:
fields[key] = str(value)
return fields


class Deck:
def __init__(self, deck_id=None, name=None):
def __init__(self, deck_id=None, name=None, options=None):
self.deck_id = deck_id
self.name = name
self.description = ''
self.creation_time = datetime.now()
self.notes = []
self.models = {} # map of model id to model
self.options = options or OptionsGroup()

def add_note(self, note):
self.notes.append(note)
Expand All @@ -261,7 +349,18 @@ def write_to_db(self, cursor, now_ts):
self.add_model(note.model)
models = {model.model_id: model.to_json(now_ts, self.deck_id) for model in self.models.values()}

cursor.execute(APKG_COL, [self.name, self.deck_id, json.dumps(models)])
params = self.options._format_fields()

params.update({
'creation_time': int(self.creation_time.timestamp()),
'modification_time': int(self.creation_time.timestamp()) * 1000,
'name': self.name,
'deck_id': self.deck_id,
'models': json.dumps(models),
'description': self.description,
})

cursor.execute(APKG_COL, params)

for note in self.notes:
note.write_to_db(cursor, now_ts, self.deck_id)
Expand Down
89 changes: 42 additions & 47 deletions genanki/apkg_col.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
APKG_COL = r'''
INSERT INTO col VALUES(
null,
1411124400,
1425279151694,
1425279151690,
:creation_time,
:modification_time,
:modification_time,
11,
0,
0,
Expand All @@ -15,7 +15,7 @@
"addToCur": true,
"collapseTime": 1200,
"curDeck": 1,
"curModel": "1425279151691",
"curModel": "' || :modification_time || '",
"dueCounts": true,
"estTimes": true,
"newBury": true,
Expand All @@ -25,7 +25,7 @@
"sortType": "noteFld",
"timeLim": 0
}',
?3,
:models,
'{
"1": {
"collapsed": false,
Expand Down Expand Up @@ -55,78 +55,73 @@
],
"usn": 0
},
"' || ?2 || '": {
"' || :deck_id || '": {
"collapsed": false,
"conf": 1,
"desc": "",
"conf": ' || :options_id || ',
"desc": "' || :description || '",
"dyn": 0,
"extendNew": 0,
"extendNew": 10,
"extendRev": 50,
"id": ' || ?2 || ',
"id": ' || :deck_id || ',
"lrnToday": [
163,
2
5,
0
],
"mod": 1425278051,
"name": "' || ?1 || '",
"name": "' || :name || '",
"newToday": [
163,
2
5,
0
],
"revToday": [
163,
5,
0
],
"timeToday": [
163,
23598
5,
0
],
"usn": -1
}
}',
'{
"1": {
"autoplay": true,
"id": 1,
"' || :options_id || '": {
"id": ' || :options_id || ',
"autoplay": ' || :autoplay_audio || ',
"lapse": {
"delays": [
10
],
"leechAction": 0,
"leechFails": 8,
"minInt": 1,
"mult": 0
"delays": ' || :lapse_steps || ',
"leechAction": ' || :leech_action || ',
"leechFails": ' || :leech_threshold || ',
"minInt": ' || :lapse_min_interval || ',
"mult": ' || :leech_interval_multiplier || '
},
"maxTaken": 60,
"maxTaken": ' || :max_time_per_answer || ',
"mod": 0,
"name": "Default",
"name": "' || :options_group_name || '",
"new": {
"bury": true,
"delays": [
1,
10
],
"initialFactor": 2500,
"bury": ' || :bury_related_new_cards || ',
"delays": ' || :new_steps || ',
"initialFactor": ' || :starting_ease || ',
"ints": [
1,
4,
' || :graduating_interval || ',
' || :easy_interval || ',
7
],
"order": 1,
"perDay": 20,
"order": ' || :order || ',
"perDay": ' || :new_cards_per_day || ',
"separate": true
},
"replayq": true,
"replayq": ' || :replay_audio_for_answer || ',
"rev": {
"bury": true,
"ease4": 1.3,
"bury": ' || :bury_related_review_cards || ',
"ease4": ' || :easy_bonus || ',
"fuzz": 0.05,
"ivlFct": 1,
"maxIvl": 36500,
"ivlFct": ' || :interval_modifier || ',
"maxIvl": ' || :max_interval || ',
"minSpace": 1,
"perDay": 100
"perDay": ' || :max_reviews_per_day || '
},
"timer": 0,
"timer": ' || :show_timer || ',
"usn": 0
}
}',
Expand Down