v0.3.2

Changes/Fixes

This release note is gonna be an essay. I just need a place to vent.

How do you count your total reading time, seriously?

In v0.2.6/7, we gained a toggle to switch between the stats from read directly from the device or the ones the app computed itself. The motivation was from #11, which showed the device stats can be seriously undercounted. A "fix" I introduced is to use the difference between the book opened and closed, within an Awake/Sleep window. It turns out to overcount time quite often.

So what we have until now is a toggle for either undercounting or overcounting your reading time.

Thinking about this sent me into a rabbit hole of what should be counted at all, and we need to know what Amazon counts and does not count and why first. It turns out that Amazon has some opinionated (but reasonable) criteria for timekeeping:

First of all, reading speed needs to be within the range of [40, 900] Words per Minute (WPM). Being too fast or too slow, you are not really reading, you are skimming, or you are idling. Both are reasonable, for English, maybe also for German (not really sure about others, but I guess they will be close).

An average proficient English reader reads 250 WPM, 900 is in fact very generous. Infinite Jest has a total word count around 577k, read at 900 WPM, you can finish it in less than 11 hours. The audiobook version takes ~65 hours, which translates into 150 WPM. Reading English or other western languages, you will likely never be undercounted.

But what if you read CJK? I haven't found any recent papers on this topic with definite figures yet (likely not looking hard enough, or they are not published in English...). According to one older paper (Trauzettel-Klosinski, 2012), the speed of reading Japanese and Chinese is actually not much different from other languages if counting by syllables per minute. According to an even older paper (Sun, 1985), the average Chinese reading speed can be ~580 characters per minute (CPM). According to this paper (Miyata, 2012), trained Japanese readers reach 2600 CPM on average, even untrained readers can do 1200 CPM.

Which is equivalent to what WPM, btw? Well, it doesn't really matter. Because your Kindle treats CPM as WPM. What's wrong is the unit. Your Kindle counts every character as a word for CJK. If we give the same generous factor to Chinese and Japanese (900/250=3.6), the upper bound should be at least 2088 for Chinese and 4320 for Japanese. But no, you got 900. The time of every page you read faster than 900 WPM will not be counted at all, so is the word counts, and the WPM will be capped at 900, not your actually WPM, and this will further affect the remaing time estimate for a book.

Having such a bound is actually not a bad idea, I have to admit. At least the lower bound is good to have. If we don't have it, we will definitely overcount (like what I did in v0.2.7). Sometimes reading will be interrupted, maybe you want a tea, or go to the toilet, or just need to pause and think about what you just read, and the Kindle might just stay opened the whole time, and you will not want to count those time in (or do you?). There's no sensor on Kindle for detecting this kind of interruption, and if it had, I probably don't want to use such privacy invasive device...

The point is, having a lower bound is good, having a upper bound is not. For english, roughly 4 mins of idleness will discard the time of that page, for CJK, 7 mins. So again CJK will be overcounted a bit here, but compare to throwing all the time that you read but read too fast, this definitely will happen less often and have less severe effect.

So what we do in this release is to count how much time were discarded due to crossing the upper bound and add them back to the total count, but leave the ones below the lower bound uncounted.

This note is long enough already, but I am not done yet.

There are two more (albeit minor) rules Amazon used to throw away your reading time:

1. The last page of a sitting is not counted. It won't nudge the count a lot for one sitting, but over a longer period time (mine, roughly over 2 months and 100 books and 556 sittings), it can amount to 2.14 hours... I guess the reason is the same as the WPM lower bound, that you will likely not in your normal speed. This is negotiable, but in this release I added it back to the count too.

2. If you ever turn back a page, for whatever reasons, the page that you turned back from will not be counted. That's 1.74 hours of mine. I actually like flipping back and forth. Those are added back too.

Ultimately, the question is what should be counted as reading. Amazon has some opinions, and I have mine, and they might not be the same as yours.

(The end of venting)

P.S. If you ever update to this release, you will need to tap the "Heal" button in the config page to correct some stale data for the last 30 days for the App Stats. Those days the backlogs no longer hold will stay stale forever and be either under- or overcounted.

Others

But still not the end of this release note yet.

There are other improvements but I won't go into much details (this release note is long enough and I don't think anyone will read it):

1. Rewrote the readinglog.log. It grew faster than the sessions.tsv in size, and records a lot of useless info. This release will replace the old one automatically.

2. Some premature performance improvment. Given my own sessions.tsv grow rate, it will start having load time problem in roughly 5 years on a 1GB RAM colorsoft, and will be around 8Mb in size, if I read and highlight a lot. FYI, the last 3 Paperwhite I owned all died within 2 years under 2 hours daily use on average. Colorsoft is just a Paperwhite with a fancier color screen, I doubt it will survive longer than that (but I do hope so). I saw faster load time on Oasis 2 already, but likely because I am not using it much now that I read mostly on the Colorsoft... I do love the form factor of the Oasis and its buttons, but colors really make me happier.

The end, truly.

Full Changelog: v0.3.1...v0.3.2