FR: Sync article annotations to Wallabag using plugin

[ghost]

Does your feature request involve difficulty completing a task? Please describe.
Currently, if one uses the Wallabag plugin to read articles on the device, there is no way to transfer highlights made with KOReader back to Wallabag.

Describe the solution you'd like
It would be nice if the Wallabag plugin supported syncing annotations from KOReader to Wallabag.

Additional context
Wallabag has an API to fetch and upload annotations (https://app.wallabag.it/api/doc).

[Frenzie]

I'll have to check with the API, but translation from one format to the other might be quite challenging.

[ghost]

Testing with one annotation, it seems the formats are somewhat similar, at least in how the highlight's position in the text is stored.

This is how Wallabag stores a highlight of a section of the KOReader wiki home page:

{'annotator_schema_version': 'v1.0', 'id': 3, 'text': 'This is a note about this highlight', 'created_at': '2021-01-14T18:48:33-0500', 'updated_at': '2021-01-14T18:48:33-0500', 'quote': "the correct path) to install/upgrade.ConfigurationAfter starting up KOReader, all you need to do is navigate the file manager to a (hidden) library folder and set it as default by long-pressing the file name. Files with supported file extensions will show up in this file manager. Touch gestures for navigating the user interface are explained in gestures and multiswipes. You can read more about lesser known useful functions here, and there's a large survey of features including explanations here.Dictionary support\nCalibre wireless connection\nEvernote export\nProgress sync\nZsync transport", 'ranges': [{'start': '/p[8]', 'startOffset': '117', 'end': '/ul[3]/li[5]/a[1]', 'endOffset': '15'}]}

KOReader stores a highlight of the same section this way, in the metadata.epub.lua:

    ["highlight"] = {
        [8] = {
            [1] = {
                ["text"] = "the correct path) to install/upgrade.\
Configuration\
After starting up KOReader, all you need to do is navigate the file manager to a (hidden) library folder and set it as default by long-pressing the file name. Files with supported file extensions will show up in this file manager. Touch gestures for navigating the user interface are explained in gestures and multiswipes. You can read more about lesser known useful functions here, and there's a large survey of features including explanations here.\
Dictionary support\
Calibre wireless connection\
Evernote export\
Progress sync\
Zsync transport",
                ["chapter"] = "koreader/koreader",
                ["pos0"] = "/body/DocFragment[3]/body/p[8]/text().117",
                ["pos1"] = "/body/DocFragment[3]/body/ul[3]/li[5]/a/text().15",
                ["drawer"] = "lighten",
                ["datetime"] = "2021-01-14 18:58:33",
            },
        },
    }

    ["bookmarks"] = {
        [1] = {
            ["highlighted"] = true,
            ["chapter"] = "koreader/koreader",
            ["pos0"] = "/body/DocFragment[3]/body/p[8]/text().117",
            ["pos1"] = "/body/DocFragment[3]/body/ul[3]/li[5]/a/text().15",
            ["page"] = "/body/DocFragment[3]/body/p[8]/text().117",
            ["text"] = "This is a note about this highlight",
            ["datetime"] = "2021-01-14 18:58:33",
            ["notes"] = "the correct path) to install/upgrade.\
Configuration\
After starting up KOReader, all you need to do is navigate the file manager to a (hidden) library folder and set it as default by long-pressing the file name. Files with supported file extensions will show up in this file manager. Touch gestures for navigating the user interface are explained in gestures and multiswipes. You can read more about lesser known useful functions here, and there's a large survey of features including explanations here.\
Dictionary support\
Calibre wireless connection\
Evernote export\
Progress sync\
Zsync transport",
        },
    }

[NiLuJe]

That looks XPath-ish on both sides, but I think CRe's idea of XPaths can be a bit peculiar at times... @poire-z might be able to drop some knowledge on that front ;).

[poire-z]

Don't know much about the real XPath syntax - but it indeed looks like names/indexes are similar on both sides, with added DocFragment and .text() stuff. So, one could try at converting between the 2 formats (outside of crengine of course :) Code involved if needed to understand the xpointers format (the V2 stuff):
https://github.com/koreader/crengine/blob/a90c0be089b08e0b0af64289fb0fdc8ba6d5ca6b/crengine/src/lvtinydom.cpp#L9614-L9874

[krillin666]

So, is this doable ? 😃

[Kreest]

Currently have a proof of concept mostly working (for sending notes) here, should be able to get it into a usable shape before the year is out

[pipoprods]

@Kreest just tried your patch and it seems that it breaks downloading articles as ePub. I get HTML files without extension instead (that I can't see from KOReader's internal file browser).

Did I miss something?

[Frenzie]

@pipoprods You probably didn't try the patch but the entire file. Sometimes that's the same thing, but in this case the entire file doesn't have #11532.

[pipoprods]

Oh yes, that's it… I'll try again later.
Thanks :)