using Joplin (opensource equivalent to Evernote) Feature request

[lbesnard]

Hi, brand new user of koreader on my kobo glo. Thanks for this amazing opensource project.

I feel like joplin could be used to upload notes as well. In the backend, Joplin can also use Nextcloud(another opensource cloud storage).

[mustafa-001]

Hi, i tinkered with Joplin a little, and I think to implement Evernote like functionality with Joplin we need to directly edit Joplins cloud stored user database, since Joplin doesn't have any Web API. To do that we need to:
1- authenticate with users cloud service (Dropbox, OneDrive, WebDAV, NextCloud)
2- download all database
3- add our notes (and hope we dont mess users other notes)
4- upload our database to cloud
5- also be careful not to delete notes from other Joplin apps trying to sync.

I think it's not much different from rewriting a note taking app from ground.

However, Joplin has an API that can be used from local internet, it wont be same but better than nothing. I wrote a small wrapper in lua for needed functions.

Logic of uploading notes can be similar to Evernote:

find guid of Koreader Notes folder
if folder does not exist create it
check if title of the book exist
if not create it
upload clipping

For UI, we can put a submenu which includes:
1- Address Settings
2- Auth Token Entry (maybe additional option to read from file)
3- Export to Joplin switch (like exporttoTXT or exporttohtml)

So whats your thoughts?

[Frenzie]

@kereviz Seems fine to me.

[lbesnard]

that sounds like a great way to do it!

[mustafa-001]

Sorry for late response. Before writing code it didn't occurred to me to check if it works on phone or e-reader. It turns out Joplin only listen on localhost, not 0.0.0.0 like most apps do. So it only works on same machine. I wrote to Joplin's forum about this, they said it's not secure but if there is pull request about changing IP in terminal version they could accept it. Still it only works on either you
-use terminal version of Joplin
-create a tunnel from external IP to localhost (eg. $socat tcp-listen:41185,reuseaddr,fork tcp:localhost:41184 on linux)

Either way it requires some manual steps . I finished basic things with it; there is some rough edges but it works. So I will create a how-to, share it or link of it on wiki.

[mustafa-001]

About annotations, it's little complicated.

Basic overview of bookmark, highlight, clippings architecture is like this:

-readerhighlight.lua saves highlighted note to two tables in books sidecar file : highlight and bookmarks. highlight.text, bookmarks.note and bookmarks.text contains highlighted part.
-when you edit/annotate/rename a bookmark you change bookmark.text item in sidecar file. readerbookmark.lua doesn't send any info about new name of bookmark into readerhighlight.lua. highlight.text and bookmarks.note stays same. (a sidenote, it's actually renaming a bookmark, you change how your note shows up in bookmarks menu, actual note is still stored in seperate index of bookmark table)
-Evernote exporter's clip.lua parses highlight table , saves it to clipboard/evernote/evernote.sdr. This file is contains clippings of whole library.
-So basically we save a note 4 times.
-Since bookmark and highlight tables have different structure we can't simply pass bookmark table to parser. It needs rewriting of clip.lua:parseHighlight. But I don't know if it's wise to change source of parsing from highlight to bookmarks ( images, page info etc.) . I guess it's worth a try.

["bookmarks"] = {
[4] = { --4 is sorting number in whole book
           ["page"] = "/body/DocFragment[4]/body/p[6]/text().0",
           ["highlighted"] = true,
           ["notes"] = "highlighted text",
           ["datetime"] = "2019-09-24 21:30:16",
           ["pos0"] = "/body/DocFragment[4]/body/p[6]/text().0",
           ["pos1"] = "/body/DocFragment[4]/body/p[7]/text().25",
           ["text"] = "Page 5 highlighted text and my annotations @ 2019-09-24 21:30:16"
       },
}

["highlight"] = {
[5] = {  --5 is page number
           [1] = {  --1 is notes number in page 
               ["datetime"] = "2019-09-24 21:30:16",
               ["drawer"] = "lighten",
               ["pos0"] = "/body/DocFragment[4]/body/p[6]/text().0",
               ["pos1"] = "/body/DocFragment[4]/body/p[7]/text().25",
               ["text"] = "highlighted text "
           }
       },
}

Also is /mnt/us/documents/My Clippings.txt Kindle's default clippings file?

[Frenzie]

Also is /mnt/us/documents/My Clippings.txt Kindle's default clippings file?

I don't know about the file, but the rest of the path sure is. ;-)

[NiLuJe]

It's Kindle, I don't know if we ought to touch it on our end.

[NiLuJe]

(read: we shouldn't. Assume it's RO.)

[poire-z]

I don't know anything about evernote.
But about the highlight and bookmarks entries, I wrote a few posts about how they are used. Mentionning some of them just in case it makes things clearer to you:
#1095 (comment) #4670 (comment) #3415 (comment)

One thing is that we can't really fix or change what is stored in what, because we must stay compatible with past bookmarks and highlights, and tools like KOHighlight that parse these files.
But I guess from elsewhere, if you're just reading them, I think we update both when adding/removing - it's just that bookmarked pages (no text selexted) only get into bookmarks.