Saving doesn't store all highlights

[richo67]

This is absolutely great piece of software I am so grateful that you made it. This is primary reason I am using Koreader everywhere.

I notice recently that not all highlights are exported. There are more showed in app on the right side them they ends up in exported text file. please see screenshots showing the right side with list of highlights. I am attaching it as well as the .lua file as well as the exported text file.

If I could code in python I would look why it is, but python is not something I could code.
Thank you in advance for your time.

-r-
All files in zip.zip

[noembryo]

Thank you for for your words.

Sorry for the delayed response, but I just happened to see this issue by accident (no notification came from GitHub :o/ )

Anyway, looking at the metadata.epub.lua it seems that some problematic highlights don't have a pos0 key.
Is there anything strange that you did when you created them?
What version of Koreader you use (I hope you use the current KoHighlights version ;o)

Any other info?

[noembryo]

OK, I think its fixed with the new release.
Nice catch! There was a problem with the regex I'm using.

Please check it and close this if it works..

[richo67]

Sorry not to react, I also got no notification from github, need to check the settings. I have just saw new release notification from mobileread.
I am using latest koreader 2019.02

[richo67]

It works fine now. Thank you for the fix. I see that you are working on 0.6 release now. Any chance having merge functionality for highlights? I often read books on kindle in night and on phone during day, would be great if I can merge those highlights I made on those two devices :-)

Thank you very much for fixing this issue!

[noembryo]

I think that merging could be dangerous because I would have to add position values that are from two (or more) different devices.
And with the constant change of the metadata.epub.lua format, its almost certain that it will get corrupted at some point..

[noembryo]

Well, after asking KoReader's devs, I'm working on adding a merging option and also a reading position sync between the same book in two different devices.

The only problem so far is that the page numbers of the added highlights will be wrong in KoHighlights (different page count in different screen sizes) but they will display OK in KoReader.
The other thing is that they have to have the same cre_dom_version (version of the CREngine)

[richo67]

You took quite a tempo in development, congratulation, great new features!

Well, after asking KoReader's devs, I'm working on adding a merging option and also a reading position sync between the same book in two different devices.
The only problem so far is that the page numbers of the added highlights will be wrong in KoHighlights (different page count in different screen sizes) but they will display OK in KoReader.
The other thing is that they have to have the same cre_dom_version (version of the CREngine)

Did you make any progress in this?

[noembryo]

Did you make any progress in this?

Well, it was added in v0.6.1.0 almost 3 months ago... ;o)

The page numbers are getting the values of the reader on which the highlight is made, but as I was told and checked, this doesn't affect the location of the highlight.
You can sort highlights by date (the default), so you won't miss the right order.

[richo67]

Wow you have light speed I need to check it out. Thanks a lot. This is really great.
I read in change log that also synchronization is working. Is it anywhere described how it works?

P.S.: I just sent you small donation. Thank you again for great software.

[noembryo]

Wow you have light speed I need to check it out. Thanks a lot. This is really great.
I read in change log that also synchronization is working. Is it anywhere described how it works?

I think its easy enough, but maybe I have to describe it at the main page here.
You have to load both metadata (e.g. your reader's and your tablet's) and then select the relevant rows of the book.
If the book has the same cre_dom_version (version of the CREngine), then, the "Merge/Sync" button gets activated and you have the options to sync the highlights or the position or both.
After that, both book's highlights and/or position is the same.

P.S.: I just sent you small donation. Thank you again for great software.

Thank you very much.. That's a first!

Edit: Added this description at the "Usage" part of the main page.

[richo67]

You have to load both metadata (e.g. your reader's and your tablet's) and then select the relevant rows of the book.

Does this mean that I have to have books in sub-folders of the same folder and scan that folder to have both version of the book visible in the same time?

This is sort of impossible as I have one on my kindle and the other as folder sync to my phone over a cloud sync.

I somehow assume that the sync and merge will be done against the stored version in KoHighlights. This doesn't work and when I archive same book from the cloud folder (and I had archived the previous from the kindle), the book is overwritten in Archive of KoHighlights (no warning).

[noembryo]

Does this mean that I have to have books in sub-folders of the same folder and scan that folder to have both version of the book visible in the same time?

No. You can scan your kindle and then scan again (without clearing the list) your folder with your cloud sync of your phone.
After that, you have both versions of the book loaded.

I somehow assume that the sync and merge will be done against the stored version in KoHighlights. This doesn't work and when I archive same book from the cloud folder (and I had archived the previous from the kindle), the book is overwritten in Archive of KoHighlights (no warning).

This is a different feature. There is only one archived version for every book.
Whenever you archive a book, you get the most recent version of it, with the most recent set of highlights.
That's why it replaces the old ones.

[richo67]

You have to load both metadata (e.g. your reader's and your tablet's) and then select the relevant rows of the book.

Does this mean that I have to have books in sub-folders of the same folder and scan that folder to have both version of the book visible in the same time?

This is sort of impossible as I have one on my kindle and the other as folder sync to my phone over a cloud sync.

I somehow assume that the sync and merge will be done against the stored version in KoHighlights. This doesn't work and when I archive same book from the cloud folder (and I had archived the previous from the kindle), the book is overwritten in Archive of KoHighlights (no warning).

[noembryo]

Does this mean that I have to have books in sub-folders of the same folder and scan that folder to have both version of the book visible in the same time?

No. As I said:

You can scan your kindle and then scan again (without clearing the list) your folder with your cloud sync of your phone.
After that, you have both versions of the book loaded.

So, you have to

• Connect your kindle
• From KoHighlights click "Scan Directory" and browse to your kindle (or its Books folder). This will load the kindle books.
• From KoHighlights click again "Scan Directory" and browse to your folder of your phone's synced files (there must be a KOReader folder there, you can select just that*). This will add your phone's books to the KoHighlights list.
• Select the two rows of the same book and the "Merge/Sync" button will get activated.

* That depends on the phone. If KOReader have write-access to the book's folder, it will create its normal "name-of-the-book.sdr" folder with the metadata next to the books.
If not, then it will write the metadata at a History folder that is inside KOReader's settings folder.