[Requst for Help] Please test dev/nov branch for backward compatibility before merge

[nobiot]

I am about to merge a development branch dev/nov that has been in the works for 6-7 months. It will add support for EPUB books with nov.el and open up more possibility to support non-file-visiting major mode such as Info.

The dev branch contains a lot of refactoring. I have been trying my best to make this change as smooth as possible for existing Org-remark users. I believe there is no break changes in the eye of users. My tests have been good so far. My old notes file work with no adjustments.

It would be great if anyone could spare some of your time to test the branch to see there is any backward compatibility issues. I intend to merge it with the main branch once two or three volunteers have their eyes on it and issues resolved.

What would be great is if you simply open the files (or webpages) where you have highlights, do things you would normally do, and see if anything breaks.

As the new code should be backward compatible as is, you should not have to do any extra config. And your notes should load just fine.

Nevertheless, there will be a few visible changes with additional features (please see below).

I am excited about the new features and will really appreciate if you could help me bring them to whoever benefits from them as smoothly as we can.

Thank you.
nobiot

1. "Icons" for highlights with annotations

The "icon" is a string at the moment and you can customize it with org-remark-icon-notes. If you set it to nil, you can disable it.

2. Automatic adjustment of highlight positions when different from original

This is a new feature that is originally designed to tackle specific rendering issue of nov.el (issue #50). Now it has become a generic feature. I think it would work nicely for webpages when they are edited as you can see in the example of Wikipedia in the image above.

I have introduced another icon to indicate that Org-remark has automatically adjusted the position. You can customize the "icon" string with org-remark-icon-position-adjusted (setting it to nil disable the icon but the adjustment still occurs) and face with org-remark-highlighter-warning.

If you are interested, more technical detail is described in this section of the user manual (already publicly visible).

3. How to set up the new EPUB support

This is not an element for backward compatibility, but you might be still interested in trying it out. It is as easy as adding the following in your configuration (the same mechanism as EWW support as described in README and user manual).

;; Optional if you would like to highlight EPUB books via nov.el
(with-eval-after-load 'nov
  (org-remark-nov-mode +1))

Let me know if you have any questions or comments. Thank you again for reading this.

@mooseyboots @vedang @nanjj @sati-bodhi @randomwangran, @karthink, @holtzermann17, @shombando, @magthe, @linwaytin, @rtrppl, @ryanprior, @ericsfraga, @darcamo, @zhewy, @QMeqGR, @Vidianos-Giannitsis, @AtomicNess123, @ouboub, @dian-yu-luo, @SylvianHemus, @basaran, @Ypot, @oatmealm

[ouboub]

>> "n" == nobiot ***@***.***> writes: I am about to merge a development branch [`dev/nov`](https://github.com/nobiot/org-remark/tree/dev/nov.el) that has been in the works for 6-7 months. It will add support for EPUB books with [nov.el](https://depp.brause.cc/nov.el/) and opens up more possibility to support non-file-visiting major mode such as Info. The dev branch contains a lot of refactoring. I have been trying my best to make this change as smooth as possible for existing Org-remark users. I believe there is no break changes in the eye of users. My tests have been good so far. My old notes file work with no adjustments. It would be great if anyone could spare some of your time to test the branch to see there is any backward compatibility issues. I intend to merge it with the main branch once two or three volunteers have their eyes on it and issues resolved.

I just pulled, so you want us to checkout the dev branch, compile and load and then test it, right? Two or three weeks ago I used the package quite a bit (for correcting matlab exams!) I will check these documents and report back. If there is any specific you want me to check, let me now (I am not sure I use all of its potential....) Uwe Brauer -- Warning: Content may be disturbing to some audiences I strongly condemn Putin's war of aggression against the Ukraine. I support to deliver weapons to Ukraine's military. I support the NATO membership of the Ukraine. I support the EU membership of the Ukraine. https://addons.thunderbird.net/en-US/thunderbird/addon/gmail-conversation-view/

[nobiot]

Hi Uwe, thank you so much for your quick response.

I just pulled, so you want us to checkout the dev branch, compile and
load and then test it, right?

Yes! It would be great if you could try opening the files that were previously working and see if they still work as if the update had never happened. With your help, I would like to ensure that there is no breaking change to users. So testing the files as you normally use them is very helpful.

Optionally, you could see how the new features work (and I'd appreciate if you could give your feedback), but I would not ask you to spend more time than opening and testing the usual activities with the files you have already used.

Two or three weeks ago I used the package quite a bit (for correcting
matlab exams!)

Is it correct that you were correcting the exams as a teacher for your students? That's a very interesting usage :) Let me know if you have any feedback how Org-remark could help you better with this type of work. Happy to hear your experience if you do not mind sharing.

Thank you again.

[basaran]

Hi, I don't use remark on a day to day basis but I can test the dev/nov from scratch on my customized doom emacs.

[nobiot]

@basaran thank you so much! Let me know how you go with Doom :)

[vedang]

I use `org-remark` when reading source code, and I also use it when reading blog posts (via eww). I've been doing both activities on the `dev/nov.el` branch since yesterday. Everything works as advertised, though I did not experience any of the new features. - I tested marking across `.org`, `.el`, `.clj` and `.js` files. - I tested marking on `eww` with archived URLs (files) as well as on the web. - I tested custom pens as created by `org-remark-create` Looking forward to using `org-remark` for `epub` files (not something I've done before).

…

On Sun, Jul 16, 2023 at 2:34 PM nobiot ***@***.***> wrote: @basaran <https://github.com/basaran> thank you so much! Let me know how you go with Doom :) — Reply to this email directly, view it on GitHub <#66 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAUUHVTWGDEXCDIHKDYW6TXQOVCTANCNFSM6AAAAAA2LIYOFI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Cheers, Vedang https://vedang.me @vedang on fosstodon.org

[nobiot]

I've been doing both activities on the dev/nov.el branch since yesterday. Everything works as advertised, though I did not experience any of the new features.

Thank you for taking the time doing this and letting me know. I also appreciate your bullet points detail.

[basaran]

Hello, I had a few remarked files, and they opened fine on the dev/nov.el branch. I also tested epub with nov.el and eww. I am not sure if this is the desired behavior but it seems org-remark is not tracking the individual remarks like it does with org-files.

Here's a screen shot, on a webpage that I have two remarks, but they both open the same marginalia document, and show the same content.

Thank you for making org-remark better and your efforts.

[vedang]

@basaran: This works fine for me. Have you tried deleting and recompiling the elc files and then restarting Emacs? That might solve the problem for you.

…

On Mon, Jul 17, 2023 at 11:20 AM basaran ***@***.***> wrote: Hello, I had a few remarked files, and they opened fine on the dev/nov.el branch. I also tested epub with nov.el and eww. I am not sure if this is the desired behavior but it seems org-remark is not tracking the individual remarks like it does with org-files. Here's a screen shot, on a webpage that I have two remarks, but they both open the same marginalia document, and show the same content. Thank you for making org-remark better and your efforts. [image: image] <https://user-images.githubusercontent.com/30809170/253861454-d2f637f2-d780-4a9f-9388-d27745d081ec.png> — Reply to this email directly, view it on GitHub <#66 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAUUHWATSI3FPMSEQRA4GTXQTHBJANCNFSM6AAAAAA2LIYOFI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

-- Cheers, Vedang https://vedang.me @vedang on fosstodon.org

[nobiot]

Thank you, @basaran.

I also tested epub with nov.el and eww. I am not sure if this is the desired behavior but it seems org-remark is not tracking the individual remarks like it does with org-files

For EWW, this sounds like an expected behaviour. For EPUB, there should be a file in the same directory as the .epub file.

I will come back to this later today when I have more time. Just quickly two things:

1. The notes file is customized in user option org-remark-notes-file-name. The default is STRING (that is, file name) 'marginalia.org'. The directory is the same as the FILE visited. An option that the user can select in customize is FUNCTION with a default function. For EPUB books, the file should depend on this customizing.

2. For EWW, the default file name is <user-emacs-directory>/marginalia.org. This is because EWW does not visit a file, thus there is no directory.

I will add a description in README to make it easy to find. I will also review the user manual. The second case might be a bit confusing because I suspect people do not really notice whether a buffer is visiting a file or not.

[Ypot]

Highlights for eww don't appear when you visit the website in the future.
If you highlight something new, old highlights appear again.

[ouboub]

>> "n" == nobiot ***@***.***> writes: Hi Uwe, thank you so much for your quick response. > I just pulled, so you want us to checkout the dev branch, compile and load and then test it, right? Yes! It would be great if you could still open the files that were previously working and see if they still work as if the update had never happened. With your help, I would like to ensure that there is no breaking change to users. So testing the files as you normally use them is very helpful.

There is a small problem I modified the Makefile to this EMACS = /opt/emacs29/bin/emacs README-elpa: README.org -emacs --batch $< -f org-ascii-export-to-ascii -mv README.txt README-elpa .PHONY: test-compile test-compile: emacs --batch --eval "(add-to-list 'load-path default-directory)" \ -f batch-byte-compile ./*.el # Check declare-function emacs --batch --eval "(check-declare-directory default-directory)" .PHONY: clean clean: find . -name "*.elc" -delete But I obtained the following error messages, and last one worried me. I am using emacs git master from February this year, and org mode about the same. So pretty recent versions I suppose. ,---- | | toplevel form: | | org-remark-convert-legacy.el:93:17: Warning: Variable ‘props’ left | uninitialized | | In toplevel form: | | org-remark.el:840:18: Warning: ‘org-remark-highlight-save’ is an | obsolete function (as of 1.2.0); use ‘org-remark-highlight-add’ instead. | | In end of data: | | org-remark.el:840:18: Warning: the function | ‘org-remark-highlight-save’ is not known to be defined. emacs --batch | --eval "(check-declare-directory default-directory)" | `----

Optionally, you could see how the new features work (and I'd appreciate if you could give your feedback), but I would not ask you to spend more time than opening and testing the usual activities with the files you have already used. > Two or three weeks ago I used the package quite a bit (for correcting matlab exams!) Is it correct that you were correcting the exams as a teacher for your students? That's a very interesting usage :) Let me know if you have any feedback how Org-remark could help you better with this type of work. Happy to hear your experience if you do not mind sharing.

I can write about this a bit later, in a private message if you don't mind. Thanks for sharing this nice and useful package.

…

Thank you again.

-- Warning: Content may be disturbing to some audiences I strongly condemn Putin's war of aggression against the Ukraine. I support to deliver weapons to Ukraine's military. I support the NATO membership of the Ukraine. I support the EU membership of the Ukraine. https://addons.thunderbird.net/en-US/thunderbird/addon/gmail-conversation-view/

[nobiot]

Highlights for eww don't appear when you visit the website in the future.
If you highlight something new, old highlights appear again.

@Ypot , Is this a problem you didn't have with the current release 1.1.0 and started to have when you tried the development branch?

If not, I suspect it's an issue loading sequence (customizing). I suggest to open a new issue and we can discuss it separately.

[nobiot]

But I obtained the following error messages, and last one worried me.

| In toplevel form: | | org-remark.el:840:18: Warning: ‘org-remark-highlight-save’ is an | obsolete function (as of 1.2.0); use ‘org-remark-highlight-add’ instead. | | In end of data: | | org-remark.el:840:18: Warning: the function | ‘org-remark-highlight-save’ is not known to be defined. emacs --batch | --eval "(check-declare-directory default-directory)"

Thanks for reporting this detail.

This is a weird warning. I get it too. It is caused by this line where I define obsolescence... Because I am making it obsolete, there is no such function.

I will change it to also add an alias, but this still causes the warning.

(define-obsolete-function-alias #'org-remark-highlight-save #'org-remark-highlight-add "1.2.0"
  "Save a single HIGHLIGHT in the marginal notes file. We no longer
save the notes file to disk; hence the name change")`

I have just patched the legacy code warning.

[nobiot]

@ouboub, We can disregard the warning -- functionally everything works despite it on my end because there is a new function. I will look if I can somehow silence it. Otherwise, I'd need to remove the line to just remove the function -- not sure if that is a good practice...

[ouboub]

>> "n" == nobiot ***@***.***> writes: @ouboub, We can disregard the warning -- functionally everything works despite it on my end because there is a new function. I will look if I can somehow silence it. Otherwise, I'd need to remove the line to just remove the function -- not sure if that is a good practice...

Ok, thanks. I just run a basic test, restarting my emacs with the nov branch, opened my old files and anything ok so far. For my workflow I don't notice any change so far. (In the coming days I will describe how I use your package for checking matlab(could be any other language) files of my students. BTW, I thought I knew how to set the size of the left margin buffer, or am I mistaken maybe there only exists the possibility to have the org-remark buffer horizontally? I am asking because for my 14 inch Laptop the the fonts size I use the left margin windows is sometimes a bit small. regards

…

-- Warning: Content may be disturbing to some audiences I strongly condemn Putin's war of aggression against the Ukraine. I support to deliver weapons to Ukraine's military. I support the NATO membership of the Ukraine. I support the EU membership of the Ukraine. https://addons.thunderbird.net/en-US/thunderbird/addon/gmail-conversation-view/

[nobiot]

BTW, I thought I knew how to set the size of the left margin buffer, or am I mistaken maybe there only exists the possibility to have the org-remark buffer horizontally?

Could you see if org-remark-notes-display-buffer-action might work for you? It might be easier to see how it works from customising. I’ll see if I can improve the user manual in this user option.

[oatmealm]

I've installed the dev branch and so far I'm not seeing any regression, though I only work with PDFs. Will play around with an eww and epub and will report if there's anything interesting I'm seeign.

Build: grafted, HEAD -> dev/nov.el, origin/dev/nov.el 883f948 2023-07-17 18:48:48 +0200

[nobiot]

I've installed the dev branch and so far I'm not seeing any regression

Thank you @oatmealm.

With all the feedback, I feel confident about merging the dev branch. I will update the documentation based on the feedback above and merge -- probably not this weekend but one after...

[nobiot]

though I only work with PDFs.

I am considering getting PDF-tool and Org-remark to work together. Still just an idea -- keeping all the capabilities of PDF-tool to interact with a PDF document, save the highlights and annotations in the PDF file as it is now. But have the highlights and annotations sync'ed with the marginalia with Org syntax... This way, we may be able to use the unified Org syntax to write annotations and get the highlights and annotations saved to the PDF document for portability...

Not sure if this is technically feasible. Interested in this direction? If so, I might open a discussion thread once I have something to show -- after the merge, probably few weeks or months down the line...

[Ypot]

I am considering getting PDF-tool and Org-remark to work together. Still just an idea -- keeping all the capabilities of PDF-tool to interact with a PDF document, save the highlights and annotations in the PDF file as it is now. But have the highlights and annotations sync'ed with the marginalia with Org syntax... This way, we may be able to use the unified Org syntax to write annotations and get the highlights and annotations saved to the PDF document for portability...

Not sure if this is technically feasible. Interested in this direction? If so, I might open a discussion thread once I have something to show -- after the merge, probably few weeks or months down the line...

I like the idea. It would be like org noter, but in real time.

[nobiot]

I like the idea. It would be like org noter, but in real time.

Thank you, @Ypot.
Is your issue resolved?

[ouboub]

>> "n" == nobiot ***@***.***> writes: > BTW, I thought I knew how to set the size of the left margin buffer, or am I mistaken maybe there only exists the possibility to have the org-remark buffer horizontally? Could you see if [`org-remark-notes-display-buffer-action`](https://github.com/nobiot/org-remark/blob/main/org-remark.el#L66) might work for you? It might be easier to see how it works from customising. I’ll see if I can improve the user manual in this user option. > BTW, I thought I knew how to set the size of the left margin buffer, or am I mistaken maybe there only exists the possibility to have the org-remark buffer horizontally? Could you see if [`org-remark-notes-display-buffer-action`](https://github.com/nobiot/org-remark/blob/main/org-remark.el#L66) might work for you? It might be easier to see how it works from customising. I’ll see if I can improve the user manual in this user option.

Well that name is not very intuitive I have to say. I run customize-option for the variable, but how can I resize the left margin, I changed the value of slot but this seems to have no effect I might change the side to up? Sorry but could the document string explain this a bit better?

…

-- Warning: Content may be disturbing to some audiences I strongly condemn Putin's war of aggression against the Ukraine. I support to deliver weapons to Ukraine's military. I support the NATO membership of the Ukraine. I support the EU membership of the Ukraine. https://addons.thunderbird.net/en-US/thunderbird/addon/gmail-conversation-view/

[nobiot]

Sorry but could the document string explain this a bit better?

Thank you for splitting the issue out into #67. My response provided there. Let me if it helps.

[nobiot]

The branch has been merged with main. Thank you for your support for this!