Upstream pdf roll

[aikrahguzar]

Hi, @vedang,
I am opening this (as a draft) to have some concrete starting point for merging support for displaying multiple pages simultaneously.

It is a draft because because some things need to be sorted out,

1. Currently this requires my fork of image-roll. Since @dalanicolai wants to focus on other projects and for me image-roll is just a means to an end, I think it would be better to import image-roll.el into pdf-roll.el. This mostly requires changing the prefix for functions and it pretty mechanical but I wanted to get yours and @dalanicolai's feedback about this. To me it seems like the best thing to do since it simplifies the life of users since they don't have to install a dependency and the file itself is around 400 lines long currently so not a very large addition especially for a project of the size of pdf-tools. I went ahead and added the image-roll code directly to the pdf-roll.el so now no external package is needed.

2. What strategy should be used to convey that the feature is experimental and might not interact well with every feature currently available? I prefer a message when pdf-view-roll-minor-mode is enabled. But I am fine with the suggestion of a flag suggested in New package image-roll (continuous-scroll) #104

3. One possible degradation in performance I know of is the loss of caching in pdf-links-read-link-action since with multiple pages displayed the labels depend on the position of a page among displayed pages. I think the current behavior can be preserved when pdf-view-roll-minor-mode is not used by caching only the first displayed page. The result will be messy but it is doable.

4. This also introduces some backward incompatibility since we need to track which page is involved for operations, e.g. pdf-view-active-region now has a different format which includes the page on which the region exists. Similarly isearch results also include information about the page. These changes are not user visible when using pdf-tools as an application but they do change some api.

5. Mostly feedback is needed from people to know what breaks. The parts I use work fine but I can't say that will be the case for everyone.

Also this PR includes #29, #113, #188 because it was too difficult for me to remove those changes from among much larger changes.

P.S. I will be travelling starting Friday and will be without my laptop and access to non-work email so I will get back to whatever discussion happens only in August.

Thanks for maintaining pdf-tools.

[rdiaz02]

The output from the "messages-larger-frame" shows lists are identical (except for last entry). I did move around in the PDF, but the switching of windows shows no changes.

The output from the "messages-smaller-frame" shows:

1. On switching to the scratch buffer, sometimes the second element is initially identical to the PDF and then changes. For example, lines 11, 12, 13. Or lines 80, 81, 82.
2. I sometimes see an error (e.g., line 28)

[rdiaz02]

Can you try evaluating this and see if it helps with the bug?

(advice-add 'pdf-roll-pre-redisplay :after
            (defun pdf-roll-set-window-point (win)
              (let ((start (pdf-roll-page-to-pos (pdf-view-current-page win))))
                (set-window-point win (+ start
                                         (if (pos-visible-in-window-p (+ 2 start) win) 2 0))))))

It did not help. Bug is still present. I checked with two different fonts (DM Mono, Julia Mono) and with both GTK and Lucid builds

[aikrahguzar]

I am stumped, did you try #224 (comment) ?

[rdiaz02]

I am stumped, did you try #224 (comment) ?

Sorry, I think I missed that before. I just tried it, in addition to the advice add, and it does not help.

I tried it also without the advice: it does not help either.

[aikrahguzar]

I am stumped, did you try #224 (comment) ?

Sorry, I think I missed that before. I just tried it, in addition to the advice add, and it does not help.

I tried it also without the advice: it does not help either.

I am out of ideas then. From the picture it looks like window-vscroll gets reset to zero when you switch to the other window. But the messages seem to be indicate that we set things up correctly.

I will ask on help-gnu-emacs@gnu.org, maybe someone there has an idea.

[aikrahguzar]

@rdiaz02 one last thing to try: does evaluating,

(put 'pdf-roll-margin 'cursor t)

help?

[rdiaz02]

(put 'pdf-roll-margin 'cursor t)

No. To make sure I am doing what you ask, this is what I am doing:

1. emacs -Q -l dm-problem.el where dm-problem.el is

(package-initialize)
(pdf-loader-install)
(split-window-right)
(set-face-attribute 'default nil :weight 'regular :font "DM Mono")
(put 'pdf-roll-margin 'cursor t)
(find-file-other-window "btac710.pdf")

2. M-x pdf-view-roll-minor-move
3. M-x other-window a bunch of times.

[aikrahguzar]

(put 'pdf-roll-margin 'cursor t)

No. To make sure I am doing what you ask, this is what I am doing:

1. emacs -Q -l dm-problem.el where dm-problem.el is

(package-initialize)
(pdf-loader-install)
(split-window-right)
(set-face-attribute 'default nil :weight 'regular :font "DM Mono")
(put 'pdf-roll-margin 'cursor t)
(find-file-other-window "btac710.pdf")

2. M-x pdf-view-roll-minor-move

3. M-x other-window a bunch of times.

Yes, you are testing it correctly. I am completely out of ideas now.

[rdiaz02]

Thanks a lot for trying to fix it!

[aikrahguzar]

@rdiaz02 if you are still motivated to get this fixed, I think the best thing is to try to reproduce it without pdf tools. I think you should be able to reproduce this behavior by starting with emacs -Q and pasting in the scratch buffer something like,

(progn
  (defun pin-vscroll-down (win)
    (set-window-vscroll win 200 t))
  (set-frame-font "JuliaMono" nil t)
  (let ((image1 (create-image "/path/to/image1"))
        (image2 (create-image "/path/to/image2"))
        (image3 (create-image "/path/to/image3")))
    (with-current-buffer (get-buffer-create "*image-scroll-test*")
      (insert " \n \n \n \n \n \n")
      (put-image image1 1)
      (put-image image2 5)
      (put-image image3 9)
      (goto-char 7)
      (add-hook 'pre-redisplay-functions #'pin-vscroll-down nil t))
    (split-window-right)
    (other-window 1)
    (switch-to-buffer "*image-scroll-test*")))

Where you replace /path/to/image* by path of actual image files. When you evaluate this using C-M-x you should see a buffer with three images and one of the images partially visible. Try to see if you the buggy movement of images in this setup. You might need to play with the code a little bit.

[rdiaz02]

Yes, I can reproduce the buggy movement of images with the code above. I just tried with three of the problematic fonts (Julia Mono, Intel One Mono, DM Mono) and both the gtk and lucid builds. (Note that if point is placed in certain places in the buffer with the images, the buggy movement of the images does not happen; but in most other places, it does happen).

[aikrahguzar]

Yes, I can reproduce the buggy movement of images with the code above. I just tried with three of the problematic fonts (Julia Mono, Intel One Mono, DM Mono) and both the gtk and lucid builds. (Note that if point is placed in certain places in the buffer with the images, the buggy movement of the images does not happen; but in most other places, it does happen).

Thanks! So I think this is probably an Emacs bug. What are the places where the buggy movement doesn't happen? If it is the space between the images then we have something new to try. Otherwise please use M-x report-emacs-bug RET to report a bug and please cc me on it.

[rdiaz02]

Thanks! So I think this is probably an Emacs bug. What are the places where the buggy movement doesn't happen? If it is the space between the images then we have something new to try.

It depends on what spaces and on the size of the frame. Some images to illustrate the variety of behaviors:

• Very large frame, point between second and third image (which is where point is placed when running the code), no issue:
  

• Move point between first and second image (not with M/C-b, since that does not achieve that, but by using the mouse) , problem appears:

• Start emacs again, but now frame has smaller size (I use XMonad, so this is as simple as starting from a different place). Note point is between second and third image, and we experience the problem:

• But, if I move point immediately before it (again, not with M/C-b, since that does not achieve that, but by using the mouse), I see no problem:
  

Summary

If it is the space between the images

the answer is: sometimes (with some frame sizes) in some of the spaces between the images the problem does not appear, but with other frame sizes or in other spaces between images the problem does appear.

[aikrahguzar]

@rdiaz02 , I think this is enough to file a file a bug report. Please do that, including the code and a detailed description of the steps needed to see the bug. If you can please try adjusting the height of the images so that the bug is reproduced independent of the frame size, this can be done using,

(progn
  (defun pin-vscroll-down (win)
    (set-window-vscroll win 200 t))
  (set-frame-font "JuliaMono" nil t)
  (let* ((height (/ (* 2 (frame-pixel-height)) 5))
         (image1 (create-image "/path/to/image1" nil nil :height height))
         (image2 (create-image "/path/to/image2" nil nil :height height))
         (image3 (create-image "/path/to/image3" nil nil :height height)))
    (with-current-buffer (get-buffer-create "*image-scroll-test*")
      (insert " \n \n \n \n \n \n")
      (put-image image1 1)
      (put-image image2 5)
      (put-image image3 9)
      (goto-char 7)
      (add-hook 'pre-redisplay-functions #'pin-vscroll-down nil t))
    (split-window-right)
    (other-window 1)
    (switch-to-buffer "*image-scroll-test*")))

where you can tweak the let binding of height so that (hopefully) the bug always appears.

[rdiaz02]

Before submitting the bug, is there any reason to (goto-char 7) ? I've just found that that does not always lead to the problem. However, going to char 11, as far as I can tell, always gives me the bug with a range of frame sizes and image sizes (height). By moving to char 11, we force at least part of the first image to not be shown; it seems that it is precisely that something on the upper part of the buffer is not shown that triggers the bug.

please cc me on it

But I do not know your email and could not find it. (In my github profile there is a link to my webpage, with email at the bottom; you can email me yours there). Alternatively, if it is simpler if you submit the bug, that is of course fine with me.

[aikrahguzar]

Before submitting the bug, is there any reason to (goto-char 7) ? I've just found that that does not always lead to the problem. However, going to char 11, as far as I can tell, always gives me the bug with a range of frame sizes and image sizes (height). By moving to char 11, we force at least part of the first image to not be shown; it seems that it is precisely that something on the upper part of the buffer is not shown that triggers the bug.

No particular reason for that position, I just placed the cursor after the second image but 11 that is after the third one also works. You should tweak the code if it leads to a clear reproduction of the bug. I can't reproduce it so I am just trying to do what we do in this pr without using any pdf-tools machinery.

please cc me on it

But I do not know your email and could not find it. (In my github profile there is a link to my webpage, with email at the bottom; you can email me yours there). Alternatively, if it is simpler if you submit the bug, that is of course fine with me.

I sent you an email so you can cc me, if you don't get it please just post the link here and I will follow the bug. I think you should submit it since I can't reproduce it and won't be able to answer any questions.

[rdiaz02]

Bug report here: https://lists.gnu.org/archive/html/bug-gnu-emacs/2024-03/msg01315.html

[aikrahguzar]

@quotuva since you have observed this problem, can you reproduce it using the recipe here https://lists.gnu.org/archive/html/bug-gnu-emacs/2024-03/msg01315.html

[rdiaz02]

I've found an example that does not depend on fonts; this does not directly fulfills Eli Zaretskii's request for a simple recipe that does not depend on fonts (https://lists.gnu.org/archive/html/bug-gnu-emacs/2024-03/msg01351.html), since, for now, it depends on pdf-tools.

Also, I do not understand it, and maybe it is a problem due to vertico. No idea. I'll have very little time in the next few days to look into this, but I am posting it here, in case anyone can shed any light.

Steps to reproduce

1. emacs -Q
2. eval this code

(package-initialize)
(pdf-loader-install)
(add-hook 'pdf-view-mode-hook (lambda() (pdf-view-roll-minor-mode 1)))
(vertico-mode 1)

3. Load a multi-page pdf, and move so that the page break between two pages is visible in one window.
4. Do a couple of M-x and maybe C-x b. No need to execute anything or switch buffers; the key is to get the minibuffer to show options.
5. Now, do C-x o (other-window) a bunch of times.
6. Observe the PDF move up and down.

Interestingly, if you open another PDF, it does not show the problem until you start doing step 4.

Edit This does not depend on pdf-tools nor on fonts (I'll try to claim the bonus points Eli was offering 😂):

1. emacs -Q
2. eval this code, which is like the one in Upstream pdf roll #224 (comment) but without font changes

(package-initialize)
(vertico-mode 1)

(progn
  (defun pin-vscroll-down (win)
    (set-window-vscroll win 200 t))
   (let* ((height (/ (* 2 (frame-pixel-height)) 15))
         (image1 (create-image "/tmp/image1.png" nil nil :height height))
         (image2 (create-image "/tmp/image2.png" nil nil :height height))
         (image3 (create-image "/tmp/image3.png" nil nil :height height)))
    (with-current-buffer (get-buffer-create "*image-scroll-test*")
      (insert " \n \n \n \n \n \n")
      (put-image image1 1)
      (put-image image2 5)
      (put-image image3 9)
      ;; With larger image sizes (goto-char 3)
      ;; also consistently triggers the problem.
      (goto-char 11)
      (add-hook 'pre-redisplay-functions #'pin-vscroll-down nil t))
    (split-window-right)
    (other-window 1)
    (switch-to-buffer "*image-scroll-test*")))

3. Go to the window with the images and do a few M-x. No need to execute anything, just have the minibuffer show options.
4. C-x o a few times. You'll see the images move up and down.

Edit 2 Note that once you experience the moving up and down, turning off vertico-mode will not solve the problem.

The image files I am using above. Place in /tmp.

[aikrahguzar]

@rdiaz02 , I can reproduce the problem with the recipe you posted on Emacs devel, which is super weird since I use vertico and haven't seen the problem.

It can also be reproduced using builtin icomplete-vertical-mode instead of vertico.

[rdiaz02]

@rdiaz02 , I can reproduce the problem with the recipe you posted on Emacs devel, which is super weird since I use vertico and haven't seen the problem.

It can also be reproduced using builtin icomplete-vertical-mode instead of vertico.

@aikrahguzar I just saw your message in the bug thread. I can reproduce it too! (The bonus points are definitely yours 😉 )

[aikrahguzar]

@rdiaz02 does (setq read-minibuffer-restore-windows nil) fix the problem for you? It seems to fix at least the minimal reproducer for me.

[rdiaz02]

If fixes, for me, this case

(setq read-minibuffer-restore-windows nil)
(progn
  (defun pin-vscroll-down (win)
    (set-window-vscroll win 200 t))
   (let* ((height (/ (* 2 (frame-pixel-height)) 5))
         (image1 (create-image "/tmp/image1.png" nil nil :height height))
         (image2 (create-image "/tmp/image2.png" nil nil :height height))
         (image3 (create-image "/tmp/image3.png" nil nil :height height)))
    (with-current-buffer (get-buffer-create "*image-scroll-test*")
      (insert " \n \n \n \n \n \n")
      (put-image image1 1)
      (put-image image2 5)
      (put-image image3 9)
      ;; With larger image sizes (goto-char 3)
      ;; also consistently triggers the problem.
      (goto-char 11)
      (add-hook 'pre-redisplay-functions #'pin-vscroll-down nil t))
    (split-window-right)
    (other-window 1)
    (switch-to-buffer "*image-scroll-test*")))

and this case too:

(setq read-minibuffer-restore-windows nil)
(package-initialize)
(pdf-loader-install)
(add-hook 'pdf-view-mode-hook (lambda() (pdf-view-roll-minor-mode 1)))
(vertico-mode 1)

But it does not fix the minimal example with font size change:

(setq read-minibuffer-restore-windows nil)
(progn
  (defun pin-vscroll-down (win)
    (set-window-vscroll win 200 t))
  ;; Any of the following leads to the bug
  (set-frame-font "JuliaMono" nil t)
  ;; (set-frame-font "DM Mono" nil t)
  ;; (set-frame-font "Intel One Mono" nil t)
  (let* ((height (/ (* 2 (frame-pixel-height)) 15))
         (image1 (create-image "/tmp/image1.png" nil nil :height height))
         (image2 (create-image "/tmp/image2.png" nil nil :height height))
         (image3 (create-image "/tmp/image3.png" nil nil :height height)))
    (with-current-buffer (get-buffer-create "*image-scroll-test*")
      (insert " \n \n \n \n \n \n")
      (put-image image1 1)
      (put-image image2 5)
      (put-image image3 9)
      ;; With larger image sizes (goto-char 3)
      ;; also triggers the problem.
      (goto-char 11)
      (add-hook 'pre-redisplay-functions #'pin-vscroll-down nil t))
    (split-window-right)
    (other-window 1)
    (switch-to-buffer "*image-scroll-test*")))

Nor the case of font change with pdf-tools

(setq read-minibuffer-restore-windows nil)
(package-initialize)
(pdf-loader-install)
(add-hook 'pdf-view-mode-hook (lambda() (pdf-view-roll-minor-mode 1)))
(split-window-right)
(set-face-attribute 'default nil :weight 'regular :font "DM Mono")
(find-file-other-window "btac710.pdf")

However, (setq read-minibuffer-restore-windows nil) allows us to revert the problem. Without that setting, once the problem started, it could not be fixed by changing fonts. Now, changing fonts after the problem appears, for example to DejaVu Sans Mono, solves the issue. (For now, I am setting (setq read-minibuffer-restore-windows nil) in my config; as I say, I can circumvent the issue just by changing fonts in the same session).

[aikrahguzar]

However, (setq read-minibuffer-restore-windows nil) allows us to revert the problem. Without that setting, once the problem started, it could not be fixed by changing fonts. Now, changing fonts after the problem appears, for example to DejaVu Sans Mono, solves the issue. (For now, I am setting (setq read-minibuffer-restore-windows nil) in my config; as I say, I can circumvent the issue just by changing fonts in the same session).

I am happy we have found a not too unreasonable workaround to the problem. The dependence on fonts is very weird but so is the fact that anything happening in minibuffer affects this at all. Hopefully Emacs devs will be able to get to the bottom of this.

I think it would be nice to mention the summary of this message on the mailing list too and it is probably better to continue this discussion there rather than here.

[rdiaz02]

I think it would be nice to mention the summary of this message on the mailing list too and it is probably better to continue this discussion there rather than here.

I just summarized it in the thread in the bug-gnu-emacs mailing list (https://lists.gnu.org/archive/html/bug-gnu-emacs/2024-04/msg00001.html )

[aikrahguzar]

@rdiaz02, can you please test the fix suggested by Eli on the mailing list and see if it fixes the font related issues too? I tried it with the icomplete-vertical-mode reproducer and that issue is fixed by it.

[rdiaz02]

Which message? Can you provide the link to the exact message from Eli where the fix is provided? (I scanned Eli's messages quickly, but I am failing to see any elisp code with a fix).

[aikrahguzar]

Which message? Can you provide the link to the exact message from Eli where the fix is provided? (I scanned Eli's messages quickly, but I am failing to see any elisp code with a fix).

There is no elisp fix, there is a small (2 character) change to the C code. The message is here
https://lists.gnu.org/archive/html/bug-gnu-emacs/2024-04/msg00331.html

He is basically asking you to apply this diff

diff --git a/src/xdisp.c b/src/xdisp.c
index 6087a25afcc..c20b7033e4d 100644
--- a/src/xdisp.c
+++ b/src/xdisp.c
@@ -20201,7 +20201,7 @@ redisplay_window (Lisp_Object window, bool just_this_one_p)
       /* The vscroll should be preserved in this case, since
 	 `pixel-scroll-precision-mode' must continue working normally
 	 when a mini-window is resized.  (bug#55312) */
-      if (!w->preserve_vscroll_p || !window_frozen_p (w))
+      if (!w->preserve_vscroll_p && !window_frozen_p (w))
 	w->vscroll = 0;
 
       w->preserve_vscroll_p = false;

then rebuild Emacs and see if you can reproduce the issue with fonts.

[rdiaz02]

Thanks. I just did and replied in the thread: both the bug as reported for emacs bugs, and the specific issue I originally reported above (#224 (comment)) are fixed with Eli's modification. Thanks a lot for the effort for getting this fixed!

[aikrahguzar]

Thanks a lot for the effort for getting this fixed!

Thanks a lot for persisting with debugging.