buttercup failures on master

[bremner]

Checklist

• [x ] I have checked existing issues for potential duplicates before creating this one.
• [x ] I have read the [Troubleshooting guide][].

Bug description

Running "buttercup -L ." with emacs 28.2+1-9 and buttercup 1.26-4
on Debian on commit 15f0759 yields the attached failures.

I also tried

EMACS_INHIBIT_AUTOMATIC_NATIVE_COMPILATION=t emacs -q -batch -L . -l buttercup -f buttercup-run-discover

There are nonetheless the same messages about native compilation (loading) failing.

This is also reported as https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1028725

buttercup.log

[jcs090218]

It should resolve in #2002.

[bremner]

Jen-Chieh Shen ***@***.***> writes:

It should resolve in #2002.

That resolves two of the failing tests, but there remain two trampoline-related failures. At this point I don't know if they are flycheck specific, or more general issues related to emacs 28 and native compilation (or to buttercup). ======================================== Utilities flycheck-buffer-saved-p considers an unmodified buffer with backing file saved Traceback (most recent call last): spy-on(buffer-file-name :and-return-value "test-buffer-name") buttercup--spy-on-and-call-replacement(buffer-file-name (lambda (&rest arg... comp-subr-trampoline-install(buffer-file-name) comp-trampoline-search(buffer-file-name) native-elisp-load("/home/bremner/.emacs.d/eln-cache/28.2-8532cd27/subr--tr... error: (native-lisp-load-failed "file does not exists" "/home/bremner/.emacs.d/eln-cache/28.2-8532cd27/subr--trampoline-6275666665722d66696c652d6e616d65_buffer_file_name_0.eln") ======================================== Utilities flycheck-buffer-saved-p considers a modified buffer with backing file unsaved Traceback (most recent call last): spy-on(buffer-file-name :and-return-value "test-buffer-name") buttercup--spy-on-and-call-replacement(buffer-file-name (lambda (&rest arg... comp-subr-trampoline-install(buffer-file-name) comp-trampoline-search(buffer-file-name) native-elisp-load("/home/bremner/.emacs.d/eln-cache/28.2-8532cd27/subr--tr... error: (native-lisp-load-failed "file does not exists" "/home/bremner/.emacs.d/eln-cache/28.2-8532cd27/subr--trampoline-6275666665722d66696c652d6e616d65_buffer_file_name_0.eln")

[jcs090218]

Interesting... Our CI does not report anything like this. See https://github.com/flycheck/flycheck/actions/runs/4238492328/jobs/7365623154#step:5:878

Edit: How do you invoke the tests? Through Cask, Eask, makem.sh?

[sergiodj]

Debian invokes buttercup -L . directly. I'm also investigating this problem and I noticed something interesting. If I get inside the chroot where the package is being built, set HOME to a proper location (e.g., /tmp/), create the .emacs.d directory, and run buttercup again, I see the same error:

...
========================================
Utilities flycheck-buffer-saved-p considers an unmodified buffer with backing file saved

Traceback (most recent call last):
  spy-on(buffer-file-name :and-return-value "test-buffer-name")
  buttercup--spy-on-and-call-replacement(buffer-file-name (lambda (&rest args) nil '"test-buffer-name"))
  comp-subr-trampoline-install(buffer-file-name)
  comp-trampoline-search(buffer-file-name)
  native-elisp-load("/tmp/.emacs.d/eln-cache/28.2-b25a1857/subr--trampoline-6275666665722d66696c652d6e616d65_buffer_file_name_0.eln")
error: (native-lisp-load-failed "file does not exists" "/tmp/.emacs.d/eln-cache/28.2-b25a1857/subr--trampoline-6275666665722d66696c652d6e616d65_buffer_file_name_0.eln")

========================================
Utilities flycheck-buffer-saved-p considers a modified buffer with backing file unsaved

Traceback (most recent call last):
  spy-on(buffer-file-name :and-return-value "test-buffer-name")
  buttercup--spy-on-and-call-replacement(buffer-file-name (lambda (&rest args) nil '"test-buffer-name"))
  comp-subr-trampoline-install(buffer-file-name)
  comp-trampoline-search(buffer-file-name)
  native-elisp-load("/tmp/.emacs.d/eln-cache/28.2-b25a1857/subr--trampoline-6275666665722d66696c652d6e616d65_buffer_file_name_0.eln")
error: (native-lisp-load-failed "file does not exists" "/tmp/.emacs.d/eln-cache/28.2-b25a1857/subr--trampoline-6275666665722d66696c652d6e616d65_buffer_file_name_0.eln")
...

However, if I inspect the contents of $HOME/.emacs.d, I see:

# tree $HOME/.emacs.d/
/tmp/.emacs.d/
└── eln-cache
    └── 28.2-b25a1857
        └── subr--trampoline-66696c652d6578697374732d70_file_exists_p_0.eln

3 directories, 1 file

So there is a .eln file there, but it's a different one than the one expected by the test. I'm still trying to figure out what's going on here.

[sergiodj]

After some more investigation, I found that the problem is not with flycheck, but probably with buttercup: jorgenschaefer/emacs-buttercup#230

I'm still puzzled as to why these failures are not reproducible in your CI, though.

[jcs090218]

May be something went wrong with .eln? I haven't tried native compilation, but I've heard is unstable. 😕 Maybe that was the issue?

[sergiodj]

It's a new feature but I wouldn't say it's unstable. However, it seems like the Emacs your CI is using doesn't have native compilation enabled, which explains why the error doesn't manifest there.

[jcs090218]

It's a new feature but I wouldn't say it's unstable.

That's good to know. :)

Yeah, we don't tested with native compilation. That's indeed true. 😅

[bbatsov]

The tests are working for me, both locally and on the CI, so I close the issue for now.