Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

org-super-agenda-test--:auto-map test failure #183

Open
Apteryks opened this issue Dec 16, 2020 · 4 comments
Open

org-super-agenda-test--:auto-map test failure #183

Apteryks opened this issue Dec 16, 2020 · 4 comments
Labels
discussion

Comments

@Apteryks
Copy link

Hello!

Attempting to update this package on Guix, I stumbled on the following test failure:

starting phase `check'
DEBUG: ARGS:  --debug --
DEBUG: Remaining args: 
DEBUG: Local time: Wed 16 Dec 2020 09:35:54 PM UTC
DEBUG: UTC: Wed 16 Dec 2020 09:35:54 PM UTC
DEBUG: TESTING...
Org version: 9.4.2
Test results loaded
Running 66 tests (2020-12-16 21:35:55+0000, selector ‘t’)
org-super-agenda-mode enabled.

   passed   1/66  org-super-agenda-test--:and (0.245904 sec)
org-super-agenda-mode enabled.

   passed   2/66  org-super-agenda-test--:anything (0.037873 sec)
org-super-agenda-mode enabled.

   passed   3/66  org-super-agenda-test--:auto-category (0.024705 sec)
org-super-agenda-mode enabled.

   passed   4/66  org-super-agenda-test--:auto-group (0.038509 sec)
org-super-agenda-mode enabled.

Test org-super-agenda-test--:auto-map backtrace:
  signal(error ("Empty result for body:307f68fcbbad5bc27a9bb104acbf...
  error("Empty result for body:%s\nSTORED-RESULT:%s\nNEW-RESU..." "307
  (if (and stored-result new-result) nil (error "Empty result for body
  (or (equal stored-result new-result) (if (and stored-result new-resu
  (let ((stored-result (ht-get org-super-agenda-test-results body-grou
  (if org-super-agenda-test-show-results nil (let ((stored-result (ht-
  (let ((body-groups-hash (secure-hash 'md5 (format "%S" (list '(org-a
  (progn (if (> (ht-size org-super-agenda-test-results) 0) nil (org-su
  (setq value-125 (progn (if (> (ht-size org-super-agenda-test-results
  (unwind-protect (setq value-125 (progn (if (> (ht-size org-super-age
  (if (unwind-protect (setq value-125 (progn (if (> (ht-size org-super
  (let (form-description-126) (if (unwind-protect (setq value-125 (pro
  (let ((value-125 (gensym "ert-form-evaluation-aborted-"))) (let (for
  (closure (t) nil (let ((value-125 (gensym "ert-form-evaluation-abort
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name org-super-agenda-test--:auto-map :doc
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests ... :test-map
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  command-line-1(("--eval" "(message \"Org version: %s\" (org-version)
  command-line()
  normal-top-level()
Test org-super-agenda-test--:auto-map condition:
    (error "Empty result for body:307f68fcbbad5bc27a9bb104acbf6997
STORED-RESULT:nil
NEW-RESULT:Day-agenda (W27):
Wednesday   5 July 2017

 Face: org-agenda-calendar-sexp
  test:        7:02...... Sunrise (12:04 of daylight)
  test:       19:07...... Sunset 

 Face: org-scheduled-previously
  ambition:   Sched. 1x:  TODO [#A] Skype with president of Antarctica                             :universe:ambition:world::meetings:

 Face: org-scheduled-today
  test:       18:00...... Scheduled:  TODO Order a pizza                                                                 :food:dinner:
  test:       Scheduled:  TODO [#B] Fix flux capacitor                                                  :spaceship:shopping:@computer:
  test:       Scheduled:  TODO Shop for groceries                                                                :food:shopping:@town:
  ideas:      Scheduled:  SOMEDAY Rewrite Emacs in Common Lisp                            :Emacs:elisp:computers:software:programming:
  test:       Scheduled:  TODO [#C] Get haircut                                                                       :personal:@town:
  ambition:   TODO Practice leaping tall                     !                                           :universe:ambition::personal:

 Face: org-upcoming-deadline
  ambition:   In   2 d.:  TODO [#A] Take over the world                                                     :universe:ambition::world:
  ambition:   In  10 d.:  TODO [#A] Take over the universe                                                         :universe:ambition:

 Face: org-upcoming-distant-deadline
  test:       In  27 d.:  TODO [#A] Spaceship lease                                                                  :bills:spaceship:
  ambition:   In   5 d.:  TODO [#B] Renew membership in supervillain club                                         :universe:ambition::
  test:       In  16 d.:  TODO [#B] Internet                                                                                   :bills:
  ambition:   In  53 d.:  WAITING Visit the moon                                                     :universe:ambition::space:travel:
  ambition:   In  77 d.:  TODO Visit Mars                                                     :universe:ambition::space:travel:planet:

 Face: org-warning
  test:       Deadline:   CHECK /r/emacs                                                                               :website:Emacs:

 Other items
               8:00...... ----------------
              10:00...... ----------------
              12:00...... now - - - - - - - - - - - - - - - - - - - - - - - - -
              12:00...... ----------------
              14:00...... ----------------
              16:00...... ----------------
              18:00...... ----------------
              20:00...... ----------------
")
   FAILED   5/66  org-super-agenda-test--:auto-map (0.024856 sec)
org-super-agenda-mode enabled.

   passed   6/66  org-super-agenda-test--:auto-planning (0.026880 sec)
org-super-agenda-mode enabled.

   passed   7/66  org-super-agenda-test--:auto-tags (0.025303 sec)
org-super-agenda-mode enabled.

   passed   8/66  org-super-agenda-test--:auto-ts (0.038638 sec)
org-super-agenda-mode enabled.

   passed   9/66  org-super-agenda-test--:category (0.038265 sec)
org-super-agenda-mode enabled.

   passed  10/66  org-super-agenda-test--:category-multi (0.024736 sec)
org-super-agenda-mode enabled.

   passed  11/66  org-super-agenda-test--:children-nil (0.038294 sec)
org-super-agenda-mode enabled.

   passed  12/66  org-super-agenda-test--:children-string (0.024958 sec)
org-super-agenda-mode enabled.

   passed  13/66  org-super-agenda-test--:children-t (0.038230 sec)
org-super-agenda-mode enabled.

   passed  14/66  org-super-agenda-test--:children-todo (0.025011 sec)
org-super-agenda-mode enabled.

   passed  15/66  org-super-agenda-test--:date (0.038042 sec)
org-super-agenda-mode enabled.

   passed  16/66  org-super-agenda-test--:deadline-after (0.025028 sec)
org-super-agenda-mode enabled.

   passed  17/66  org-super-agenda-test--:deadline-before (0.038469 sec)
org-super-agenda-mode enabled.

   passed  18/66  org-super-agenda-test--:deadline-future (0.024987 sec)
org-super-agenda-mode enabled.
   passed  19/66  org-super-agenda-test--:deadline-nil (0.037246 sec)
org-super-agenda-mode enabled.

   passed  20/66  org-super-agenda-test--:deadline-nil-agenda (0.024985 sec)
org-super-agenda-mode enabled.

   passed  21/66  org-super-agenda-test--:deadline-past (0.038632 sec)
org-super-agenda-mode enabled.

   passed  22/66  org-super-agenda-test--:deadline-t (0.025152 sec)
org-super-agenda-mode enabled.

   passed  23/66  org-super-agenda-test--:deadline-today (0.038798 sec)
org-super-agenda-mode enabled.

   passed  24/66  org-super-agenda-test--:discard (0.006047 sec)
org-super-agenda-mode enabled.

   passed  25/66  org-super-agenda-test--:effort< (0.038394 sec)
org-super-agenda-mode enabled.

   passed  26/66  org-super-agenda-test--:effort> (0.024929 sec)
org-super-agenda-mode enabled.

   passed  27/66  org-super-agenda-test--:file-path (0.038147 sec)
org-super-agenda-mode enabled.

   passed  28/66  org-super-agenda-test--:habit (0.025105 sec)
org-super-agenda-mode enabled.

   passed  29/66  org-super-agenda-test--:heading-regexp (0.038570 sec)
org-super-agenda-mode enabled.

   passed  30/66  org-super-agenda-test--:log (0.024789 sec)
org-super-agenda-mode enabled.

   passed  31/66  org-super-agenda-test--:not (0.038210 sec)
org-super-agenda-mode enabled.

   passed  32/66  org-super-agenda-test--:order (0.024982 sec)
org-super-agenda-mode enabled.

   passed  33/66  org-super-agenda-test--:order-multi (0.038345 sec)
org-super-agenda-mode enabled.

   passed  34/66  org-super-agenda-test--:pred (0.025472 sec)
org-super-agenda-mode enabled.

   passed  35/66  org-super-agenda-test--:priority (0.039220 sec)
org-super-agenda-mode enabled.

   passed  36/66  org-super-agenda-test--:priority< (0.025757 sec)
org-super-agenda-mode enabled.

   passed  37/66  org-super-agenda-test--:priority<= (0.039076 sec)
org-super-agenda-mode enabled.

   passed  38/66  org-super-agenda-test--:priority> (0.025865 sec)
org-super-agenda-mode enabled.

   passed  39/66  org-super-agenda-test--:priority>= (0.039209 sec)
org-super-agenda-mode enabled.

   passed  40/66  org-super-agenda-test--:regexp (0.024956 sec)
org-super-agenda-mode enabled.

   passed  41/66  org-super-agenda-test--:scheduled-after (0.038465 sec)
org-super-agenda-mode enabled.

   passed  42/66  org-super-agenda-test--:scheduled-before (0.025016 sec)
org-super-agenda-mode enabled.

   passed  43/66  org-super-agenda-test--:scheduled-future (0.039905 sec)
org-super-agenda-mode enabled.

   passed  44/66  org-super-agenda-test--:scheduled-nil (0.024881 sec)
org-super-agenda-mode enabled.

   passed  45/66  org-super-agenda-test--:scheduled-past (0.038335 sec)
org-super-agenda-mode enabled.

   passed  46/66  org-super-agenda-test--:scheduled-t (0.038723 sec)
org-super-agenda-mode enabled.

   passed  47/66  org-super-agenda-test--:scheduled-today (0.025100 sec)
org-super-agenda-mode enabled.

   passed  48/66  org-super-agenda-test--:tag (0.038430 sec)
org-super-agenda-mode enabled.

   passed  49/66  org-super-agenda-test--:tag-with-inheritance (0.024425 sec)
org-super-agenda-mode enabled.

   passed  50/66  org-super-agenda-test--:time-grid (0.024751 sec)
org-super-agenda-mode enabled.

   passed  51/66  org-super-agenda-test--:todo (0.038259 sec)
org-super-agenda-mode enabled.

   passed  52/66  org-super-agenda-test--agenda-with-grid-and-todo-with-children (0.043301 sec)
org-super-agenda-mode enabled.

   passed  53/66  org-super-agenda-test--auto-dir-name (0.038344 sec)
org-super-agenda-mode enabled.

   passed  54/66  org-super-agenda-test--auto-groups (0.025837 sec)
org-super-agenda-mode enabled.

   passed  55/66  org-super-agenda-test--auto-outline-path (0.038898 sec)
org-super-agenda-mode enabled.

   passed  56/66  org-super-agenda-test--auto-parent (0.025320 sec)
org-super-agenda-mode enabled.

   passed  57/66  org-super-agenda-test--auto-property (0.039044 sec)
org-super-agenda-mode enabled.

   passed  58/66  org-super-agenda-test--discard-with-2-regexps (0.024579 sec)
org-super-agenda-mode enabled.

   passed  59/66  org-super-agenda-test--forward-looking (0.039219 sec)
org-super-agenda-mode enabled.

   passed  60/66  org-super-agenda-test--heading-regexp (0.025122 sec)
org-super-agenda-mode enabled.

   passed  61/66  org-super-agenda-test--main-example (0.039689 sec)
org-super-agenda-mode enabled.

   passed  62/66  org-super-agenda-test--no-groups (0.025011 sec)
org-super-agenda-mode enabled.

   passed  63/66  org-super-agenda-test--priority<B-with-order-100 (0.039515 sec)
org-super-agenda-mode enabled.

   passed  64/66  org-super-agenda-test--priority>=B (0.025843 sec)
org-super-agenda-mode enabled.
   passed  65/66  org-super-agenda-test--someday-tags-view-Emacs (0.016425 sec)
org-super-agenda-mode enabled.

   passed  66/66  org-super-agenda-test--time-grid-with-unprioritized-order-100 (0.025172 sec)

Ran 66 tests, 65 results as expected, 1 unexpected (2020-12-16 21:35:57+0000, 2.403023 sec)

1 unexpected results:
   FAILED  org-super-agenda-test--:auto-map

command "test/run" "--debug" failed with status 1

It may be due to using dependencies newer than those listed in the Package-Requires header:

$ ./pre-inst-env guix show emacs-org-super-agenda
name: emacs-org-super-agenda
version: 1.2
outputs: out
systems: x86_64-linux i686-linux
dependencies: emacs-dash@2.17.0 emacs-f@0.20.0 emacs-ht@2.2 emacs-org@9.4.2 emacs-s@1.12.0 emacs-ts@0.2
+ util-linux@2.35.1
location: /home/maxim/src/guix/gnu/packages/emacs-xyz.scm:14859:2
homepage: https://github.com/alphapapa/org-super-agenda
license: GPL 3+
synopsis: Supercharged Org agenda  
description: This package allows items in the Org agenda to be grouped into sections while preserving
+ the structure imposed by any timestamps.

Thank you!
@alphapapa
Copy link
Owner

alphapapa commented Dec 16, 2020 •
edited
Loading

This happens on Emacs >= 27.1 because it hashes lambdas differently than Emacs <= 26.3, and since the test's argument is a lambda, that is what the test result is keyed on, so the test fails to find the test result in the test data.

This can't easily be fixed without either dropping support for Emacs <= 26.3 (which shouldn't be done for, probably, years to come), or making a likely convoluted change to the test suite to handle Emacs versions differently.

I might try the latter at some point. In the meantime, this test failure is harmless and a false negative.

@Apteryks
Copy link
Author

I disabled the test and pushed the package update in Guix with this commit: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=d6dd120be6a527494e9204fa227842db466171e6.

Thanks for the prompt feedback!

@alphapapa
Copy link
Owner

Thanks. A couple of comments:

+       (modify-phases %standard-phases
+         (add-after 'unpack 'prepare-for-tests.el
+           (lambda _
+             (make-file-writable "test/test.el")
+             (emacs-substitute-variables "test/test.el"
+               ("org-super-agenda-test-results-file"
+                (string-append (getcwd) "/test/results.el")))

What is the purpose of modifying that variable? I guess it's to correct the CWD while running the tests, however that seems like an awkward way of doing it, and it may indicate doing something wrong.

Specifically, I see earlier in the diff:

+       #:test-command '("test/run" "--debug")

That's not the way I'm running the tests now. Please see the Makefile in the repo. To run the tests, you should only need to run make tests from the repo root. If you use that instead, that variable substitution will probably not be required. (I should probably remove that old test/run script now that I'm using makem.sh here.)

Also:

+             ;; The following test fail (see:
+             ;; https://github.com/alphapapa/org-super-agenda/issues/183).

Would you please note in the comment that the test only fails on Emacs >= 27.1?

Thanks for your work on this.

@Apteryks
Copy link
Author

Thanks. A couple of comments:

+       (modify-phases %standard-phases
+         (add-after 'unpack 'prepare-for-tests.el
+           (lambda _
+             (make-file-writable "test/test.el")
+             (emacs-substitute-variables "test/test.el"
+               ("org-super-agenda-test-results-file"
+                (string-append (getcwd) "/test/results.el")))

What is the purpose of modifying that variable? I guess it's to correct the CWD while running the tests, however that seems like an awkward way of doing it, and it may indicate doing something wrong.

Without adapting the org-super-agenda-test-results-file variable, the test suite fails with:

Opening input file: No such file or directory, /tmp/guix-build-emacs-org-super-agenda-1.2.drv-0/source/test/test/results.el

I'm guessing that's due to the default value being resolved based on the location of the '.git' file in the project, which doesn't exist in the Guix checkout of the project (for reasons mentioned alphapapa/org-ql#166 (comment)).

Specifically, I see earlier in the diff:

+       #:test-command '("test/run" "--debug")

That's not the way I'm running the tests now. Please see the Makefile in the repo. To run the tests, you should only need to run make tests from the repo root. If you use that instead, that variable substitution will probably not be required. (I should probably remove that old test/run script now that I'm using makem.sh here.)

I'd like to, but I can't, because the makem.sh depends on the git metadata being present.

Also:

+             ;; The following test fail (see:
+             ;; https://github.com/alphapapa/org-super-agenda/issues/183).

Would you please note in the comment that the test only fails on Emacs >= 27.1?

There's currently only one version of Emacs package in Guix, and that's 27.1. So adding this information wouldn't bring much in this context.

Thanks for your work on this.

My pleasure!

Thanks for your helpful replies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
discussion
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@alphapapa @Apteryks