Compile error (possibly with stable-set-difference)

[Ambrevar]

I get the following compilation error in GitHub's CI when using CCL on the latest Serapeum (https://github.com/atlas-engineer/nyxt/actions/runs/4807278934/jobs/8555838069?pr=2934):

 ;Compiler warnings for "home:work;nyxt;nyxt;_build;serapeum;lists.lisp.newest" :
caught error #<SIMPLE-ERROR #x3020079D05ED>
Serious errors encountered during compilation of "/home/runner/work/nyxt/nyxt/_build/serapeum/lists.lisp"
;   In MEM: Illegal reference to lexically defined variable #:UTEST-NOT344863.
;   In MEM: Illegal reference to lexically defined variable #:UTEST344862.
;   In MEM: Illegal reference to lexically defined variable #:UKEY344861.
;   In STABLE-SET-DIFFERENCE: Invalid program: Illegal reference to lexically defined variable #:UKEY344861.
;   In STABLE-SET-DIFFERENCE: Unused lexical variable #:UTEST-NOT344863
;   In STABLE-SET-DIFFERENCE: Unused lexical variable #:UTEST344862
;   In STABLE-SET-DIFFERENCE: Unused lexical variable #:UKEY344861
Error: Process completed with exit code 17.

Interestingly I cannot reproduce locally. Any idea?

[aadcg]

@Ambrevar I'm able to reproduce locally. Notice that Guix's version of serapeum is a bit old. You need to ensure that you're running the tests locally with the right version. For example, by cloning serapeum at ~/common-lisp/serapeum/.

 Did 1812 checks.
    Pass: 1810 (99%)
    Skip: 0 ( 0%)
    Fail: 2 ( 0%)

 Failure Details:
 --------------------------------
 STABLE-SET-DIFFERENCE in LISTS []: 
      Unexpected Error: #<CCL::SIMPLE-PROGRAM-ERROR #x302002DB1ADD>
Invalid program: In the form ((LIST1 (LAMBDA NIL (COERCE (RANGE 100) 'LIST))) (LIST2 (LAMBDA NIL (SHUFFLE (FILTER-MAP (LAMBDA (X) (WHICHEVER X NIL)) LIST1))))), (LIST1 (LAMBDA NIL (COERCE (RANGE 100) 'LIST))) is not a symbol or lambda expression..
 --------------------------------
 --------------------------------
 LENGTH-FNS in SEQUENCES []: 
      Unexpected Error: #<CCL::SIMPLE-PROGRAM-ERROR #x302002D6A69D>
Invalid program: In the form ((LEN (LAMBDA NIL (RANDOM-IN-RANGE 2 10))) (SEQ-LENS (LAMBDA NIL (LOOP REPEAT LEN COLLECT (RANDOM 10)))) (SEQ-TYPES (LAMBDA NIL (LOOP REPEAT LEN COLLECT (RANDOM-ELT '(LIST VECTOR INTEGER)))))), (LEN (LAMBDA NIL (RANDOM-IN-RANGE 2 10))) is not a symbol or lambda expression..
 --------------------------------

[aadcg]

Sorry, my above comment was with respect to testing serapeum on CCL.

With respect to loading serapeum on CCL, the subject of this issue, I can't reproduce it locally as well. I only get some benign warnings.

;Compiler warnings for "home:common-lisp;serapeum;clos.lisp.newest" :
;   In an anonymous lambda form at position 3807: Variable DEFAULT not ignored.
; Warning: Lisp compilation had style-warnings while compiling #<CL-SOURCE-FILE "serapeum" "level0" "clos">
; While executing: UIOP/LISP-BUILD:CHECK-LISP-COMPILE-WARNINGS, in process sly-channel-1-mrepl-remote-1(11).
;Compiler warnings for "home:common-lisp;serapeum;strings.lisp.newest" :
;   In DO-STRINGS: Unused lexical variable BODY
;   In DO-STRINGS: Unused lexical variable MORE?
;   In DO-STRINGS: Unused lexical variable S
;   In STRING-JOIN: Unused lexical variable MORE?
;   In STRING-JOIN: Unused lexical variable MORE?
; Warning: Lisp compilation had style-warnings while compiling #<CL-SOURCE-FILE "serapeum" "level1" "strings">
; While executing: UIOP/LISP-BUILD:CHECK-LISP-COMPILE-WARNINGS, in process sly-channel-1-mrepl-remote-1(11).

[Ambrevar]

In GitHub's CI sometimes warnings are reported as error (don't quite know why).

[ruricolist]

@Ambrevar I'm able to reproduce locally. Notice that Guix's version of serapeum is a bit old. You need to ensure that you're running the tests locally with the right version. For example, by cloning serapeum at ~/common-lisp/serapeum/.

 Did 1812 checks.
    Pass: 1810 (99%)
    Skip: 0 ( 0%)
    Fail: 2 ( 0%)

 Failure Details:
 --------------------------------
 STABLE-SET-DIFFERENCE in LISTS []: 
      Unexpected Error: #<CCL::SIMPLE-PROGRAM-ERROR #x302002DB1ADD>
Invalid program: In the form ((LIST1 (LAMBDA NIL (COERCE (RANGE 100) 'LIST))) (LIST2 (LAMBDA NIL (SHUFFLE (FILTER-MAP (LAMBDA (X) (WHICHEVER X NIL)) LIST1))))), (LIST1 (LAMBDA NIL (COERCE (RANGE 100) 'LIST))) is not a symbol or lambda expression..
 --------------------------------
 --------------------------------
 LENGTH-FNS in SEQUENCES []: 
      Unexpected Error: #<CCL::SIMPLE-PROGRAM-ERROR #x302002D6A69D>
Invalid program: In the form ((LEN (LAMBDA NIL (RANDOM-IN-RANGE 2 10))) (SEQ-LENS (LAMBDA NIL (LOOP REPEAT LEN COLLECT (RANDOM 10)))) (SEQ-TYPES (LAMBDA NIL (LOOP REPEAT LEN COLLECT (RANDOM-ELT '(LIST VECTOR INTEGER)))))), (LEN (LAMBDA NIL (RANDOM-IN-RANGE 2 10))) is not a symbol or lambda expression..
 --------------------------------

This particular failure looks like it might be caused by an old version of fiveam that doesn't export for-all*.

[Ambrevar]

The problem is most likely due to the use of an old dependency. Nyxt's pipeline pins all deps, so this may be why.
Any idea which dep could interfere with the loading of lists.lisp?

[ruricolist]

I don't think lists.lisp is a dependency problem. I will test with CCL 1.12.1 as soon as I have the chance and see if that's the problem.

[ruricolist]

I've tested with CCL 1.12.1 but was not able to reproduce.

[ruricolist]

Actually, I think I know what the problem is. I've pushed a commit that should fix it.

[Ambrevar]

Wow, it worked! Thanks!

[Ambrevar]

Reopening, because it fails on ECL:

common-lisp/ecl/serapeum/tests/lists.lisp.
;;; OPTIMIZE levels: Safety=2, Space=0, Speed=3, Debug=0
;;;
;;; End of Pass 1.
;;; Finished compiling /gnu/store/4yabby7a7x09r2zn0k4cgvdliwm13gia-ecl-serapeum-0.0.0-10.2bd4b3c/share/common-lisp/ecl/serapeum/tests/lists.lisp.
;;;
An error occurred during initialization:
In form
((LIST1 (LAMBDA () (COERCE (RANGE 100) 'LIST)))
 (LIST2
  (LAMBDA () (SHUFFLE (FILTER-MAP (LAMBDA (X) (WHICHEVER X NIL)) LIST1)))))
FUNCTION: Not a valid argument (LIST1 (LAMBDA () (COERCE (RANGE 100) 'LIST)))..

Also the GitHub CI is reporting issues.

[ruricolist]

The ECL problem looks like old fiveam again. As for GitHub CI, which one do you mean?

[Ambrevar]

Guix has Fiveam 1.4.2, which is the latest stable. If it's really the problem, could it be possible to support the latest stable Fiveam? Otherwise we would need to package a special Fiveam just for Serapeum.

I'm talking about serapeum's CI: https://github.com/ruricolist/serapeum/actions/runs/4814783572

[aadcg]

Reopening, because it fails on ECL

It might be worth noting that the CI for ECL was disabled in commit db41855.

What's the motivation behind this commit @ruricolist? Thanks.

[ruricolist]

I believe ECL was consistently failing to install.

[ruricolist]

I've opened an issue on the fiveam repo suggesting a new stable release: lispci/fiveam#100. In the meantime I've removed the for-all* macro from the test suite.

[ruricolist]

Note that ccl-bin also regularly fails to install with Roswell in CI.

[Ambrevar]

Yes, I know this issue, to work around it you can specify the CCL version, for instance replace ccl-bin with ccl-bin/1.12.1.

Alternatively, use Guix instead of Roswell, it's much more reliable but won't work on macOS. Although in my experience Roswell does not support installing ECL nor CCL on macOS, so the difference would only matter for SBCL on macOS.

[ruricolist]

Thanks! Pinning the versions seems to have fixed the Roswell installs for CCL and ECL.

[Ambrevar]

Any idea about the ECL issue then?

[ruricolist]

Sorry, what ECL issue do you mean?

[Ambrevar]

Issue #149 (comment), but never mind, I tried again on Serapeum's master and it's working now. A fix must have landed since then.