Strange "PACKAGE-INFERRED-SYSTEM" error when some dir in ql:*local-project-directories* indexes asdf.asd itself

[joaotavora]

Hi,

This is a minimal test case with SBCL 1.2.11 running on Mac OS 10.9. If I load this minimal file I get a strange error described at the bottom.

If I remove the line adding my projects dir to ql:*local-project-directories*, everything works dandy. But after it is evaluated, "raw" asdf loading of systems is broken.

It doesn't happen on Linux I also suspect it doesn't happen with old Macosx SBCL versions bundling older ASDF's. Pretty sure it also happens with a much more recent sbcl compiled from source but I can't easily get compilation working again to double check.

Here's the file:

(load "~/quicklisp/setup.lisp")
(push #p"~/Source/Lisp/" ql:*local-project-directories*) ; got lots of
                                                         ; lisp
                                                         ; projects
                                                         ; here, but
                                                         ; no "dummy"
(require 'asdf) ; for clarity, not really needed
(push (pathname "~/tmp/dummy/")
      asdf:*central-registry*) ; created with quickproject
(asdf:load-system :dummy)

And here is the error

$ sbcl --no-sysinit --no-userinit --load mini-test.lisp
This is SBCL 1.2.11, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.

debugger invoked on a ASDF/FIND-COMPONENT:MISSING-DEPENDENCY:
  Component "asdf/defsystem" not found, required by
  #<PACKAGE-INFERRED-SYSTEM "asdf">

Type HELP for debugger help, or (SB-EXT:EXIT) to exit from SBCL.

restarts (invokable by number or by possibly-abbreviated name):
  0: [RETRY                        ] Retry loading asdf/defsystem.
  1:                                 Retry ASDF operation.
  2: [CLEAR-CONFIGURATION-AND-RETRY] Retry ASDF operation after resetting the
                                     configuration.
  3: [RETRY                        ] Retry EVAL of current toplevel form.
  4: [CONTINUE                     ] Ignore error and continue loading file "/private/tmp/mini-test.lisp".
  5: [ABORT                        ] Abort loading file "/private/tmp/mini-test.lisp".
  6:                                 Ignore runtime option --load "mini-test.lisp".
  7:                                 Skip rest of --eval and --load options.
  8:                                 Skip to toplevel READ/EVAL/PRINT loop.
  9: [EXIT                         ] Exit SBCL (calling #'EXIT, killing the process).

(ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> "asdf/defsystem" NIL)
0] backtrace

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {1002DE6773}>
0: (ASDF/FIND-COMPONENT:RESOLVE-DEPENDENCY-NAME #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> "asdf/defsystem" NIL)
1: (ASDF/PLAN:MAP-DIRECT-DEPENDENCIES #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:PREPARE-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> #<CLOSURE (LAMBDA (ASDF/PLAN::O ASDF/PLAN::C) :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A9579B}>)
2: ((LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION))
3: ((:METHOD ASDF/PLAN:CALL-WHILE-VISITING-ACTION (ASDF/PLAN:PLAN-TRAVERSAL T T T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:PREPARE-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> #<CLOSURE (LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A9573B}>) [fast-method]
4: (ASDF/PLAN:MAP-DIRECT-DEPENDENCIES #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:PREPARE-OP > #<ASDF/COMPONENT:MODULE "asdf" "build"> #<CLOSURE (LAMBDA (ASDF/PLAN::O ASDF/PLAN::C) :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A9285B}>)
5: ((LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION))
6: ((:METHOD ASDF/PLAN:CALL-WHILE-VISITING-ACTION (ASDF/PLAN:PLAN-TRAVERSAL T T T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:PREPARE-OP > #<ASDF/COMPONENT:MODULE "asdf" "build"> #<CLOSURE (LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A927FB}>) [fast-method]
7: (ASDF/PLAN:MAP-DIRECT-DEPENDENCIES #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:PREPARE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "asdf" "build" "asdf"> #<CLOSURE (LAMBDA (ASDF/PLAN::O ASDF/PLAN::C) :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A8E83B}>)
8: ((LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION))
9: ((:METHOD ASDF/PLAN:CALL-WHILE-VISITING-ACTION (ASDF/PLAN:PLAN-TRAVERSAL T T T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:PREPARE-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "asdf" "build" "asdf"> #<CLOSURE (LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A8E7DB}>) [fast-method]
10: (ASDF/PLAN:MAP-DIRECT-DEPENDENCIES #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "asdf" "build" "asdf"> #<CLOSURE (LAMBDA (ASDF/PLAN::O ASDF/PLAN::C) :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A8A42B}>)
11: ((LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION))
12: ((:METHOD ASDF/PLAN:CALL-WHILE-VISITING-ACTION (ASDF/PLAN:PLAN-TRAVERSAL T T T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/LISP-ACTION:CL-SOURCE-FILE "asdf" "build" "asdf"> #<CLOSURE (LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A8A10B}>) [fast-method]
13: (ASDF/PLAN:MAP-DIRECT-DEPENDENCIES #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/COMPONENT:MODULE "asdf" "build"> #<CLOSURE (LAMBDA (ASDF/PLAN::O ASDF/PLAN::C) :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A8853B}>)
14: ((LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION))
15: ((:METHOD ASDF/PLAN:CALL-WHILE-VISITING-ACTION (ASDF/PLAN:PLAN-TRAVERSAL T T T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/COMPONENT:MODULE "asdf" "build"> #<CLOSURE (LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A87E4B}>) [fast-method]
16: (ASDF/PLAN:MAP-DIRECT-DEPENDENCIES #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> #<CLOSURE (LAMBDA (ASDF/PLAN::O ASDF/PLAN::C) :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A85E5B}>)
17: ((LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION))
18: ((:METHOD ASDF/PLAN:CALL-WHILE-VISITING-ACTION (ASDF/PLAN:PLAN-TRAVERSAL T T T)) #<ASDF/PLAN:SEQUENTIAL-PLAN {1004A60D63}> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> #<CLOSURE (LAMBDA NIL :IN ASDF/PLAN:TRAVERSE-ACTION) {1004A846CB}>) [fast-method]
19: ((:METHOD ASDF/PLAN:MAKE-PLAN (T ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) NIL #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> :VERBOSE NIL) [fast-method]
20: ((:METHOD ASDF/OPERATE:OPERATE (ASDF/OPERATION:OPERATION ASDF/COMPONENT:COMPONENT)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> :VERBOSE NIL) [fast-method]
21: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> :VERBOSE NIL)
22: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
23: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) #<ASDF/LISP-ACTION:LOAD-OP > #<ASDF/PACKAGE-INFERRED-SYSTEM:PACKAGE-INFERRED-SYSTEM "asdf"> :VERBOSE NIL) [fast-method]
24: ((SB-PCL::EMF ASDF/OPERATE:OPERATE) #<unused argument> #<unused argument> ASDF/LISP-ACTION:LOAD-OP :ASDF :VERBOSE NIL)
25: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
26: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP :ASDF :VERBOSE NIL) [fast-method]
27: (ASDF/OPERATE:LOAD-SYSTEM :ASDF :VERBOSE NIL)
28: (ASDF/UPGRADE:UPGRADE-ASDF)
29: ((LAMBDA NIL :IN ASDF/OPERATE:OPERATE))
30: (ASDF/CACHE:CALL-WITH-ASDF-CACHE #<CLOSURE (LAMBDA NIL :IN ASDF/OPERATE:OPERATE) {1004A48CFB}> :OVERRIDE NIL :KEY NIL)
31: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP :DUMMY) [fast-method]
32: ((:METHOD ASDF/OPERATE:OPERATE :AROUND (T T)) ASDF/LISP-ACTION:LOAD-OP :DUMMY) [fast-method]
33: (ASDF/OPERATE:LOAD-SYSTEM :DUMMY)
34: (SB-INT:SIMPLE-EVAL-IN-LEXENV (ASDF/OPERATE:LOAD-SYSTEM :DUMMY) #<NULL-LEXENV>)
35: (EVAL-TLF (ASDF/OPERATE:LOAD-SYSTEM :DUMMY) 3 #<NULL-LEXENV>)
36: ((FLET SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (ASDF/OPERATE:LOAD-SYSTEM :DUMMY) 3)
37: ((LAMBDA (SB-KERNEL:FORM &KEY SB-FASL::CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) (ASDF/OPERATE:LOAD-SYSTEM :DUMMY) :CURRENT-INDEX 3)
38: (SB-C::%DO-FORMS-FROM-INFO #<CLOSURE (LAMBDA (SB-KERNEL:FORM &KEY SB-FASL::CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) {1002DFA8CB}> #<SB-C::SOURCE-INFO {1002DFA883}> SB-C::INPUT-ERROR-IN-LOAD)
39: (SB-INT:LOAD-AS-SOURCE #<SB-INT:FORM-TRACKING-STREAM for "file /private/tmp/mini-test.lisp" {1002DF8563}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
40: ((FLET SB-FASL::LOAD-STREAM :IN LOAD) #<SB-INT:FORM-TRACKING-STREAM for "file /private/tmp/mini-test.lisp" {1002DF8563}> NIL)
41: (LOAD #P"mini-test.lisp" :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST T :EXTERNAL-FORMAT :DEFAULT)
42: (SB-IMPL::PROCESS-EVAL/LOAD-OPTIONS ((:LOAD . "mini-test.lisp")))
43: (SB-IMPL::TOPLEVEL-INIT)
44: ((FLET #:WITHOUT-INTERRUPTS-BODY-85 :IN SAVE-LISP-AND-DIE))
45: ((LABELS SB-IMPL::RESTART-LISP :IN SAVE-LISP-AND-DIE))

0] 

l

[joaotavora]

UPDATE: It does happen in linux. The reason it didn't earlier is that I had a copy of the actual ASDF project under ~/Source/Lisp. Also, it happens with a much more recent sbcl 1.4.1.

[joaotavora]

OK, I have found the culprit. The system-index.txt in the ~/Source/Lisp directory of my example indexes a copy of asdf.asd itself. This particular copy apparently conflicts with the version bundled with sbcl causing the very strange error. The line in question was this

/home/capitaomorte/Source/Lisp/portacle/build/asdf/asdf.asd

This line appeared as a consequence of me downloading the portacle project into that directory.
If I remove the line and/or delete the directory, everything's fine.

Also, as I mentioned in the previous comment, the error is also averted if there is another mention to another asdf.asd file that happens to not be problematic. That was the case with some asdf version I had downloaded before.

Not really sure if this is a bug in ASDF or quicklisp's local project searcher, which maybe should skip any asdf.asd files it finds when crawling ql:*local-project-directories*

[joaotavora]

quicklisp's local project searcher, which maybe should skip any asdf.asd files it finds when crawling ql:local-project-directories

Or maybe it can refrain from even trying to load ASDF. This change to local-projects.lisp fixes the problem:

diff -u --label /home/capitaomorte/quicklisp/quicklisp/local-projects.lisp --label \#\<buffer\ local-projects.lisp\> /home/capitaomorte/quicklisp/quicklisp/local-projects.lisp /tmp/buffer-content-TJSMAW
--- /home/capitaomorte/quicklisp/quicklisp/local-projects.lisp
+++ #<buffer local-projects.lisp>
@@ -95,13 +95,15 @@
 to use the local project directory and cache to find systems."
   ;; as a special case, don't attempt to load ASDF itself, since that
   ;; probably spells trouble
-  (dolist (directory *local-project-directories*)
-    (when (probe-directory directory)
-      (let ((system-index (ensure-system-index directory)))
-        (when system-index
-          (let ((system (find-system-in-index system-name system-index)))
-            (when system
-              (return system))))))))
+  (unless (string= (string-upcase system-name)
+                   "ASDF")
+    (dolist (directory *local-project-directories*)
+      (when (probe-directory directory)
+        (let ((system-index (ensure-system-index directory)))
+          (when system-index
+            (let ((system (find-system-in-index system-name system-index)))
+              (when system
+                (return system)))))))))
 
 (defun list-local-projects ()
   "Return a list of pathnames to local project system files."

Diff finished.  Wed Apr 11 22:37:38 2018

[joaotavora]

ping? I'm still getting bit by this from time to time. To reproduce it, just download any project that has a problematic asdf.asdfile somewhere inside its tree into your ql local projects directory (portacle is a good example). Then try to load anything with vanilla ASDF.