Depending on a lisp package in lispPackages doesn't make the package's asd files available to ASDF

[SquircleSpace]

Issue description

I have a simple derivation that depends on some lispPackages. Here's a distilled version.

# default.nix
{ pkgs ? import ../nixpkgs {} }:
pkgs.stdenv.mkDerivation rec {
  name = "test";
  src = ./.;
  env = pkgs.buildEnv { name = name; paths = buildInputs; };
  buildInputs = [
    pkgs.lispPackages.fset
    sbcl
  ];
  LD_LIBRARY_PATH = "${pkgs.stdenv.lib.makeLibraryPath buildInputs}";
}

Before 4db842f, I could run nix-shell and the following command would work.

sbcl --eval '(require :asdf)' --eval '(asdf:load-system :fset)'

Post 4db842f, the above command does not successfully load fset.

Technical details

 - system: `"x86_64-linux"`
 - host os: `Linux 4.9.80, NixOS, 17.09.2968.b34a5f6d874 (Hummingbird)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 1.11.16`
 - channels(brad): `""`
 - channels(root): `"nixos-17.09.2968.b34a5f6d874"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos/nixpkgs`

[7c6f434c]

There are three questions here, I guess. The motivation for the chage was provided by upstream

1. How to get stuff working in the current state.

common-lisp.sh --eval '(require :asdf)' --eval '(asdf:load-system :fset)'

or
.

cl-wrapper.sh sbcl --eval '(require :asdf)' --eval '(asdf:load-system :fset)'

(these two are basically the same command).

2. At what level should that be wrapped and how the result should look like. I am not sure; I am mixing my use of unwrapped SBCL with QuickLisp or other solutions, and the use of lispPackages; but maybe it is time to create lispPackages.sbcl with a wrapper cl-wrapper.sh ${pkgs.sbcl}/bin/sbcl "$@".

3. Where, when and how that should be documented. Well, descibing a Common Lisp wrapper without describing lispPackages seems strange, and it always feels like the interface is going to change significantly any moment…

[SquircleSpace]

Ah! Excellent!

How do other compiler toolchains work? Do they have an unwrapped compiler and a wrapped compiler? I mean, at some level they must have an unwrapped compiler (gotta wrap something!), but is the unwrapped compiler exposed as a top-level package like SBCL? I'm still pretty new to nix -- sorry if this is a basic question.

[7c6f434c]

Frankly, Lisp support in Nixpkgs is quite immature, and your work has been a significant step forward. (Thanks again for your efforts) GCC is built as an unwrapped compiler, which is accessible as `gcc.cc` package, then wrapped with `wrapGCC` to obtain the top-level `cc`. With GHC my understanding is that it is recommended to use `ghcWithPackages` expression that builds a package-set-specific wrappers. Actually, the current clwrapper package has a property that makes little sense for many other languages: it is possible, although sometimes annoying, to write code that will handle lisp packages correctly on multiple Lisp implementations.

[SquircleSpace]

Got it. As I mentioned back in #32344, I've been thinking about something like sbclWithPackages as a solution for pre-compilation issues.

[aakropotkin]

Reviving this because there are so few posts about CLISP on NixOS and this took me a really long time to debug. Hopefully it saves someone.

My dirty solution to use sbcl (NOT common-lisp.sh!):

1. Added a million lispPackage dependencies to my user's profile.
2. Installed Quicklisp and initialized with quicklisp init.
3. Filled out SBCLRC with the following

;-*- mode: lisp -*- file: .sbclrc

(defun in-home-dir (subdir)
  (merge-pathnames subdir (user-homedir-pathname)))

;; Lets make sure we didn't get some wonky temporary HOME
(princ #\newline)
(princ (concatenate 'string "Home: " (namestring (user-homedir-pathname))))

#-quicklisp
(let ((quicklisp-init (in-home-dir "quicklisp/setup.lisp")))
  (when (probe-file quicklisp-init)
    (load quicklisp-init)))
#-asdf (require :asdf)

(ql:quickload :linedit :silent t)
(ql:quickload :cffi :silent t)
(ql:quickload :cl-ppcre :silent t)

;; Pretty this up with `let` and what not...
(defparameter *home-libs* (in-home-dir ".nix-profile/lib/"))
(defparameter *nix-cl-settings-dir*
  (in-home-dir ".nix-profile/lib/common-lisp-settings"))
(defparameter *nix-cl-settings-scripts*
  (directory (concatenate 'string (namestring *nix-cl-settings-dir*)
                          "/*-path-config.sh")))

(defun source-file (f)
  (concatenate 'string "source " (namestring f) "; "))

(defparameter *nix-cl-libs*
  (cl-ppcre:split ":" (uiop:run-program
                       (format nil "~{~A~}"
                               (nconc
                                (mapcar #'source-file *nix-cl-settings-scripts*)
                                '("echo \"${NIX_LISP_LD_LIBRARY_PATH}\";")))
                       :output
                       '(:string :stripped t))))

(pushnew *home-libs* cffi:*foreign-library-directories*)
(loop for l in *nix-cl-libs* do (pushnew (car(directory l))
                                         cffi:*foreign-library-directories*))


;;; Check for `--no-linedit` command-line option.
;; For any of this to work you need to have installed `linedit`
;; To do so run `(ql:quickload "linedit")`
(if (member "--no-linedit" sb-ext:*posix-argv* :test 'equal)
  (setf sb-ext:*posix-argv*
        (remove "--no-linedit" sb-ext:*posix-argv* :test 'equal))
  (when (interactive-stream-p *terminal-io*)
    (require :sb-aclrepl)
    (linedit:install-repl :wrap-current t)))

I swear that this is what quicklisp-to-nix is supposed to do, but it refused to work for me on anything with library dependencies, so I resorted to this. This also conveniently "just works" with Emacs and all that jazz without any fussing. cl+ssl FINALLY works with this method unlike common-lisp.sh and quicklisp-to-nix.

[stale]

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

1. Search for maintainers and people that previously touched the related code and @ mention them in a comment.
2. Ask on the NixOS Discourse.
3. Ask on the #nixos channel on irc.freenode.net.

[floyza]

Still important to me

[doyougnu]

Reviving this because there are so few posts about CLISP on NixOS and this took me a really long time to debug. Hopefully it saves someone.

My dirty solution to use sbcl (NOT common-lisp.sh!):
....

This really should be in the documentation somewhere, even if its just a work around for the moment. I have no idea what common-lisp.sh is supposed to do, nor clwrapper.sh or how to use them. At least with the above solution I can get hacking with common lisp and sbcl!

[doyougnu]

FYI for anyone who comes after. I got SLIME to work through nix-shell in a thoroughly hacky way:

1. you need the nix-sandbox package
2. Then you need to set inferior-lisp-program in a .dir-locals.el in your project root or wherever you want:

((lisp-mode . ((eval . (progn
                         (setq-local
                         inferior-lisp-program (concat
                                                "nix-shell "
                                                (nix-find-sandbox ".")
                                                " --run 'sbcl'"
                                                )))))))

(nix-find-sanbox ".") looks for shell.nix or default.nix then the rest is just constructing nix-shell --run 'sbcl'. Note that this doesn't actually use the sandboxes from nix-sandbox. You'll know you've succeeded when slime loads and you see:

This is SBCL 2.1.2.nixos, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.

If you have the this in your .sbclrc then you should be able to load slime and do:

; SLIME 2.26.1
CL-USER> (ql:quickload '("hunchentoot"))
To load "hunchentoot":
  Load 1 ASDF system:
    hunchentoot
; Loading "hunchentoot"
.
("hunchentoot")

and see that libssl successfully loaded. Note that I have openssl.dev installed via home-manager, but hunchentoot is installed via the shell.nix:

;; shell.nix
let
  description = "A very basic flake";
  pkgs = import <nixpkgs> { system = "x86_64-linux"; };
  in
  with pkgs;
  mkShell {
      buildInputs = [sbcl lispPackages.hunchentoot];
    }

Any suggestions for a better way appreciated.

[stale]

I marked this as stale due to inactivity. → More info