Script fails due to missing jq binary

[sloane-shark]

Just tried this out and was faced with the following error:

;;; Computing Hangul syllable names.: replacing existing signature
/bin/sh: jq: command not found
Unhandled UIOP/RUN-PROGRAM:SUBPROCESS-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING
                                                         {7008820403}>:
  Subprocess with command "(/usr/bin/plutil -convert json -- orig && /usr/bin/plutil -convert json -- bare-wrapper && jq --argjson keys \"[\\\"CFBundleDevelopmentRegion\\\",\\\"CFBundleDocumentTypes\\\",\\\"CFBundleGetInfoString\\\",\\\"CFBundleIconFile\\\",\\\"CFBundleIdentifier\\\",\\\"CFBundleInfoDictionaryVersion\\\",\\\"CFBundleName\\\",\\\"CFBundleShortVersionString\\\",\\\"CFBundleURLTypes\\\",\\\"NSAppleEventsUsageDescription\\\",\\\"NSAppleScriptEnabled\\\",\\\"NSDesktopFolderUsageDescription\\\",\\\"NSDocumentsFolderUsageDescription\\\",\\\"NSDownloadsFolderUsageDescription\\\",\\\"NSPrincipalClass\\\",\\\"NSRemovableVolumesUsageDescription\\\",\\\"NSServices\\\",\\\"UTExportedTypeDeclarations\\\"]\" \"to_entries |[.[]| select(.key as \\$item| \\$keys | index(\\$item) >= 0) ] | from_entries\" < orig > filtered && (cat bare-wrapper filtered | jq -s add > final) && /usr/bin/plutil -convert xml1 -- final)"
 exited with error code 127

Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {7008820403}>
0: (SB-DEBUG::DEBUGGER-DISABLED-HOOK #<UIOP/RUN-PROGRAM:SUBPROCESS-ERROR {7005ED82D3}> #<unused argument> :QUIT T)
1: (SB-DEBUG::RUN-HOOK SB-EXT:*INVOKE-DEBUGGER-HOOK* #<UIOP/RUN-PROGRAM:SUBPROCESS-ERROR {7005ED82D3}>)
2: (INVOKE-DEBUGGER #<UIOP/RUN-PROGRAM:SUBPROCESS-ERROR {7005ED82D3}>)
3: (ERROR #<UIOP/RUN-PROGRAM:SUBPROCESS-ERROR {7005ED82D3}>)
4: ((LAMBDA (SH::C) :IN SH:RUN/NIL) #<UIOP/RUN-PROGRAM:SUBPROCESS-ERROR {7005ED82D3}>)
5: (SB-KERNEL::%SIGNAL #<UIOP/RUN-PROGRAM:SUBPROCESS-ERROR {7005ED82D3}>)
6: (CERROR "IGNORE-ERROR-STATUS" UIOP/RUN-PROGRAM:SUBPROCESS-ERROR :COMMAND #<(SIMPLE-ARRAY CHARACTER (819)) (/usr/bin/plutil -convert json -- orig && /usr/bin/plutil -convert json -- bare-wrapper && jq --argjson keys "[\"CFBundleDevelopmentRegion\",\"CFBundleDocumentTypes\",\"CFBundleGetInfoString\",\"CFBun... {7005EC3E8F}> :CODE 127 :PROCESS NIL)
7: (UIOP/RUN-PROGRAM::%CHECK-RESULT 127 :COMMAND #<(SIMPLE-ARRAY CHARACTER (819)) (/usr/bin/plutil -convert json -- orig && /usr/bin/plutil -convert json -- bare-wrapper && jq --argjson keys "[\"CFBundleDevelopmentRegion\",\"CFBundleDocumentTypes\",\"CFBundleGetInfoString\",\"CFBun... {7005EC3E8F}> :PROCESS NIL :IGNORE-ERROR-STATUS NIL)
8: (UIOP/RUN-PROGRAM::%USE-SYSTEM #<(SIMPLE-ARRAY CHARACTER (819)) (/usr/bin/plutil -convert json -- orig && /usr/bin/plutil -convert json -- bare-wrapper && jq --argjson keys "[\"CFBundleDevelopmentRegion\",\"CFBundleDocumentTypes\",\"CFBundleGetInfoString\",\"CFBun... {7005EC3E8F}> :IGNORE-ERROR-STATUS NIL :HOST NIL :OUTPUT T :ERROR-OUTPUT T :SHOW NIL)
9: ((FLET SH::RUN-IT :IN SH:RUN/NIL))
10: ((FLET "F1" :IN COPY-PATHS))
11: (COPY-PATHS "/nix/store/n1rs0c0xm3mfyn234ylq93lyhl3wn28f-terminal-notifier-2.0.0/Applications/terminal-notifier.app//Contents/Info.plist" "/Applications/Nix Trampolines/terminal-notifier.app/Contents/Info.plist" #<unused argument>)
12: (SYNC-TRAMPOLINES "/Applications/Nix Apps" "/Applications/Nix Trampolines")
13: (SB-INT:SIMPLE-EVAL-IN-LEXENV (MAIN) #<NULL-LEXENV>)
14: (SB-EXT:EVAL-TLF (MAIN) 38 NIL)
15: ((LABELS SB-FASL::EVAL-FORM :IN SB-INT:LOAD-AS-SOURCE) (MAIN) 38)
16: ((LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) (MAIN) :CURRENT-INDEX 38)
17: (SB-C::%DO-FORMS-FROM-INFO #<FUNCTION (LAMBDA (SB-KERNEL:FORM &KEY :CURRENT-INDEX &ALLOW-OTHER-KEYS) :IN SB-INT:LOAD-AS-SOURCE) {70088303EB}> #<SB-C::SOURCE-INFO {7008830423}> SB-C::INPUT-ERROR-IN-LOAD)
18: (SB-INT:LOAD-AS-SOURCE #<SB-SYS:FD-STREAM for "file /nix/store/mr1kr6995i4r38gc1m52q5789bqqyxnc-mac-app-util/bin/.mac-app-util-wrapped" {7008830003}> :VERBOSE NIL :PRINT NIL :CONTEXT "loading")
19: ((LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) #<SB-SYS:FD-STREAM for "file /nix/store/mr1kr6995i4r38gc1m52q5789bqqyxnc-mac-app-util/bin/.mac-app-util-wrapped" {7008830003}> NIL)
20: (SB-FASL::CALL-WITH-LOAD-BINDINGS #<FUNCTION (LABELS SB-FASL::LOAD-STREAM-1 :IN LOAD) {104FD09EB}> #<SB-SYS:FD-STREAM for "file /nix/store/mr1kr6995i4r38gc1m52q5789bqqyxnc-mac-app-util/bin/.mac-app-util-wrapped" {7008830003}> NIL #<SB-SYS:FD-STREAM for "file /nix/store/mr1kr6995i4r38gc1m52q5789bqqyxnc-mac-app-util/bin/.mac-app-util-wrapped" {7008830003}>)
21: (LOAD #<SB-SYS:FD-STREAM for "file /nix/store/mr1kr6995i4r38gc1m52q5789bqqyxnc-mac-app-util/bin/.mac-app-util-wrapped" {7008830003}> :VERBOSE NIL :PRINT NIL :IF-DOES-NOT-EXIST :ERROR :EXTERNAL-FORMAT :DEFAULT)
22: ((FLET SB-IMPL::LOAD-SCRIPT :IN SB-IMPL::PROCESS-SCRIPT) #<SB-SYS:FD-STREAM for "file /nix/store/mr1kr6995i4r38gc1m52q5789bqqyxnc-mac-app-util/bin/.mac-app-util-wrapped" {7008830003}>)
23: ((FLET SB-UNIX::BODY :IN SB-IMPL::PROCESS-SCRIPT))
24: ((FLET "WITHOUT-INTERRUPTS-BODY-11" :IN SB-IMPL::PROCESS-SCRIPT))
25: (SB-IMPL::PROCESS-SCRIPT "/nix/store/mr1kr6995i4r38gc1m52q5789bqqyxnc-mac-app-util/bin/.mac-app-util-wrapped")
26: (SB-IMPL::TOPLEVEL-INIT)
27: ((FLET SB-UNIX::BODY :IN SB-IMPL::START-LISP))
28: ((FLET "WITHOUT-INTERRUPTS-BODY-3" :IN SB-IMPL::START-LISP))
29: (SB-IMPL::%START-LISP)

unhandled condition in --disable-debugger mode, quitting

Seems like the CL program expects jq to be installed as a system dependency, but I think it would be more correct to make it available by way of the nix environment, rather than expecting it to be ambiently available.

Curiously, I do have jq in my user PATH, but it's installed by nix, so I wonder if that makes it hard for the CL program to find.

[sloane-shark]

side note: I think the tool may also be leaving some files around in my cwd when crashing, but I imagine that may be because of the crash

[sloane-shark]

I tried adding jq to the build inputs of the CL package derivation using overrideAttrs, e.g.

mac-app-util = inputs.mac-app-util.packages.${system}.default.overrideAttrs (old: {
# tried with both of these
    buildInputs = old.buildInputs ++ [ jq ];
    propagatedBuildInputs = [ jq ];
});

but this doesn't seem to work?

edit: even installing jq via homebrew doesn't seem to fix this for me :/

[hraban]

I tried adding jq to the build inputs of the CL package derivation using overrideAttrs, e.g.

mac-app-util = inputs.mac-app-util.packages.${system}.default.overrideAttrs (old: {
# tried with both of these
    buildInputs = old.buildInputs ++ [ jq ];
    propagatedBuildInputs = [ jq ];
});

but this doesn't seem to work?

That's a great attempt, but it doesn't work because in the end buildInputs is just a way for the nixpkgs stdenv mkderivation builder to make certain packages available to the build environment, and thereby to setuphooks. It doesn't magically bake the PATH, or do anything, really--the magic happens generally in other setuphooks. E.g.: adding a package to your buildInputs makes it available to the C compiler.. how? Because the C compiler has a setup hook which reads that buildInputs and adds it to an environment variable that the C compiler later reads.

However, none of this persists to the final binary or library, unless you tell it to. You have to explicitly bake paths into binaries, if you want them available. For libraries this is done using patchelf, but for hooking stuff into the PATH, you have to create a wrapper. See 19b3e7a for the solution.

(For completeness' sake: if you wanted to do this without my involvement (say you just want to patch some obstinate upstream project which won't change), you could create a derivation which is just a wrapper of my mac-app-util, and which adds jq to the path.)

edit: even installing jq via homebrew doesn't seem to fix this for me :/

Yeah this makes sense: brew stuff gets installed in /usr/local/bin or /opt/homebrew/bin, neither of which is in the PATH during nix builds.

side note: I think the tool may also be leaving some files around in my cwd when crashing, but I imagine that may be because of the crash

I'd love to hear more about this 👂 👀

[hraban]

In case you're curious, here's an example of a clearer PR which solves this issue from scratch: seppeljordan/nix-prefetch-github#45

This also suggests another way to solve it:

              mac-app-util = inputs.mac-app-util.packages.${pkgs.stdenv.system}.default.overrideAttrs (old: {
                postInstall = old.postInstall + ''
                  wrapProgram "$out/bin/mac-app-util" --suffix PATH : "${pkgs.jq}/bin"
                '';
              });

I used to complain about git having poor UI... 🤷‍♀️

[hraban]

side note: I think the tool may also be leaving some files around in my cwd when crashing, but I imagine that may be because of the crash

oops you're right I found it. fixed, thanks.