Ever find that a command works in your shell, but not in Emacs?
– @purcell
There are two known billion-dollar problems in software engineering: null pointer exceptions and caching. Over time of maintaining various projects I realised that there is a third billion-dollar problem - PATH
value. Whenever something doesn’t build, whenever something doesn’t work as expected, the blood trail often leads to… PATH
value. Sometimes it’s some other environment variable, but in most of the cases, it’s just PATH
.
Time to time I help Emacs+ users with PATH
related issues, so I figured that Emacs+ should provide some solution out of box. 10 messages from users in an esoteric project is like Maidan in Kyiv (yup, now I am bringing politics in here). So there is a huge problem needs solving. From thought to solution the path is quick.
TL;DR, your PATH
value is now being injected into Emacs.app
during build, so it is picked up by Emacs whenever you run it from Finder, Docker, Spotlight or launchd
system. Works for emacs-plus@28
and emacs-plus@29
.
Even though I talk about problems related to PATH
value, it’s value is actually correct, just not something we expect when it comes to using external binaries from within Emacs.
Disclaimer. In the next few paragraphs I am explaining how macOS runs applications, but it might be not fully accurate as I am not an expert in these low level details. At least, speculation stands, so bear with me. And if you know better then me, please do tell me, so we can improve this information.
User applications in macOS are running in the login shell environment. From PATH
perspective, it means that it’s value is inherited from login shell. So what’s the value? If we take a look at /etc/profile
and /etc/zshenv
we see that they all run path_helper
, which already appeared on my blog in Fixing PATH in fish with nix-darwin post. In short, it’s a helper for constructing PATH
environment variable by taking entries from /etc/paths
file and all files under /etc/paths.d/
. It actually also constructs MANPATH
entries, but it’s not important right now.
So by default, applications have PATH
value defined by result of path_helper
invocation.
$ cat /etc/paths
/usr/local/bin
/usr/bin
/bin
/usr/sbin
/sbin
$ ls -l /etc/paths.d
-rwxr-xr-x 1 root wheel 23 Apr 22 20:47 100-rvictl*
$ cat /etc/paths.d/100-rvictl
/Library/Apple/usr/bin
$ /usr/libexec/path_helper -s
PATH="/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/Apple/usr/bin"; export PATH;
And this is pretty much the same PATH
most users see unless they tweak things. As it turns out there are several ways to do it. I am not going to dig into each of them, but roughly, you have the following options:
- Add a file to
/etc/paths.d/
with values you want to append toPATH
by default. This method affects everything, so use cautiously. - Create a
my.startup.plist
in~/Library/LaunchAgents
that callslaunchctl setenv
to modifyPATH
. This method affects all applications run by user. - Set LSEnvironment property in applications
Info.plist
file. This method affects only specific application. - purcell/exec-path-from-shell, which is a great alternative of previous method, but for Emacs specifically. The biggest added value is that it works dynamically and is based on your current shell settings.
At this point it might become obvious, which method Emacs+ takes to solve PATH
problem for its users. No, it’s not the last option, as it’s up to user to install external packages and setup them. Less intrusive, but more error-prone solution is to modify Emacs.apo/Info.plist
file to include the value of the PATH
during installation.
Turns out, brew
uses a sanitized environment, which they call superenv
. If you think about it, such environment makes a lot of sense as you want to avoid any environment dependent stuff to mess up installation process. But that also means that Emacs+ formula can’t access user PATH
value.
The good part is that it’s possible to switch to standard environment in formula definition:
class EmacsPlusAt29 < EmacsBase # which is actually just Formula
# ...
env :std
# ...
end
Now it’s possible to access PATH
value and inject it to the Info.plist
.
prefix.install "nextstep/Emacs.app"
# inject PATH to Info.plist
path = ENV['PATH'].split("#{HOMEBREW_PREFIX}/Library/Homebrew/shims/shared:").last
system "/usr/libexec/PlistBuddy -c 'Add :LSEnvironment dict' '#{prefix}/Emacs.app/Contents/Info.plist'"
system "/usr/libexec/PlistBuddy -c 'Add :LSEnvironment:PATH string' '#{prefix}/Emacs.app/Contents/Info.plist'"
system "/usr/libexec/PlistBuddy -c 'Set :LSEnvironment:PATH #{path}' '#{prefix}/Emacs.app/Contents/Info.plist'"
system "/usr/libexec/PlistBuddy -c 'Print :LSEnvironment' '#{prefix}/Emacs.app/Contents/Info.plist'"
system "touch '#{prefix}/Emacs.app'"
Notice that I remove all entries from PATH
that come before (inclusive) #{HOMEBREW_PREFIX}/Library/Homebrew/shims/shared
. This is because brew
still appends some stuff to the PATH
during build process and we definitely don’t want that stuff to be part of PATH
, right?
That’s basically it. With these small changes you can enjoy expected PATH
value without the need to use purcell/exec-path-from-shell. Though this package should be used if you want to get other stuff from your interactive shell.
While it all might sounds like an effort not worth result, this change actually provides two extra points.
I could at last remove flags manipulations related to native-comp
feature:
if build.with? "native-comp"
gcc_ver = Formula["gcc"].any_installed_version
gcc_ver_major = gcc_ver.major
gcc_lib="#{HOMEBREW_PREFIX}/lib/gcc/#{gcc_ver_major}"
ENV.append "CFLAGS", "-I#{Formula["gcc"].include}"
ENV.append "CFLAGS", "-I#{Formula["libgccjit"].include}"
ENV.append "CFLAGS", "-I#{Formula["gmp"].include}"
ENV.append "CFLAGS", "-I#{Formula["libjpeg"].include}"
ENV.append "LDFLAGS", "-L#{gcc_lib}"
ENV.append "LDFLAGS", "-I#{Formula["gcc"].include}"
ENV.append "LDFLAGS", "-I#{Formula["libgccjit"].include}"
ENV.append "LDFLAGS", "-I#{Formula["gmp"].include}"
ENV.append "LDFLAGS", "-I#{Formula["libjpeg"].include}"
end
It’s not that bad to have them, but since Emacs own configuration.ac
supports brew
during libgccjit
check, I’d rather let Emacs developers do the work that they know how to do (in contrast with my doings).
Another perk is also related to native-comp
feature, but now it affects users in a more direct fashion. Native compilation normally starts before any custom user code in init.el
and people often run into problems related to environment troubles.
So all that is nice. Hopefully I will not need to revert this injection. Because at this point of time, injection happens in emacs-plus@28
and emacs-plus@29
. But most importantly, there is a blog post about PATH
injection. So business here is serious, you can’t simply step back.
Safe travels folks! And use Emacs responsibly.