Skip to content

Conversation

savil
Copy link
Collaborator

@savil savil commented Jun 13, 2023

Summary

Motivation
In elixir projects, iex -S mix <script> has the iex interpreter invoking the
mix build tool. However, the mix build tool has been wrapped by devbox in a bash script.
And this leads to a failure as iex is unable to interpret a bash script.

Fix
Binaries should be able to directly invoke other installed binaries.

We implemented the binary-wrappers so that the shell-environment is correctly
configured prior to the installed binary being invoked. In this case, the iex
binary wrapper will run thereby correctly configuring the environment. Any subsequent
binary invoked in the same sub-shell will inherit this up-to-date environment. So,
the other binaries can be directly invoked (instead of the binary-wrapper scripts).

Implementation
In this PR, we change the PATH that is set within a binary-wrapper.

Specifically, the PATH has the form:

<project-dir>/.devbox/virtenv/.wrappers/bin : <project-dir>/.devbox/nix/profile/default/bin

Within the binary-wrapper, we now omit the .wrappers/bin directory from PATH
by using a new hidden flag --omit-wrappers-from-path on devbox shellenv.

perf analysis

I expect the perf impact to be minimal.

  1. Speed up: Binaries invoking other binaries will be a bit faster since they no longer first execute the binary-wrapper.

  2. Slowdown: we remove one layer of caching. Previously, we'd check the shellenv-hash in the binary wrapper, and skip invoking devbox shellenv if the hash was up-to-date. Now, on every invocation of a binary-wrapper we'll call devbox shellenv. This should still be fast due to the internal caching we do within devbox shellenv.

Fixes #1142

How was it tested?

BEFORE: doing devbox run -- iex -S mix would result in the error in the linked issue.

AFTER:

> devbox run -- iex -S mix
* creating .nix-mix/archives/hex-2.0.6
* creating .nix-mix/elixir/1-14/rebar3
Resolving Hex dependencies...
Resolution completed in 0.015s
Unchanged:
  cowboy 2.9.0
  cowlib 2.11.0
  ranch 1.8.0
All dependencies are up to date
Erlang/OTP 25 [erts-13.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Hello World!
Interactive Elixir (1.14.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Goodbye World

15:14:23.204 [notice] Application elixir_hello exited: shutdown

BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution

Also - did test plan of #1093 to verify devbox update continues working. A sanity check since we dropped ShellEnvHash from the wrappers and I wanted to ensure this works as expected.

Copy link
Collaborator Author

savil commented Jun 13, 2023

Current dependencies on/for this PR:

This comment was auto-generated by Graphite.

@savil
Copy link
Collaborator Author

savil commented Jun 13, 2023

gah, I have a lot of local changes that didn't get added to the commit here. Updating!

@savil savil force-pushed the savil/wrapper-path-fix-v1 branch from 96d423b to 70589b0 Compare June 13, 2023 23:32
@savil savil force-pushed the savil/wrapper-path-fix-v1 branch from 70589b0 to b3b8636 Compare June 13, 2023 23:33
Copy link
Contributor

@mikeland73 mikeland73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no blocking comments

*/ -}}

if [[ "${{ .ShellEnvHashKey }}" != "{{ .ShellEnvHash }}" ]] && [[ -z "${{ .ShellEnvHashKey }}_GUARD" ]]; then
if [[ -z "${{ .ShellEnvHashKey }}_GUARD" ]]; then
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the guard can be removed (the guard is in place to avoid infinite wrapper loop but since we're removing it from path it can't happen.

This is hard to test so we may be better off leaving as is for now. Maybe add a comment

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I'll leave a comment. Would rather be super conservative with these changes.

}

envStr, err := box.PrintEnv(cmd.Context(), flags.runInitHook)
envStr, err := box.PrintEnv(cmd.Context(), flags.runInitHook, flags.omitWrappersFromPath)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When adding a second boolean is a good time to switch to a struct :P

Copy link
Collaborator Author

@savil savil Jun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I knew you'd say this :p

My personal threshold was three args, but sure 👍🏾

@savil savil merged commit e1d264d into main Jun 14, 2023
@savil savil deleted the savil/wrapper-path-fix-v1 branch June 14, 2023 11:48
savil added a commit that referenced this pull request Jun 14, 2023
…revent a binary from invoking a wrapped-binary. (#1151)"

This reverts commit e1d264d.
savil added a commit that referenced this pull request Jun 15, 2023
…revent a binary from invoking a wrapped-binary. (#1151)"

This reverts commit e1d264d.
savil added a commit that referenced this pull request Jun 15, 2023
…revent a binary from invoking a wrapped-binary. (#1151)" (#1159)

## Summary

Reverting due to regression:
Within a devbox script if any environment variable has been exported,
and
if a devbox-plugin also sets that env-var, then the devbox-plugin
env-var
will be set for the wrapped-binary.

Specifically, in the `lapp-stack/devbox.json`, we have:
```
      "run_test": [
        "mkdir -p /tmp/devbox/lapp",
        "export PGHOST=/tmp/devbox/lapp",
        "initdb",
```

but the PGHOST that `initdb` sees is a different value i.e. the one set
by the devbox plugin.

In the code being reverted, the wrapped-bin's bash script would always
invoke
`eval $(devbox shellenv --omit-wrappers-from-path=true)` in order to
omit wrappers
from PATH. A side-effect is that the PGHOST would also get overridden.


## How was it tested?

no testing done.
straight revert, tests should pass.
savil added a commit that referenced this pull request Jun 15, 2023
…ers (#1160)

## Summary

This is an alternate to #1151 (see explanation in #1159)

In this approach, the wrapped-binary always calls:
`eval $(devbox shellenv only-path-without-wrappers)` 
to ensure that the PATH is set without the `wrappers/bin` directory.

In contrast to #1151, we do not always set any other environment
variables

Fixes #1142

## How was it tested?

issue from #1142 is fixed:
```
> devbox run -- iex -S mix
* creating .nix-mix/archives/hex-2.0.6
* creating .nix-mix/elixir/1-14/rebar3
Resolving Hex dependencies...
Resolution completed in 0.016s
Unchanged:
  cowboy 2.9.0
  cowlib 2.11.0
  ranch 1.8.0
All dependencies are up to date
Erlang/OTP 25 [erts-13.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Hello World!
Interactive Elixir (1.14.4) - press Ctrl+C to exit (type h() ENTER for help)
iex(1)> Goodbye World

17:18:09.349 [notice] Application elixir_hello exited: shutdown

BREAK: (a)bort (A)bort with dump (c)ontinue (p)roc info (i)nfo
       (l)oaded (v)ersion (k)ill (D)b-tables (d)istribution
```

- [ ] testscripts should pass
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Wrappers interfere with iex -s mix
2 participants