(GH-2922) Attempt to restore Ruby environment on local #2956

lucywyman · 2021-07-27T13:47:37Z

When running over the local transport with bundled-ruby set to
false, Bolt will attempt to restore the original Ruby environment
variables, specifically:

GEM_PATH
GEM_HOME
RUBYLIB
RUBYLIB_PREFIX
RUBYOPT
RUBYPATH
RUBYSHELL

The Bolt binary wrapper explicitly unsets these variables to provide a
clean environment for the Bolt process to run in, which affects running
Bolt from a system package, and Bundler can also modify these
environment variables, saving their original values to
BUNDLER_ORIG_<env_var>. Changes to the Ruby environment that Bolt runs
in are incidentally picked up by Ruby tasks and plans running over the
local transport, which opens a new subprocess to execute. The binary
wrapper now stores those environment variables in BOLT_ORIG_<env_var>
environment variables, and when executing with bundled-ruby explicitly set
to false (not unconfigured), Bolt will attempt to restore the
environment variables first from BOLT_ORIG_<env_var>, then from
BUNDLER_ORIG_<env_var>.

!bug

Attempt to restore Ruby environment on local with bundled-ruby false (#2922)
When executing, Bolt will unset several Ruby environment variables in
it's own environment to avoid unexpected behavior. This can interfere
with running Ruby on the local transport with Bolt though. Bolt will now
attempt to restore Ruby environment variables like GEM_HOME and
GEM_PATH when running over the local transport with bundled-ruby set
to false.

lib/bolt/transport/local/connection.rb

lucywyman · 2021-07-27T17:05:15Z

Relies on puppetlabs/bolt-vanagon#170

lucywyman · 2021-07-28T17:53:10Z

Phew, got a successful acceptance run (with an actual value for GEM_PATH):

  * execute `bolt task run test::ruby_env` on localhost via local transport
    
    numb-separation.delivery.puppetlabs.net (numb-separation.delivery.puppetlabs.net) 10:50:08$ GEM_PATH=$(gem env GEM_PATHS) bolt task run test::ruby_env --targets localhost --modulepath /tmp/local_task.5LiGPf/modules
      Started on localhost...
      Finished on localhost:
        /root/.gem/ruby/2.5.0:/var/lib/gems/2.5.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.5.0:/usr/share/rubygems-integration/2.5.0:/usr/share/rubygems-integration/all
      Successful on 1 target: localhost
      Ran on 1 target in 0.08 sec
    
    numb-separation.delivery.puppetlabs.net (numb-separation.delivery.puppetlabs.net) executed in 1.77 seconds
    
    numb-separation.delivery.puppetlabs.net (numb-separation.delivery.puppetlabs.net) 10:50:10$ gem env GEM_PATHS
      /root/.gem/ruby/2.5.0:/var/lib/gems/2.5.0:/usr/lib/x86_64-linux-gnu/rubygems-integration/2.5.0:/usr/share/rubygems-integration/2.5.0:/usr/share/rubygems-integration/all
    
    numb-separation.delivery.puppetlabs.net (numb-separation.delivery.puppetlabs.net) executed in 0.21 seconds

lib/bolt/shell/powershell.rb

lib/bolt/shell/bash.rb

When running over the `local` transport with `bundled-ruby` set to false, Bolt will attempt to restore the original Ruby environment variables, specifically: * GEM_PATH * GEM_HOME * RUBYLIB * RUBYLIB_PREFIX * RUBYOPT * RUBYPATH * RUBYSHELL The Bolt binary wrapper explicitly unsets these variables to provide a clean environment for the Bolt process to run in, which affects running Bolt from a system package, and Bundler can also modify these environment variables, saving their original values to `BUNDLER_ORIG_<env_var>`. Changes to the Ruby environment that Bolt runs in are incidentally picked up by Ruby tasks and plans running over the local transport, which opens a new subprocess to execute. The binary wrapper now stores those environment variables in `BOLT_ORIG_<env_var>` environment variables, and when executing with `bundled-ruby` explicitly set to false (not unconfigured), Bolt will attempt to restore the environment variables first from `BOLT_ORIG_<env_var>`, then from `BUNDLER_ORIG_<env_var>`. !bug * **Attempt to restore Ruby environment on local with bundled-ruby false** ([puppetlabs#2922](puppetlabs#2922)) When executing, Bolt will unset several Ruby environment variables in it's own environment to avoid unexpected behavior. This can interfere with running Ruby on the local transport with Bolt though. Bolt will now attempt to restore Ruby environment variables like `GEM_HOME` and `GEM_PATH` when running over the local transport with `bundled-ruby` set to false.

beechtom

LGTM. 👍

lib/bolt/transport/local/connection.rb

The environment variable for setting the beaker hosts file was changed from `BEAKER_HOSTS` to `BEAKER_HOSTS_FILE` in puppetlabs#2956, which caused Bolt's acceptance tests to begin failing. Tests began failing because the vanagon integration job in Jenkins generates the hosts file and sets the path to the file in `BEAKER_HOSTS`. This results in Bolt's acceptance tests generating a new hosts file, since `BEAKER_HOSTS_FILE` is unset. Generating the hosts file fails since the rake task runs `beaker-hostgenerator` instead of the correct `bundle exec beaker-hostgenerator`. This resets the environment variable to `BEAKER_HOSTS` as expected by the Jenkins job and beaker itself, and updates the shell command executed by the acceptance test rake task. !no-release-note

lucywyman force-pushed the GH-2922 branch 3 times, most recently from c266311 to dbc6aab Compare July 27, 2021 16:17

donoghuc reviewed Jul 27, 2021

View reviewed changes

lib/bolt/transport/local/connection.rb Outdated Show resolved Hide resolved

lucywyman force-pushed the GH-2922 branch 2 times, most recently from 5bfad64 to 9d41fbd Compare July 27, 2021 17:04

lucywyman force-pushed the GH-2922 branch 5 times, most recently from 9487f48 to d06d70c Compare July 27, 2021 22:34

lucywyman marked this pull request as ready for review July 27, 2021 23:02

lucywyman requested a review from a team as a code owner July 27, 2021 23:02

lucywyman force-pushed the GH-2922 branch 3 times, most recently from 1098cd4 to a60f501 Compare July 28, 2021 15:53

lucywyman linked an issue Jul 28, 2021 that may be closed by this pull request

Provide a way to not leak bolt runner ruby environment to tasks run over local transport #2922

Closed

lucywyman force-pushed the GH-2922 branch from a60f501 to aa51192 Compare July 28, 2021 17:53

beechtom reviewed Jul 28, 2021

View reviewed changes

lib/bolt/shell/powershell.rb Outdated Show resolved Hide resolved

lucywyman force-pushed the GH-2922 branch from aa51192 to d3f809d Compare July 29, 2021 19:53

donoghuc reviewed Jul 29, 2021

View reviewed changes

lib/bolt/shell/bash.rb Outdated Show resolved Hide resolved

lucywyman force-pushed the GH-2922 branch from d3f809d to 8c4fd5a Compare July 30, 2021 20:39

lucywyman force-pushed the GH-2922 branch from 8c4fd5a to a5a9cb9 Compare August 2, 2021 16:34

beechtom approved these changes Aug 2, 2021

View reviewed changes

lib/bolt/transport/local/connection.rb Show resolved Hide resolved

beechtom merged commit 2551ee1 into puppetlabs:main Aug 2, 2021

beechtom mentioned this pull request Aug 3, 2021

(maint) Fix broken acceptance tests #2968

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(GH-2922) Attempt to restore Ruby environment on local #2956

(GH-2922) Attempt to restore Ruby environment on local #2956

lucywyman commented Jul 27, 2021

lucywyman commented Jul 27, 2021

lucywyman commented Jul 28, 2021

beechtom left a comment

(GH-2922) Attempt to restore Ruby environment on local #2956

(GH-2922) Attempt to restore Ruby environment on local #2956

Conversation

lucywyman commented Jul 27, 2021

lucywyman commented Jul 27, 2021

lucywyman commented Jul 28, 2021

beechtom left a comment

Choose a reason for hiding this comment