Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Building an omnibus package writes to the install destination #840

Closed
drbrain opened this issue May 17, 2018 · 3 comments
Closed

Building an omnibus package writes to the install destination #840

drbrain opened this issue May 17, 2018 · 3 comments

Comments

@drbrain
Copy link

drbrain commented May 17, 2018

Description

When I'm building the next version of an omnibus-packaged application omnibus obliterates the existing files I installed using a previous package. If the build fails I may have a corrupt or non-working application.

If I'm building for the first time omnibus expects to be able to write to the installed locations. If this directory is only root-writable the omnibus build will fail with Errno::EACCESS as it tries to delete the installed files.

Additionally, a macOS .pkg requires root permissions to install the package, which omnibus should not need to build that package.

Instead omnibus should build the installer in a directory completely under its ownership.

Omnibus Version

5.6.12

Platform Version

macOS 10.13.4:
Darwin asx.local 17.5.0 Darwin Kernel Version 17.5.0: Fri Apr 13 19:32:32 PDT 2018; root:xnu-4570.51.2~1/RELEASE_X86_64 x86_64

Ruby 2.5.0:
ruby 2.5.0p0 (2017-12-25 revision 61468) [x86_64-darwin17]

Replication Case

  1. git clone git@github.com:chef/chef-dk.git
  2. cd chef-dk/omnibus
  3. bundle install --binstubs
  4. bundle exec omnibus build chefdk

It doesn't matter which omnibus application I'm trying to build, omnibus always tries to write to the install path.

Build Output

$ bundle exec omnibus build chefdk
                    [CLI] I | 2018-05-16T17:44:41-07:00 | Using config from 'omnibus.rb'
[2018-05-16T17:44:42-07:00] INFO: The plugin path /etc/chef/ohai/plugins does not exist. Skipping...
 [Software: delivery-cli] W | 2018-05-16T17:44:42-07:00 | Version master for software delivery-cli was not parseable. Comparison methods such as #satisfies? will not be available for this version.
 [Software: delivery-cli] W | 2018-05-16T17:44:42-07:00 | Version master for software delivery-cli was not parseable. Comparison methods such as #satisfies? will not be available for this version.
 [Software: config_guess] W | 2018-05-16T17:44:42-07:00 | Version master for software config_guess was not parseable. Comparison methods such as #satisfies? will not be available for this version.
      [Software: chef-dk] W | 2018-05-16T17:44:42-07:00 | Version local_source for software chef-dk was not parseable. Comparison methods such as #satisfies? will not be available for this version.
      [Software: chef-dk] W | 2018-05-16T17:44:42-07:00 | Version local_source for software chef-dk was not parseable. Comparison methods such as #satisfies? will not be available for this version.
Building chefdk 3.0.35+20180517004441...
 [Software: config_guess] I | 2018-05-16T17:44:42-07:00 | Resolving manifest entry for config_guess
 [Software: config_guess] W | 2018-05-16T17:44:42-07:00 | Version master for software config_guess was not parseable. Comparison methods such as #satisfies? will not be available for this version.
 [Software: config_guess] W | 2018-05-16T17:44:42-07:00 | Version master for software config_guess was not parseable. Comparison methods such as #satisfies? will not be available for this version.
  [Software: preparation] I | 2018-05-16T17:44:42-07:00 | Resolving manifest entry for preparation
[NullFetcher: preparation] I | 2018-05-16T17:44:42-07:00 | Fetching `preparation' (nothing to fetch)
#<Thread:0x00007fcd9af02ca0@/usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:57 run> terminated with exception (report_on_exception is true):
Traceback (most recent call last):
	17: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:61:in `block (2 levels) in initialize'
	16: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:61:in `catch'
	15: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:62:in `block (3 levels) in initialize'
	14: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:62:in `loop'
	13: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:64:in `block (4 levels) in initialize'
	12: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/project.rb:1066:in `block (3 levels) in download'
	11: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/software.rb:897:in `fetch'
	10: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetchers/null_fetcher.rb:53:in `fetch'
	 9: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetcher.rb:178:in `create_required_directories'
	 8: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetcher.rb:178:in `each'
	 7: from /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetcher.rb:179:in `block in create_required_directories'
	 6: from /usr/local/lib/ruby/2.5.0/fileutils.rb:193:in `mkdir_p'
	 5: from /usr/local/lib/ruby/2.5.0/fileutils.rb:193:in `each'
	 4: from /usr/local/lib/ruby/2.5.0/fileutils.rb:208:in `block in mkdir_p'
	 3: from /usr/local/lib/ruby/2.5.0/fileutils.rb:208:in `reverse_each'
	 2: from /usr/local/lib/ruby/2.5.0/fileutils.rb:210:in `block (2 levels) in mkdir_p'
	 1: from /usr/local/lib/ruby/2.5.0/fileutils.rb:232:in `fu_mkdir'
/usr/local/lib/ruby/2.5.0/fileutils.rb:232:in `mkdir': Permission denied @ dir_s_mkdir - /var/cache (Errno::EACCES)
bundler: failed to load command: omnibus (/usr/local/lib/ruby/gems/2.5.0/bin/omnibus)
Errno::EACCES: Permission denied @ dir_s_mkdir - /var/cache
  /usr/local/lib/ruby/2.5.0/fileutils.rb:232:in `mkdir'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:232:in `fu_mkdir'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:210:in `block (2 levels) in mkdir_p'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:208:in `reverse_each'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:208:in `block in mkdir_p'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:193:in `each'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:193:in `mkdir_p'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetcher.rb:179:in `block in create_required_directories'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetcher.rb:178:in `each'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetcher.rb:178:in `create_required_directories'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/fetchers/null_fetcher.rb:53:in `fetch'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/software.rb:897:in `fetch'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/project.rb:1066:in `block (3 levels) in download'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:64:in `block (4 levels) in initialize'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:62:in `loop'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:62:in `block (3 levels) in initialize'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:61:in `catch'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/thread_pool.rb:61:in `block (2 levels) in initialize'
@scotthain
Copy link
Contributor

Omnibus defaults to using /var/cache/omnibus as the root directory, and expects read/write/execute privileges. If this doesn't work for your setup you can override (with varying levels of granularity) the build, cache, package, and other directories using the configuration options found here: https://www.rubydoc.info/github/chef/omnibus/Omnibus/Config

@drbrain
Copy link
Author

drbrain commented May 17, 2018

The documentation of omnibus requiring write access to /var/cache/omnibus is not easily discoverable from the README. There's a small clue in the config section. Clicking the link to the Config documentation shows me a list of methods I can tweak but no real description of what they mean or how they interrelate. I don't know what is meant by "The “base” directory where Omnibus will store it's data." This should be part of the Get Started section.

But now that I know this I've made the directory writable by my user and run again, this time with a generated project.

  1. $ omnibus new MY_PROJECT_NAME
  2. cd omnibus-MY_PROJECT_NAME/
  3. bundle install --binstubs
  4. (create config/software/preparation.rb and config/software/version-manifest.rb that are missing, why?)
  5. bin/omnibus build MY_PROJECT_NAME

I expect this to not need root permissions and to not write to my existing application, but it does:

$ bin/omnibus build MY_PROJECT_NAME
                    [CLI] I | 2018-05-17T10:43:56-07:00 | Using config from 'omnibus.rb'
[2018-05-17T10:43:56-07:00] INFO: The plugin path /etc/chef/ohai/plugins does not exist. Skipping...
           [BuildVersion] W | 2018-05-17T10:43:56-07:00 | Could not extract version information from 'git describe'! Setting version to 0.0.0.
Building MY_PROJECT_NAME 0.0.0+20180517174356...
  [Software: preparation] I | 2018-05-17T10:43:56-07:00 | Resolving manifest entry for preparation
[NullFetcher: preparation] I | 2018-05-17T10:43:56-07:00 | Fetching `preparation' (nothing to fetch)
[Software: version-manifest] I | 2018-05-17T10:43:56-07:00 | Resolving manifest entry for version-manifest
[NullFetcher: version-manifest] I | 2018-05-17T10:43:56-07:00 | Fetching `version-manifest' (nothing to fetch)
Traceback (most recent call last):
	17: from bin/omnibus:29:in `<main>'
	16: from bin/omnibus:29:in `load'
	15: from /usr/local/lib/ruby/gems/2.5.0/gems/omnibus-5.6.8/bin/omnibus:16:in `<top (required)>'
	14: from /usr/local/lib/ruby/gems/2.5.0/gems/omnibus-5.6.8/lib/omnibus/cli.rb:42:in `execute!'
	13: from /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/base.rb:466:in `start'
	12: from /usr/local/lib/ruby/gems/2.5.0/gems/omnibus-5.6.8/lib/omnibus/cli/base.rb:33:in `dispatch'
	11: from /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor.rb:387:in `dispatch'
	10: from /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/invocation.rb:126:in `invoke_command'
	 9: from /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.20.0/lib/thor/command.rb:27:in `run'
	 8: from /usr/local/lib/ruby/gems/2.5.0/gems/omnibus-5.6.8/lib/omnibus/cli.rb:89:in `build'
	 7: from /usr/local/lib/ruby/gems/2.5.0/gems/omnibus-5.6.8/lib/omnibus/project.rb:1073:in `build'
	 6: from /usr/local/lib/ruby/2.5.0/fileutils.rb:193:in `mkdir_p'
	 5: from /usr/local/lib/ruby/2.5.0/fileutils.rb:193:in `each'
	 4: from /usr/local/lib/ruby/2.5.0/fileutils.rb:208:in `block in mkdir_p'
	 3: from /usr/local/lib/ruby/2.5.0/fileutils.rb:208:in `reverse_each'
	 2: from /usr/local/lib/ruby/2.5.0/fileutils.rb:210:in `block (2 levels) in mkdir_p'
	 1: from /usr/local/lib/ruby/2.5.0/fileutils.rb:232:in `fu_mkdir'
/usr/local/lib/ruby/2.5.0/fileutils.rb:232:in `mkdir': Permission denied @ dir_s_mkdir - /opt/MY_PROJECT_NAME (Errno::EACCES)

Similarly for chef-dk from above:

$ bundle exec omnibus build chefdk
[…]
   [PathFetcher: chef-dk] I | 2018-05-17T10:46:45-07:00 | Digesting /Users/erichodel/tmp/chef-dk with sha256
bundler: failed to load command: omnibus (/usr/local/lib/ruby/gems/2.5.0/bin/omnibus)
Errno::EACCES: Permission denied @ apply2files - /opt/chefdk/LICENSES/.gitkeep
  /usr/local/lib/ruby/2.5.0/fileutils.rb:1054:in `utime'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:1054:in `block in touch'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:1051:in `each'
  /usr/local/lib/ruby/2.5.0/fileutils.rb:1051:in `touch'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/licensing.rb:119:in `prepare'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/licensing.rb:59:in `block in create_incrementally'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/licensing.rb:57:in `tap'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/licensing.rb:57:in `create_incrementally'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/project.rb:1075:in `build'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/cli.rb:89:in `build'
  /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.19.1/lib/thor/command.rb:27:in `run'
  /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in `invoke_command'
  /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.19.1/lib/thor.rb:359:in `dispatch'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/cli/base.rb:33:in `dispatch'
  /usr/local/lib/ruby/gems/2.5.0/gems/thor-0.19.1/lib/thor/base.rb:440:in `start'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/lib/omnibus/cli.rb:42:in `execute!'
  /usr/local/lib/ruby/gems/2.5.0/bundler/gems/omnibus-7e9c9a6a71c1/bin/omnibus:16:in `<top (required)>'
  /usr/local/lib/ruby/gems/2.5.0/bin/omnibus:23:in `load'
  /usr/local/lib/ruby/gems/2.5.0/bin/omnibus:23:in `<top (required)>'

@drbrain
Copy link
Author

drbrain commented May 25, 2018

@scotthain should I create a new issue, or is the update in the above comment enough to re-open this one?

@drbrain drbrain mentioned this issue Jun 12, 2018
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@drbrain @scotthain