New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix problems with volume_extend recipe #59

Merged
merged 1 commit into from Jun 10, 2016

Conversation

Projects
None yet
3 participants
@dpattmann
Contributor

dpattmann commented May 24, 2016

Hi @shortdudey123,

the behaviour of the LVM cookbook has changed in version > 2.0.

To use a newer version of the LVM cookbook someone has to rewrite the volume_extend.rb recipe. 馃懠

Best,
Dennis

Show outdated Hide outdated metadata.rb Outdated
@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 May 24, 2016

Owner

Also, can you rebase on master so tests pass :)

Owner

shortdudey123 commented May 24, 2016

Also, can you rebase on master so tests pass :)

@shortdudey123 shortdudey123 added the Bug label May 24, 2016

@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 May 24, 2016

Owner

Can you also file an issue here with the info about what lvm 2+ breaks and we will take a look at it

Owner

shortdudey123 commented May 24, 2016

Can you also file an issue here with the info about what lvm 2+ breaks and we will take a look at it

@dpattmann

This comment has been minimized.

Show comment
Hide comment
@dpattmann

dpattmann May 25, 2016

Contributor

From the LVM Changelog:

  • The gems are now installed when the provider is first used instead of in the default recipe.

The ruby gem is now missing...

Bug from Chef run

       ================================================================================
       Recipe Compile Error in /tmp/kitchen/cache/cookbooks/gluster/recipes/server.rb
       ================================================================================

       LoadError
       ---------
       cannot load such file -- lvm

       Cookbook Trace:
       ---------------
         /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:25:in `block in from_file'
         /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `each'
         /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `from_file'
         /tmp/kitchen/cache/cookbooks/gluster/recipes/server.rb:25:in `from_file'

       Relevant File Content:
       ----------------------
       /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:

        18:  # limitations under the License.
        19:  #
        20:  node['gluster']['server']['volumes'].each do |volume_name, volume_values|
        21:    # 1. Get the current size of the logical volume
        22:    # 2. Compare to the size set for the gluster volume
        23:    # 3. If different, run a resize action against that volume
        24:    # ToDO: change hardcoded VG name gluster into an attribute
        25>>   require 'lvm'
        26:  
        27:    LVM::LVM.new do |lvm|
        28:      lvm.logical_volumes.each do |lv|
        29:        # I'm ignoring these as I think this layout helps readability
        30:        # rubocop:disable Style/Next, Style/CaseIndentation, Lint/EndAlignment
        31:        if lv.full_name.to_s == "gluster/#{volume_name}"
        32:          lv_size_cur = lv.size.to_i
        33:          # Borrowed from the lvm cookbook
        34:          volume_lv_size_req = case volume_values['size']

       Platform:
       ---------
       x86_64-linux
Contributor

dpattmann commented May 25, 2016

From the LVM Changelog:

  • The gems are now installed when the provider is first used instead of in the default recipe.

The ruby gem is now missing...

Bug from Chef run

       ================================================================================
       Recipe Compile Error in /tmp/kitchen/cache/cookbooks/gluster/recipes/server.rb
       ================================================================================

       LoadError
       ---------
       cannot load such file -- lvm

       Cookbook Trace:
       ---------------
         /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:25:in `block in from_file'
         /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `each'
         /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `from_file'
         /tmp/kitchen/cache/cookbooks/gluster/recipes/server.rb:25:in `from_file'

       Relevant File Content:
       ----------------------
       /tmp/kitchen/cache/cookbooks/gluster/recipes/volume_extend.rb:

        18:  # limitations under the License.
        19:  #
        20:  node['gluster']['server']['volumes'].each do |volume_name, volume_values|
        21:    # 1. Get the current size of the logical volume
        22:    # 2. Compare to the size set for the gluster volume
        23:    # 3. If different, run a resize action against that volume
        24:    # ToDO: change hardcoded VG name gluster into an attribute
        25>>   require 'lvm'
        26:  
        27:    LVM::LVM.new do |lvm|
        28:      lvm.logical_volumes.each do |lv|
        29:        # I'm ignoring these as I think this layout helps readability
        30:        # rubocop:disable Style/Next, Style/CaseIndentation, Lint/EndAlignment
        31:        if lv.full_name.to_s == "gluster/#{volume_name}"
        32:          lv_size_cur = lv.size.to_i
        33:          # Borrowed from the lvm cookbook
        34:          volume_lv_size_req = case volume_values['size']

       Platform:
       ---------
       x86_64-linux
@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 May 25, 2016

Owner

hmm ok, since volume_extend should never expand a volume on the first chef run (since it was just created anyway) maybe a guard needs to be put on the include_recipe instead of chasing the LVM cookbook around again

Owner

shortdudey123 commented May 25, 2016

hmm ok, since volume_extend should never expand a volume on the first chef run (since it was just created anyway) maybe a guard needs to be put on the include_recipe instead of chasing the LVM cookbook around again

@dpattmann

This comment has been minimized.

Show comment
Hide comment
@dpattmann

dpattmann May 26, 2016

Contributor

Couldn't we just extend the server_install.rb recipe with the code from the lvm::default recipe in version 1.6.1 and remove the lvm cookbook completely from this cookbook?

Contributor

dpattmann commented May 26, 2016

Couldn't we just extend the server_install.rb recipe with the code from the lvm::default recipe in version 1.6.1 and remove the lvm cookbook completely from this cookbook?

@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 May 26, 2016

Owner

You would need to copy the LWRP's in as well and i would rather not do that

Owner

shortdudey123 commented May 26, 2016

You would need to copy the LWRP's in as well and i would rather not do that

@dpattmann

This comment has been minimized.

Show comment
Hide comment
@dpattmann

dpattmann May 27, 2016

Contributor

I found another problem with the LVM cookbook... 馃槖

If you have defined a volume like gv0 during the first chef-run gluster::volume_extend is executed before the package 'lvm2' resource from the lvm::default cookbook.

Synchronizing Cookbooks:
  - gluster (5.0.1)
  - apt (3.0.0)
  - compat_resource (12.9.1)
  - yum (3.8.2)
  - lvm (1.6.1)
Compiling Cookbooks...
Recipe: lvm::default
  * chef_gem[di-ruby-lvm-attrib] action install (up to date)
  * chef_gem[di-ruby-lvm] action install (up to date)
[2016-05-27T14:36:53+00:00] WARN: This server is not configured for this volume
[2016-05-27T14:36:53+00:00] WARN: I can't invite myself into the pool!

  ================================================================================
  Recipe Compile Error in /var/chef/cache/cookbooks/gluster/recipes/server.rb
  ================================================================================

  Errno::ENOENT
  -------------
  No such file or directory - /sbin/lvm

  Cookbook Trace:
  ---------------
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:27:in `new'
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:27:in `block in from_file'
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `each'
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `from_file'
    /var/chef/cache/cookbooks/gluster/recipes/server.rb:25:in `from_file'

  Relevant File Content:
  ----------------------
  /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:

   20:  node['gluster']['server']['volumes'].each do |volume_name, volume_values|
   21:    # 1. Get the current size of the logical volume
   22:    # 2. Compare to the size set for the gluster volume
   23:    # 3. If different, run a resize action against that volume
   24:    # ToDO: change hardcoded VG name gluster into an attribute
   25:    require 'lvm'
   26:  
   27>>   LVM::LVM.new do |lvm|
   28:      lvm.logical_volumes.each do |lv|
   29:        # I'm ignoring these as I think this layout helps readability
   30:        # rubocop:disable Style/Next, Style/CaseIndentation, Lint/EndAlignment
   31:        if lv.full_name.to_s == "gluster/#{volume_name}"
   32:          lv_size_cur = lv.size.to_i
   33:          # Borrowed from the lvm cookbook
   34:          volume_lv_size_req = case volume_values['size']
   35:            when /^(\d+)(k|K)$/
   36:              (Regexp.last_match[1].to_i * 1024)


  Running handlers:
[2016-05-27T14:36:53+00:00] ERROR: Running exception handlers
  Running handlers complete
[2016-05-27T14:36:53+00:00] ERROR: Exception handlers complete
  Chef Client failed. 0 resources updated in 01 seconds
[2016-05-27T14:36:53+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2016-05-27T14:36:53+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2016-05-27T14:36:53+00:00] ERROR: No such file or directory - /sbin/lvm
[2016-05-27T14:36:53+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

So I think we should install the lvm package and ruby-lvm gems inside the gluster::server_install.rb recipe and include a LVM cookbook >= 1.5.1 to have the LWPR's accessible.

Contributor

dpattmann commented May 27, 2016

I found another problem with the LVM cookbook... 馃槖

If you have defined a volume like gv0 during the first chef-run gluster::volume_extend is executed before the package 'lvm2' resource from the lvm::default cookbook.

Synchronizing Cookbooks:
  - gluster (5.0.1)
  - apt (3.0.0)
  - compat_resource (12.9.1)
  - yum (3.8.2)
  - lvm (1.6.1)
Compiling Cookbooks...
Recipe: lvm::default
  * chef_gem[di-ruby-lvm-attrib] action install (up to date)
  * chef_gem[di-ruby-lvm] action install (up to date)
[2016-05-27T14:36:53+00:00] WARN: This server is not configured for this volume
[2016-05-27T14:36:53+00:00] WARN: I can't invite myself into the pool!

  ================================================================================
  Recipe Compile Error in /var/chef/cache/cookbooks/gluster/recipes/server.rb
  ================================================================================

  Errno::ENOENT
  -------------
  No such file or directory - /sbin/lvm

  Cookbook Trace:
  ---------------
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:27:in `new'
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:27:in `block in from_file'
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `each'
    /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:20:in `from_file'
    /var/chef/cache/cookbooks/gluster/recipes/server.rb:25:in `from_file'

  Relevant File Content:
  ----------------------
  /var/chef/cache/cookbooks/gluster/recipes/volume_extend.rb:

   20:  node['gluster']['server']['volumes'].each do |volume_name, volume_values|
   21:    # 1. Get the current size of the logical volume
   22:    # 2. Compare to the size set for the gluster volume
   23:    # 3. If different, run a resize action against that volume
   24:    # ToDO: change hardcoded VG name gluster into an attribute
   25:    require 'lvm'
   26:  
   27>>   LVM::LVM.new do |lvm|
   28:      lvm.logical_volumes.each do |lv|
   29:        # I'm ignoring these as I think this layout helps readability
   30:        # rubocop:disable Style/Next, Style/CaseIndentation, Lint/EndAlignment
   31:        if lv.full_name.to_s == "gluster/#{volume_name}"
   32:          lv_size_cur = lv.size.to_i
   33:          # Borrowed from the lvm cookbook
   34:          volume_lv_size_req = case volume_values['size']
   35:            when /^(\d+)(k|K)$/
   36:              (Regexp.last_match[1].to_i * 1024)


  Running handlers:
[2016-05-27T14:36:53+00:00] ERROR: Running exception handlers
  Running handlers complete
[2016-05-27T14:36:53+00:00] ERROR: Exception handlers complete
  Chef Client failed. 0 resources updated in 01 seconds
[2016-05-27T14:36:53+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2016-05-27T14:36:53+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2016-05-27T14:36:53+00:00] ERROR: No such file or directory - /sbin/lvm
[2016-05-27T14:36:53+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

So I think we should install the lvm package and ruby-lvm gems inside the gluster::server_install.rb recipe and include a LVM cookbook >= 1.5.1 to have the LWPR's accessible.

@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 May 27, 2016

Owner

lvm::default is run before gluster::volume_extend per gluster::server which installs the gems but not the package yet is what you mean?

Owner

shortdudey123 commented May 27, 2016

lvm::default is run before gluster::volume_extend per gluster::server which installs the gems but not the package yet is what you mean?

@dpattmann

This comment has been minimized.

Show comment
Hide comment
@dpattmann

dpattmann May 30, 2016

Contributor

Yes. 馃槈

The lvm::default recipe is included before gluster::volume_extend but that doesn鈥檛 mean that chef is going to execute all resources at this time. Chef sometimes doesn鈥檛 care about the order of included recipes when you include them from different cookbook. 馃槥

...
...
...

I made some more tests and I think we must install the lvm2 package during the compile phase like the ruby lvm gems. 馃槖

Contributor

dpattmann commented May 30, 2016

Yes. 馃槈

The lvm::default recipe is included before gluster::volume_extend but that doesn鈥檛 mean that chef is going to execute all resources at this time. Chef sometimes doesn鈥檛 care about the order of included recipes when you include them from different cookbook. 馃槥

...
...
...

I made some more tests and I think we must install the lvm2 package during the compile phase like the ruby lvm gems. 馃槖

@dpattmann

This comment has been minimized.

Show comment
Hide comment
@dpattmann

dpattmann May 30, 2016

Contributor

I updated my pull request to fix this issue. :-)

Contributor

dpattmann commented May 30, 2016

I updated my pull request to fix this issue. :-)

@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 Jun 3, 2016

Owner

Adding lvm2 to the package list and installing it compile phase seems reasonable

maybe a guard needs to be put on the include_recipe instead of chasing the LVM cookbook around again

Thoughts on ^^?

Owner

shortdudey123 commented Jun 3, 2016

Adding lvm2 to the package list and installing it compile phase seems reasonable

maybe a guard needs to be put on the include_recipe instead of chasing the LVM cookbook around again

Thoughts on ^^?

@dpattmann

This comment has been minimized.

Show comment
Hide comment
@dpattmann

dpattmann Jun 6, 2016

Contributor

That's a good idea, maybe we should check if di-ruby-lvm is installed.

include_recipe 'gluster::volume_extend' if Gem::Specification.find_by_name('di-ruby-lvm')

How about that?

Contributor

dpattmann commented Jun 6, 2016

That's a good idea, maybe we should check if di-ruby-lvm is installed.

include_recipe 'gluster::volume_extend' if Gem::Specification.find_by_name('di-ruby-lvm')

How about that?

@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 Jun 6, 2016

Owner

That would work since server_setup.rb would call the LVM LWRP and have the gem installed

Add that guard and validate that it works correctly

Owner

shortdudey123 commented Jun 6, 2016

That would work since server_setup.rb would call the LVM LWRP and have the gem installed

Add that guard and validate that it works correctly

@dpattmann

This comment has been minimized.

Show comment
Hide comment
@dpattmann

dpattmann Jun 7, 2016

Contributor

I updated my PR :-)

Contributor

dpattmann commented Jun 7, 2016

I updated my PR :-)

Show outdated Hide outdated recipes/server.rb Outdated
Show outdated Hide outdated recipes/server.rb Outdated
@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 Jun 9, 2016

Owner

Can you also add a line to CHANGELOG.md with your PR link and title near the top in the Unreleased section?

Owner

shortdudey123 commented Jun 9, 2016

Can you also add a line to CHANGELOG.md with your PR link and title near the top in the Unreleased section?

@dpattmann dpattmann changed the title from Use LVM cookbook in a version lower then 2.0 to Fix problems with volume_extend recipe Jun 10, 2016

@dpattmann dpattmann closed this Jun 10, 2016

Install lvm package during compile phase and don't execute volume_ext鈥
鈥nd recipe unless di-ruby-lvm gem is installed

@dpattmann dpattmann reopened this Jun 10, 2016

@shortdudey123

This comment has been minimized.

Show comment
Hide comment
@shortdudey123
Owner

shortdudey123 commented Jun 10, 2016

@dpattmann thanks!!

@shortdudey123 shortdudey123 merged commit 128da38 into shortdudey123:master Jun 10, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
@laurencepettitt

This comment has been minimized.

Show comment
Hide comment
@laurencepettitt

laurencepettitt Jun 22, 2016

Is it worth bumping the version number after this commit as it is a bug fix?
I am having trouble with berkshelf getting this commit because it sees no difference between this and any other commit at version 5.0.1.

laurencepettitt commented on f46d411 Jun 22, 2016

Is it worth bumping the version number after this commit as it is a bug fix?
I am having trouble with berkshelf getting this commit because it sees no difference between this and any other commit at version 5.0.1.

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 Jun 22, 2016

Owner

want to close one other issue before bumping the version
Do bundle exec berks update gluster and it will pick up this one assuming you have the correct commit hash in your Berksfile

Owner

shortdudey123 replied Jun 22, 2016

want to close one other issue before bumping the version
Do bundle exec berks update gluster and it will pick up this one assuming you have the correct commit hash in your Berksfile

This comment has been minimized.

Show comment
Hide comment
@laurencepettitt

laurencepettitt Jun 23, 2016

Great thanks, it is working now. However I am still not exactly sure why! If you have any interest, although its not specific to your cookbook, perhaps you might be able to throw some light on this quirky thing:

I have a cookbook called 'servu-stack' which depends upon a cookbook called 'servu-gluster' which is a wrapper cookbook and thus depends upon this cookbook, called 'gluster'. Dependancy tree like this:

servu-stack
聽聽鈫 servu-gluster
聽聽聽聽聽聽鈫 gluster

In the 'servu-gluster' cookbook's Berksfile it defines this commit:

cookbook 'gluster', git: 'https://github.com/shortdudey123/chef-gluster.git', :ref => '128da'

Running berks install in the 'servu-gluster' cookbook correctly pulls down this cookbook into the Berkshelf, at this commit.

However, running berks install in the 'servu-stack' cookbook, pulls this cookbook into the Berkshelf at the wrong (previous) commit.

So I added into the Berksfile of the 'servu-stack' cookbook a line, exactly like above:

cookbook 'gluster', git: 'https://github.com/shortdudey123/chef-gluster.git', :ref => '128da'

This meant the 'servu-stack' now resolves to the correct dependancy. However, I do not understand why this is necessary, as surely Berkshelf should look down the dependancy tree and resolve the correct dependency of a dependant, in the same way as the dependant does?

laurencepettitt replied Jun 23, 2016

Great thanks, it is working now. However I am still not exactly sure why! If you have any interest, although its not specific to your cookbook, perhaps you might be able to throw some light on this quirky thing:

I have a cookbook called 'servu-stack' which depends upon a cookbook called 'servu-gluster' which is a wrapper cookbook and thus depends upon this cookbook, called 'gluster'. Dependancy tree like this:

servu-stack
聽聽鈫 servu-gluster
聽聽聽聽聽聽鈫 gluster

In the 'servu-gluster' cookbook's Berksfile it defines this commit:

cookbook 'gluster', git: 'https://github.com/shortdudey123/chef-gluster.git', :ref => '128da'

Running berks install in the 'servu-gluster' cookbook correctly pulls down this cookbook into the Berkshelf, at this commit.

However, running berks install in the 'servu-stack' cookbook, pulls this cookbook into the Berkshelf at the wrong (previous) commit.

So I added into the Berksfile of the 'servu-stack' cookbook a line, exactly like above:

cookbook 'gluster', git: 'https://github.com/shortdudey123/chef-gluster.git', :ref => '128da'

This meant the 'servu-stack' now resolves to the correct dependancy. However, I do not understand why this is necessary, as surely Berkshelf should look down the dependancy tree and resolve the correct dependency of a dependant, in the same way as the dependant does?

This comment has been minimized.

Show comment
Hide comment
@shortdudey123

shortdudey123 Jun 23, 2016

Owner

Berkshelf only locks its dependency tree for the Berksfile you give it. It doesn't do that if you do berks install between Berksfile's

Owner

shortdudey123 replied Jun 23, 2016

Berkshelf only locks its dependency tree for the Berksfile you give it. It doesn't do that if you do berks install between Berksfile's

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment