New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The version of elasticsearch can only be set via elasticsearch/settings databag #68

Closed
jakelandis opened this Issue Feb 27, 2013 · 10 comments

Comments

Projects
None yet
2 participants
@jakelandis

jakelandis commented Feb 27, 2013

While trying to set the (non-default) version via a chef environment [1] attribute it does not actually install that version. It will install the default version, unless you set the version via the elasticsearch/settings.

To reproduce:

  1. If using chef-server, navigate to the node, edit, attributes (bottom left), select elasticsearch, delete (red minus button), save node. (this is to get the attributes to a clean state)
  2. Delete the version from the elasticsearch/settings data bag (or the entire data bag).
  3. Set the version to a non-default value via an environment (a role would probably show the same behavior).
  4. Run chef-client (with the -l debug to see the version)- it will install the default version, not the version specified by the environment.

This was reproduced with the 0.20.0 version of the cookbook.

[1]
"elasticsearch" => {
"version" => "0.20.5"

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Feb 27, 2013

Member

Hi, Chef attribute behaviour is notoriously tricky, if memory serves well. The Chef 11 changelog hints at some bugfixes. Are you 100% sure that you're setting the attribute in environment properly? Can you test this with Chef 11?

Member

karmi commented Feb 27, 2013

Hi, Chef attribute behaviour is notoriously tricky, if memory serves well. The Chef 11 changelog hints at some bugfixes. Are you 100% sure that you're setting the attribute in environment properly? Can you test this with Chef 11?

karmi added a commit that referenced this issue Feb 27, 2013

[FIX] Fix incorrect attribute setting in `default.rb`
The default.rb attributes file used Chef 11 incompatible syntax for setting the node default attribute,
throwing the `Chef::Exceptions::ImmutableAttributeModification` exception.

See: http://www.opscode.com/blog/2013/02/05/chef-11-in-depth-attributes-changes/

Related: #68, #69
@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Feb 27, 2013

Member

OK, spent couple of hours hunting this. Notes:

  • There were fatal errors regarding setting attributes on Chef 11, which are now fixed (see attached commit)
  • I was able to successfully and consistently upgrade/downgrade Elasticsearch in Chef Server and Chef Solo

With Chef Solo, the game is all the same: change configurations in Vagrant file or data bag, run vagrant provision

With Chef Server (hosted at Opscode), this are the steps I took:

First, upload all the cookbooks to Chef Server:

berks upload

Create a new virtual machine in Vagrant:

bundle exec vagrant up precise64 --no-provision

Create a knife.rb configuration file for Chef (use bundle exec vagrant ssh-config precise64 on Vagrant),
specifying correct Chef API endpoint, SSH users, ports, identity files, etc.

Then, bootstrap the machine via Chef:

knife bootstrap 127.0.0.1 --run-list 'role[elasticsearch-vagrant]' --config ./tmp/knife-vagrant.rb --sudo

Now, the important part, to change the version with

a) environment,
b) node attributes

To change the version with environment, simply create it with Chef web console or knife,
and set overrides attribute to a different version. Run Chef on the machine:

sudo chef-client

I believe this will work the same with roles.

To change the version by editing node attributes, edit it with:

knife node edit precise64 --config ./tmp/knife-vagrant.rb

and run chef.

In all cases, the correct version is downloaded, symlinked, and Elasticsearch itself is restarted.

Verify this workflow on your end, please, and get back to me.

Member

karmi commented Feb 27, 2013

OK, spent couple of hours hunting this. Notes:

  • There were fatal errors regarding setting attributes on Chef 11, which are now fixed (see attached commit)
  • I was able to successfully and consistently upgrade/downgrade Elasticsearch in Chef Server and Chef Solo

With Chef Solo, the game is all the same: change configurations in Vagrant file or data bag, run vagrant provision

With Chef Server (hosted at Opscode), this are the steps I took:

First, upload all the cookbooks to Chef Server:

berks upload

Create a new virtual machine in Vagrant:

bundle exec vagrant up precise64 --no-provision

Create a knife.rb configuration file for Chef (use bundle exec vagrant ssh-config precise64 on Vagrant),
specifying correct Chef API endpoint, SSH users, ports, identity files, etc.

Then, bootstrap the machine via Chef:

knife bootstrap 127.0.0.1 --run-list 'role[elasticsearch-vagrant]' --config ./tmp/knife-vagrant.rb --sudo

Now, the important part, to change the version with

a) environment,
b) node attributes

To change the version with environment, simply create it with Chef web console or knife,
and set overrides attribute to a different version. Run Chef on the machine:

sudo chef-client

I believe this will work the same with roles.

To change the version by editing node attributes, edit it with:

knife node edit precise64 --config ./tmp/knife-vagrant.rb

and run chef.

In all cases, the correct version is downloaded, symlinked, and Elasticsearch itself is restarted.

Verify this workflow on your end, please, and get back to me.

@jakelandis

This comment has been minimized.

Show comment
Hide comment
@jakelandis

jakelandis Feb 27, 2013

I was able to verify the latest code + chef 11 (client) resolves this issue.

Using the latest code revision (ac4bb01) , Chef Server 10.? , and changing the version of chef on the client node.

Chef: 10.24.0 (incorrectly downloads and installs the default version in the specified version directory)

New install of 0.20.2 (version set via environment) notice the jar version in the lib directory:

   [2013-02-27T09:58:52-06:00] INFO: *** version: 0.20.5
   [2013-02-27T09:58:52-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.5.tar.gz

  ls -l /usr/local/elasticsearch
  /usr/local/elasticsearch -> /usr/local/elasticsearch-0.20.2
 ls  /usr/local/elasticsearch-0.20.2/lib
 elasticsearch-0.20.5.jar  ....

Update to 0.20.3 (version set via environment) notice the jar version in the lib directory:

 [2013-02-27T10:03:02-06:00] INFO: *** version: 0.20.5
 [2013-02-27T10:03:02-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.5.tar.gz

 ls -l /usr/local/elasticsearch
 /usr/local/elasticsearch -> /usr/local/elasticsearch-0.20.3
 ls /usr/local/elasticsearch-0.20.3/lib
 elasticsearch-0.20.5.jar  .....
Chef: 11.4.0 (downloads and installs the correct version)

New install of 0.20.2 (version set via environment):

      [2013-02-27T09:48:08-06:00] INFO: *** version: 0.20.2
      [2013-02-27T09:48:08-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.2.tar.gz

Update to 0.20.3 (version set via environment):

      [2013-02-27T09:51:24-06:00] INFO: *** version: 0.20.3
      [2013-02-27T09:51:24-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.3.tar.gz

Note:
I added the following lines to attributes/default.rb for debugging:
Chef::Log.info "* version: #{node.elasticsearch[:version]}"
Chef::Log.info "
* download_url: " + default.elasticsearch[:download_url]

jakelandis commented Feb 27, 2013

I was able to verify the latest code + chef 11 (client) resolves this issue.

Using the latest code revision (ac4bb01) , Chef Server 10.? , and changing the version of chef on the client node.

Chef: 10.24.0 (incorrectly downloads and installs the default version in the specified version directory)

New install of 0.20.2 (version set via environment) notice the jar version in the lib directory:

   [2013-02-27T09:58:52-06:00] INFO: *** version: 0.20.5
   [2013-02-27T09:58:52-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.5.tar.gz

  ls -l /usr/local/elasticsearch
  /usr/local/elasticsearch -> /usr/local/elasticsearch-0.20.2
 ls  /usr/local/elasticsearch-0.20.2/lib
 elasticsearch-0.20.5.jar  ....

Update to 0.20.3 (version set via environment) notice the jar version in the lib directory:

 [2013-02-27T10:03:02-06:00] INFO: *** version: 0.20.5
 [2013-02-27T10:03:02-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.5.tar.gz

 ls -l /usr/local/elasticsearch
 /usr/local/elasticsearch -> /usr/local/elasticsearch-0.20.3
 ls /usr/local/elasticsearch-0.20.3/lib
 elasticsearch-0.20.5.jar  .....
Chef: 11.4.0 (downloads and installs the correct version)

New install of 0.20.2 (version set via environment):

      [2013-02-27T09:48:08-06:00] INFO: *** version: 0.20.2
      [2013-02-27T09:48:08-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.2.tar.gz

Update to 0.20.3 (version set via environment):

      [2013-02-27T09:51:24-06:00] INFO: *** version: 0.20.3
      [2013-02-27T09:51:24-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.3.tar.gz

Note:
I added the following lines to attributes/default.rb for debugging:
Chef::Log.info "* version: #{node.elasticsearch[:version]}"
Chef::Log.info "
* download_url: " + default.elasticsearch[:download_url]

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Feb 27, 2013

Member

So, on Chef 10.24.0 it still downloads the incorrect version? Trying it with Chef Server now.

Note: The ark cookbook should print a line with the download URL.

Member

karmi commented Feb 27, 2013

So, on Chef 10.24.0 it still downloads the incorrect version? Trying it with Chef Server now.

Note: The ark cookbook should print a line with the download URL.

@jakelandis

This comment has been minimized.

Show comment
Hide comment
@jakelandis

jakelandis Feb 27, 2013

Hmm... trying to repeat the test above, but specifying the version via the elasticsearch/settings data bag. It appears in both 10.x and 11.x then version is now being ignored:

[2013-02-27T11:08:41-06:00] INFO: Loaded settings: data_bag_item["elasticsearch", "settings", {"id"=>"settings", "version"=>"0.20.2"}]
[2013-02-27T11:08:41-06:00] INFO: *** version: 0.20.5
[2013-02-27T11:08:41-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.5.tar.gz

Should i log a separate issue?

jakelandis commented Feb 27, 2013

Hmm... trying to repeat the test above, but specifying the version via the elasticsearch/settings data bag. It appears in both 10.x and 11.x then version is now being ignored:

[2013-02-27T11:08:41-06:00] INFO: Loaded settings: data_bag_item["elasticsearch", "settings", {"id"=>"settings", "version"=>"0.20.2"}]
[2013-02-27T11:08:41-06:00] INFO: *** version: 0.20.5
[2013-02-27T11:08:41-06:00] INFO: *** download_url: http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.5.tar.gz

Should i log a separate issue?

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Feb 27, 2013

Member

I'll try also the data_bag way with Chef Solo and Vagrant, still debugging Chef 10.22.0

Member

karmi commented Feb 27, 2013

I'll try also the data_bag way with Chef Solo and Vagrant, still debugging Chef 10.22.0

@karmi karmi closed this in 9a68134 Feb 27, 2013

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Feb 27, 2013

Member

Added a guard against overwriting the version by using the normal precedence, should be working now either with Chef Solo or Chef Server.

Member

karmi commented Feb 27, 2013

Added a guard against overwriting the version by using the normal precedence, should be working now either with Chef Solo or Chef Server.

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Feb 28, 2013

Member

@jakelandis Could you verify the current master, please?

Member

karmi commented Feb 28, 2013

@jakelandis Could you verify the current master, please?

@jakelandis

This comment has been minimized.

Show comment
Hide comment
@jakelandis

jakelandis Feb 28, 2013

Testing against 2.0.4 tag, with chef server (10.x)

Chef: 11.4.0

  • Install version 0.20.2
    • version configured via databag - GOOD
    • version configured via environment - GOOD
  • Update to version 0.20.3
    • version configured via databag - GOOD
    • version configured via environment - GOOD

Chef: 10.24.0

  • Install version 0.20.2
    • version configured via databag - GOOD
    • version configured via environment - BAD [2]
  • Update to version 0.20.3
    • version configured via databag - BAD [1]
    • version configured via environment - BAD [3]

[1] It is downloading the prior version ... but there are stale values in the node (it would work if you delete and re-add back the node, or delete the node settings prior to running chef-client)
ls elasticsearch-0.20.3/lib/
elasticsearch-0.20.2.jar
From chef server node JSON:
{"version":"0.20.3","host":"http://download.elasticsearch.org","repository":"elasticsearch/elasticsearch","filename":"elasticsearch-0.20.2.tar.gz","download_url":"http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.2.tar.gz",.....

[2] It is downloading the default version in the specified version's path
ls elasticsearch-0.20.2/lib
elasticsearch-0.20.5.jar

[3] It is downloading the default version in the specified version's path
ls elasticsearch-0.20.3/lib
elasticsearch-0.20.5.jar

jakelandis commented Feb 28, 2013

Testing against 2.0.4 tag, with chef server (10.x)

Chef: 11.4.0

  • Install version 0.20.2
    • version configured via databag - GOOD
    • version configured via environment - GOOD
  • Update to version 0.20.3
    • version configured via databag - GOOD
    • version configured via environment - GOOD

Chef: 10.24.0

  • Install version 0.20.2
    • version configured via databag - GOOD
    • version configured via environment - BAD [2]
  • Update to version 0.20.3
    • version configured via databag - BAD [1]
    • version configured via environment - BAD [3]

[1] It is downloading the prior version ... but there are stale values in the node (it would work if you delete and re-add back the node, or delete the node settings prior to running chef-client)
ls elasticsearch-0.20.3/lib/
elasticsearch-0.20.2.jar
From chef server node JSON:
{"version":"0.20.3","host":"http://download.elasticsearch.org","repository":"elasticsearch/elasticsearch","filename":"elasticsearch-0.20.2.tar.gz","download_url":"http://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-0.20.2.tar.gz",.....

[2] It is downloading the default version in the specified version's path
ls elasticsearch-0.20.2/lib
elasticsearch-0.20.5.jar

[3] It is downloading the default version in the specified version's path
ls elasticsearch-0.20.3/lib
elasticsearch-0.20.5.jar

@karmi

This comment has been minimized.

Show comment
Hide comment
@karmi

karmi Mar 1, 2013

Member

Thanks for all this testing! I think what we're hitting here is this issue, where the node attributes are indeed “stale”. I think it's fair to advice using Chef 11 if anybody needs to configure the version attribute from role/environment...

Member

karmi commented Mar 1, 2013

Thanks for all this testing! I think what we're hitting here is this issue, where the node attributes are indeed “stale”. I think it's fair to advice using Chef 11 if anybody needs to configure the version attribute from role/environment...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment