Respect chef-solo-search paths

[skyeagle]

There is some discrepancy. The chef-solo-search uses data bags for nodes data but this gem uses nodes root path pointed out in readme.

[matschaffer]

Oh woah, I didn't notice that and assumed it only did data bag searching. Interesting trick :)

I'd be willing to go with that, though we will probably have to allow a fallback to nodes with a deprecation warning. We should also start really using chef-solo-search rather than the copied-in code we have now, but I think there's another ticket for that.

[skyeagle]

Btw, why are they use node instead of nodes path inside data_bags? Maybe it should be discussed with the authors of chef-solo-search.

[matschaffer]

Couldn't hurt to ask. Seems like it could be just a style decision.

On Oct 25, 2012, at 15:18, Anton Orel notifications@github.com wrote:

Btw, why are they use node instead of nodes path inside data_bags? Maybe it
should be discussed with the authors of chef-solo-search.

—
Reply to this email directly or view it on
GitHubhttps://github.com//issues/117#issuecomment-9790390.

[tjwallace]

I'm sure you've all figured this out already, but I symlink'd data_bags/node to nodes for a quick fix.

[tjwallace]

Actually on further inspect the symlink won't work. chef-solo-search expects the node json files to be in this format:

{
  "id": "vagrant",
  "name": "vagrant-vm",
  "chef_environment": "_default",
  "json_class": "Chef::Node",
  "chef_type": "node",
  "automatic": {},
  "normal": {},
  "default": {
    "attribute": "value"
  },
  "override": {},
  "run_list": [
    "role[monitoring]"
  ]
}

while chef-solo -j expects it to be in this format:

{
  "attribute": "value",
  "run_list": [
    "role[monitoring]"
  ]
}

I've opened a pull request for chef (chef/chef#673) to allow the chef-solo -j option to take a json file like the first.

We could generate the smaller json files from the large ones when we call knife solo cook:

large_json = JSON.parse(File.read('large.json'))
small_json = Chef::Node.json_create(large_json).to_hash.to_json

@skyeagle I think it was named node instead of nodes because search with a chef server uses search(:node, ...).

[skyeagle]

@tjwallace thanks for investigation and clear explanation.

[tmatilai]

@tjwallace wow, nice job!

After #199 the symlink on the local machine does not help any more, but that is not a problem as we will control the directory names on the node and thus we can use "node" instead of "nodes", even if it is inconsistent with other names.

[tmatilai]

Ah it is './data_bags/node/', not './node/'. Not enough coffee yet.
I'm afraid we can not safely unconditionally use a directory inside 'data_bags/'. We could check if 'data_bags/node' exist and do some magic if not, but I think a better approach would be to add configuration for chef-solo-search. The relevant line is currently here.

[tjwallace]

I've created a pull request on chef-solo-search (edelight/chef-solo-search#19) that would allow node files written in both formats to be searchable. This would allo the node symlink to work.

As an interm solution, I've created a symlink from ./data_bags/nodes to ./nodes. It can be searched with search(:nodes).

[tmatilai]

@tjwallace did you see the previous commit on that file? Isn't that already enough?

And as I said, the symlink won't work in the next knife-solo release, as we won't sync the whole directory structure any more. cook could create the symlink on the node (after checking that data_bags/node does not exist), but I think a better option would be to add optional Chef::Config[:node_path] to chef-solo-search.

[tjwallace]

@tmatilai no Chef::Node.json_create will not accept the same data format as chef-solo -j. The best solution is to have chef/chef#673 merged in, and then all of the node json files can be written in the more detailed format (with a json_class field), and both chef-solo -j and chef-solo-search will be able to read the same json files.

Also it looks like #199 will sync the entire data_bags directory - so if there is a symlink there it will get sync'd as well?

[tmatilai]

OK, I haven't looked at all what those methods do. Just saw the commit message.

Also it looks like #199 will sync the entire data_bags directory - so if there is a symlink there it will get sync'd as well?

Ah, you're right! Still smells a bit of a kludge. =)

[tmatilai]

@tjwallace Now that we have full control of solo.rb this is easier to address. I think we could:

• Dynamically generate simplified dna.json from the ./nodes/name.json (and/or from command line parameters) that gets passed to chef-solo -j
• Generate more complete ./nodes/name.json in NodeConfigCommand#generate_node_config

I created edelight/chef-solo-search#28 to be able to pass our uploaded nodes directory to chef-solo-search, but that is not a blocker for this issue.

Then I had a wild idea that we could maybe even allow chef-solo run to save updated node attributes to the original name.json by overriding Chef::Node#save and Chef::Client#save_updated_node. I hacked a quick spike and it could really work. This kind of feature might even fit into chef-solo-search itself.

[tjwallace]

@tmatilai I like the idea of generating simplified dna.json files specifically for chef-solo -j - and it should be easy to do. And having the ability to save updated node attributes is a great idea!