Documentation on kitchen functions

[dgarstang]

Would it possible to provide some documentation on what all the kitchen commands are and specifically what they do? i.e. 'converge', 'verify', 'test' etc. I can't find this anywhere including the getting started guide.

[spheromak]

kitchen help
Commands:
  kitchen console                                 # Kitchen Console!
  kitchen converge [INSTANCE|REGEXP|all]          # Converge one or more instances
  kitchen create [INSTANCE|REGEXP|all]            # Create one or more instances
  kitchen destroy [INSTANCE|REGEXP|all]           # Destroy one or more instances
  kitchen diagnose [INSTANCE|REGEXP|all]          # Show computed diagnostic configuration
  kitchen driver                                  # Driver subcommands
  kitchen driver create [NAME]                    # Create a new Kitchen Driver gem project
  kitchen driver discover                         # Discover Test Kitchen drivers published on RubyGems
  kitchen driver help [COMMAND]                   # Describe subcommands or one specific subcommand
  kitchen exec INSTANCE|REGEXP -c REMOTE_COMMAND  # Execute command on one or more instance
  kitchen help [COMMAND]                          # Describe available commands or one specific command
  kitchen init                                    # Adds some configuration to your cookbook so Kitchen can rock
  kitchen list [INSTANCE|REGEXP|all]              # Lists one or more instances
  kitchen login INSTANCE|REGEXP                   # Log in to one instance
  kitchen setup [INSTANCE|REGEXP|all]             # Setup one or more instances
  kitchen test [INSTANCE|REGEXP|all]              # Test one or more instances
  kitchen verify [INSTANCE|REGEXP|all]            # Verify one or more instances
  kitchen version                                 # Print Kitchen's version information

[dgarstang]

I'm aware of the cli help. However, it really isn't clear what 'setup', 'test', 'verify' and 'converge' and so on actually do.

[sethvargo]

$ kitchen help converge
Usage:
  kitchen converge [INSTANCE|REGEXP|all]

Options:
  -c, [--concurrency=N]              # Run a converge against all matching instances concurrently. Only N instances will run at the same time if a number is given.
  -p, [--parallel], [--no-parallel]  # [Future DEPRECATION, use --concurrency] Run a converge against all matching instances concurrently.
  -l, [--log-level=LOG_LEVEL]        # Set the log level (debug, info, warn, error, fatal)

Converge one or more instances

If you describe what you would like to see in the descriptions, we can update them. It's rather unclear to me what you are asking.

[scotthain]

I think the kitchen.ci site actually has a pretty good description of the commands and what they do in relation to a workflow. It also has some extra examples and steps that are common, but it does go into detail about create, converge, verify, test, etc.

A good place to start is here: http://kitchen.ci/docs/getting-started/creating-cookbook

[dgarstang]

I'm not seeing anything. I guess I'll write a script to crawl all the docs, and then grep for 'kitchen setup' and 'kitchen verify' to plead my case.

[sethvargo]

... @dgarstang why the passive aggressive attitude? I asked you to clarify what you are looking for and you come back angry. What would you expect to see? Where do you expect to see it?

You've effectively said "it's broken", but have given me no indication of where/what to fix or would you think the fix should be. We would love to help, but you need to clearly explain the issue.

[MarkGibbons]

I had to spend a couple days in the code to figure out what the kitchen commands did. Most of the people I'm trying to get to use test-kitchen are not going to do that. I'm working a group of brand new chef users and the test-kitchen documentation and cli help are not making enough sense. Here's a couple places we are running into difficulties. I've tried writing up my notes a couple times and never came up with anything that made sense to contribute.

• The circular definitions are not clear and are frustrating. In the cli output below kitchen help converge => "run a converge", ok but what does converge mean? A summary of what the commands do would be nice. I just went through kitchen.ci looking for simple descriptions of what the commands do and couldn't find clear basic descriptions.
• I don't see kitchen setup command described in kitchen.ci. Setup is mentioned as part of verify, but it's not clear that "setup" in that context is the same as the kitchen setup command. The sequence init, converge, login, verify, test is described nicely but the description is very much clearer to someone that knows what to expect than it is to a beginner. Showing the output of the runs, see http://kitchen.ci/docs/getting-started/running-converge, would be helped with a summary of what will happen. Looking in the middle of the page for the simple explanation, well tl;dr in too many cases.
• One thing that is really not clear is that the steps imply that preceding steps will be done. I've been asked: What order would I run setup, vs converge, vs create, vs verify? If I run verify do I need to run any of the other commands? Shouldn't I do setup before init? The order of processing is the opposite of some people's intuition.
• It's not clear that test wipes it all out and runs through all the steps.
• On a side note, it's also really hard to find out how to code .kitchen.yml. Running kitchen diagnose seems like the best way to find the hidden options. Seems obscure though.

[dgarstang]

Thanks Mark. I completely agree with your assessment. Thanks for taking the time to write that. I got as far as this before I saw your reply...

wget --limit-rate=200k --no-clobber --convert-links --random-wait -r -p -E -e robots=off -U mozilla http://kitchen.ci/docs/getting-started/
grep -r "kitchen setup" <-- nothing except for the page with the cli help output.

Doug.

[sethvargo]

@MarkGibbons

On a side note, it's also really hard to find out how to code .kitchen.yml. Running kitchen diagnose seems like the best way to find the hidden options. Seems obscure though.

Let's keep this out of this conversation. I do agree the options are a bit magical, but let's keep this to CLI documentation to avoid going down a rabbit hole. Please open another issue to track the lack of docs around .kitchen.yml configs.

[damm]

I think the problem for a lot of people coming to kitchen are there are a lot of commands.

Don't take this harshly I just think a lot of people won't use every function and for a new user it can be scary to see too many knobs.

kitchen help
Starting Out:
  kitchen init                                    # Adds some configuration to your cookbook 

Setting it up:
  kitchen converge [INSTANCE|REGEXP|all]          # Converge one or more 

Using it:
  kitchen converge [INSTANCE|REGEXP|all]          # Converge one or more 
  kitchen test [INSTANCE|REGEXP|all]              # Test one or more instances
  kitchen verify [INSTANCE|REGEXP|all]            # Verify one or more instances

Shutting it Down:
  kitchen destroy [INSTANCE|REGEXP|all]           # Destroy one or more instances

When it breaks:
  kitchen diagnose [INSTANCE|REGEXP|all]          # Show computed diagnostic configuration
  kitchen login INSTANCE|REGEXP                   # Log in to one instance

Everything else:
  kitchen list [INSTANCE|REGEXP|all]              # Lists one or more instances
kitchen --help --all
Commands:
  kitchen console                         # Kitchen Console!
  kitchen converge [INSTANCE|REGEXP|all]  # Converge one or more instances
  kitchen create [INSTANCE|REGEXP|all]    # Create one or more instances
  kitchen destroy [INSTANCE|REGEXP|all]   # Destroy one or more instances
  kitchen diagnose [INSTANCE|REGEXP|all]  # Show computed diagnostic configuration
  kitchen driver                          # Driver subcommands
  kitchen driver create [NAME]            # Create a new Kitchen Driver gem project
  kitchen driver discover                 # Discover Test Kitchen drivers published on RubyGems
  kitchen driver help [COMMAND]           # Describe subcommands or one specific subcommand
  kitchen help [COMMAND]                  # Describe available commands or one specific command
  kitchen init                            # Adds some configuration to your cookbook so Kitchen can rock
  kitchen list [INSTANCE|REGEXP|all]      # Lists one or more instances
  kitchen login INSTANCE|REGEXP           # Log in to one instance
  kitchen setup [INSTANCE|REGEXP|all]     # Setup one or more instances
  kitchen test [INSTANCE|REGEXP|all]      # Test one or more instances
  kitchen verify [INSTANCE|REGEXP|all]    # Verify one or more instances
  kitchen version                         # Print Kitchen's version information

Better descriptions could help; but a short help / long help maybe the quickest way to make it easier without removing any options.

[dgarstang]

I'm good with that if the long help tells me what the difference is between 'kitchen converge' and 'kitchen setup' and 'kitchen verify' and so on.

[damm]

The long help isn't very helpful.

kitchen setup is a shorthand to:

1. Kitchen create
2. Kitchen converge

Which is exactly the same thing that kitchen converge does. The original point I was trying to make is the help is overly lengthy with redundant options that are not self explanatory.

Even if there was no redundancy it's a big vague.

[fnichol]

I'd much prefer an issue in https://github.com/test-kitchen/kitchen-docs to address these questions, please. While reference docs are still in the process of being written, I'm sure you can imagine it's a lot of work to maintain an OSS project when it's not your primary day job.

@dgarstang would a page explaining each of the kitchen actions (i.e. create, converge, setup, verify, destroy, and test) help to address this?

[dgarstang]

fnichol, I think it would. It would need to detail the specifics of what it does. I.e., when is the run_list from the platform used? When is the run_list from the test suite used? Which actions are combinations of other actions, and so forth.

[rayrod2030]

I have been and will always be a HUGE fan of this project and all the other stuff that @fnichol and @sethvargo have written for the chef community. It has literally changed my life as an infrastructure/automation engineer every single day. That being said I have to agree with @MarkGibbons and others that the documentation could use a bit of clarity in regards to the .kitchen.yml file and it's sections. I think the kitchen commands themselves are not terribly difficult to understand but some could use a bit of clarity as well.

In the end we all want these projects to thrive so I think everybody that has suggested ideas in regards to the docs in this issue should take it upon themselves to help contribute better documentation via pull requests.

[sethvargo]

I don't think anyone is saying "no, we don't want documentation". But as an author of such a tool, it is very easy to make assumptions about your user's level of understanding. That's why having a list of things that are unclear will make it more apparent where we need to focus documentation efforts.

[damm]

That's why having your Users contribute back what they expect and deal with; helps the developer understand how their tools are used (better). Without that you have a deaf ear trying to listen to something you are not hearing.

Contribute Back

Giving feedback (creating an issue) and or helping the changes (a pull request)

[dgarstang]

@damn, if your suggesting that I, as a user contribute back to the documentation, I could do that. However, since I have no idea whatsoever what, say 'kitchen setup' does (because there's no reference besides the source), my contribution would be something like 'kitchen setup - does something but I don't know what'.

[damm]

@dgarstang I'm suggesting you've done a great deal by just creating the issue and making @fnichol aware of your feelings.

Now if there's anyway you can help improve the documentation it is more than welcome but I don't feel that is required.

[MarkGibbons]

I'll have a pull request that covers the issues I brought up within the week. I'm pushing it past a mix of beginners and experienced users for feedback. I'll have a working draft on github soon.

Mark

[MarkGibbons]

Pull request #450 covers the problems I had with test-kitchen documentation.

Mark

[sethvargo]

Fixed in #450. Thanks @MarkGibbons!