multiple targets / parallel execution for inspec exec

[kmf]

It would be awesome to provide inspec cli a list of targets .... perhaps similar to a inventory file in ansible.

[clintoncwolfe]

This would likely need to be implemented on a major version change.

Currently, each InSpec run sets up a connection (via train) to exactly one target, performs OS detection, runs the tests, and reports on the outcome.

Conceivably, we could take a list of targets, fork (with new options for concurrency), perform the runs, then combine the outcomes into each requested report format.

I don't know what gotchas lie ahead in that path.

[clintoncwolfe]

As a workaround today, one can at least use shell scripting to run inspec serially over a list of targets. And one could go farther, using Perl or Ruby etc. to add forking.

[mgrobelin]

A simple shell loop will do what you intend, but beware:

• expect ~300 MB mem usage per running InSpec instance, so even when you run a "small" profile on 10 hosts in parallel, you will quickly saturate 3 GB of RAM on the system that executes the tests
• the formatters will not summarize your test results without post-processing as already pointed out by @clintoncwolfe ; therefore I usually temporary save the results as xml (e.g. by --reporter cli junit:/tmp/${h}_${profile}.xml) for each host first and combine them into one combined report later

all_hosts="host1.local host2.local host3.local"
ssh_user="mysshuser"

# sequential
for h in $all_hosts} ; do inspec exec [...] --target ssh://${ssh_user}@${h} ; done

# parallel
for h in $all_hosts ; do inspec exec [...] --target ssh://${ssh_user}@${h} & done

[thbar]

I'm also looking for ways to accelerate the spec runs (either for N targets but also for 1 target). I'm new to this space, maybe my questions will be obvious!

• Is kitchen test worth looking at to use the parallel target support & handle what @mgrobelin describes?
• Are there any discussion to parallelise checks ran on a single target? (inspec exec would create a number of threads to evaluate the resources in parallel?)

Thanks!

[thbar]

Answering my own question on parallelism for a single target: RSpec (which Inspec relies on) doesn't yet support built-in parallelism (see discussion). This means that we may have to jump through hoops to bring some form of parallelism to the inspec runner (for a single target).

[clintoncwolfe]

@thbar - Thanks for that research into threading on RSpec, that opened my eyes quite a bit!

As for single-target performance, we track known performance issues. Some of the newer resources - like the AWS resources - have some known performance issues. If you find anything specific to a particular resource, or use case, we'd love to hear about it in a new issue. Thanks!

[thbar]

There is some very significant discussion going on in another sphere, which could be applied here with good benefits (especially given how slow remote SSH specs can be):

• RSpec incremental summarized output grosser/parallel_tests#708
• https://review.discourse.org/t/dev-add-a-new-way-to-run-specs-in-parallel-with-better-output-7778/4363/2
• https://twitter.com/samsaffron/status/1142048824526958592

[zopanix]

Hey, I don't know if it is better to create a separate issue or maybe another issue already exists. But it would be great to be able to inspec multiple target types with a single profile.

For example, If I deploy an AKS cluster on Azure and validating it with inspec-azure and at the same time being able to validate what is deployed on that AKS cluster using inspec-kubernetes. All inside the same profile that would be great. thx

[tohch4]

So I see this is a longstanding issue, and I have a similar need to @zopanix, but different cloud, super similar use case. @clintoncwolfe, is this something people can pick up or is a feature this complex that requires planning, designing, and feedback from core devs before moving forward?

[clintoncwolfe]

I sketched out one possible approach above. I still think this would be an InSpec 5 feature - as it would be a big change to several major components - but I think if the community wanted to get started on this we would welcome it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. We value your input and contribution. Please leave a comment if this issue still affects you.