Idempotence check?

[madAndroid]

Hi,

Is there a way to check whether a test run is idempotent within the same test run? Beaker has this functionality, and I've been wondering if there's anything similar in test-kitchen/kitchen-puppet.

I've attempted to use detailed_exit_codes, and this appears to cause the run to fail due to a non-zero exit code; would it be worth including a change that sets accepted_exit_codes to be used alongside this setting? This would also open us up to checking that exit code to see whether a subsequent run made any changes, and check for idempotence using that approach.

Please let me know if this is an acceptable change, and I will look at sending a pull-request when I've got something that's usable.

[neillturner]

Well in my ansible plugin i have a parameter idempotency_test set to false
https://github.com/neillturner/kitchen-ansible/blob/master/provisioner_options.md.
this basically runs the test twice so we can implement similar here in the puppet plugin.
Feel free to create a pull request. I could implement in a few weeks but currently trying to test and release existing changes in several of my plugins.

[madAndroid]

Cheers for the pointers - I'll take a look, and see if I can come up with something usable, and send a PR once it's working as expected.

[ytsarev]

@madAndroid we have puppet_whitelist_exit_code already implemented.
Isn't it exactly what you want ?
As in https://github.com/neillturner/kitchen-puppet/blob/master/provisioner_options.md

Whitelist exit code expected from puppet run. Intended to be used together with puppet_detailed_exitcodes.

[madAndroid]

Ah yes, thanks I hadn't spotted that, it will surely help when creating the idempotence test... but we'll still need to add code that executes a second test run and checks whether any changes have been made?

[nicolasvan]

@madAndroid this is already supported by test-kitchen (1.10+) test-kitchen/test-kitchen#1055 (comment)

[neillturner]

thanks for spotting test-kitchen/test-kitchen#1055 (comment) i should add these options to the docs and explain them.

[madAndroid]

That's awesome - I'm very new to Test-Kitchen, and wasn't aware of that functionality - I'll try those settings, hopefully it will do exactly what we're looking for.

[neillturner]

closing this as i believe it is base test-kitchen functionality now.

[madAndroid]

Thanks! ☺️

On 17 Nov 2016 6:29 pm, "Neill Turner" notifications@github.com wrote:

closing this as i believe it is base test-kitchen functionality now.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#141 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABQp3Tp6jVLDS29KcD9gzcBwhJwS68NBks5q_IDrgaJpZM4KeNtn
.