Integration tests in Golang

[maximilien]

Allows for:

1. allows assertions on different parts of the command (input and output)
2. allows and more complex workflow with logic between steps
3. should be scalable and easier to understand as tests get complicated

[rhuss]

Sorry, a bit late to the party, but I think running kn for integrations tests is the closest it gets to how it really is used. Not that I'm totally opposed of putting it to go, but just curious what kind of complex workflows do you have in mind ? Some tests where the output of one is the input to the next ?

Shell script is not so hard to understand and very accessible, too. Together with an IT framework like https://github.com/sstephenson/bats it's not that bad.

allows assertions on different parts of the command (input and output)

Which assertions on the input data do you have in mind ?

---

However, Docker does all its CLI testing also within Go, e.g. https://github.com/docker/cli/blob/master/e2e/container/kill_test.go Seems to work there nicely, too.

So, to summarize, I'm undecided ;-)

[maximilien]

I simply think Golang gives the best overall flexibility.

Simple case is dealing with parsing and creating assertions on output. See how I had to use a Regexp in the PR #121 I submitted since the output for listing services includes all kinds of formatting and if you want to assert of service present and state, the resulting bash could be already hairy.

I don't oppose keeping a simple e2e test in bash like we have now for the basic workflow. This could be great to give a warm feeling that the kn client + cluster work. But for complex workflows that pushes the command options, scales services, creates revisions, etc etc. Having them in Golang will go a long way to make resulting integration tests maintainable.

Any other thoughts? Or objections?

[sixolet]

I'm 100% for a couple simple smoke tests in bash and more complex stuff in a real programming language like go.

…

On Jun 11, 2019, at 3:50 PM, dr.max ***@***.***> wrote: I simply think Golang gives the best overall flexibility. Simple case is dealing with parsing and creating assertions on output. See how I had to use a Regexp in the PR #121 I submitted since the output for listing services includes all kinds of formatting and if you want to assert of service present and state, the resulting bash could be already hairy. I don't oppose keeping a simple e2e test in bash like we have now for the basic workflow. This could be great to give a warm feeling that the kn client + cluster work. But for complex workflows that pushes the command options, scales services, creates revisions, etc etc. Having them in Golang will go a long way to make resulting integration tests maintainable. Any other thoughts? Or objections? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[maximilien]

Closing this since #121 is merged. Will add a separate e2e-smoke-tests.sh for the bash code this #121 replaced. Dev can then decide what to run themselves.