Inconsistent behaviour with ENTRYPOINT

[goffinf]

Not sure if this utility (which is excellent BTW) is capable of working with images/containers which have a defined ENTRYPOINT, or whether there's is anything additional to define if the image being used has one ?

I have an image which executes Terraform and uses an ENTRYPOINT script. I created some simple command tests for other binaries that are present within the image and they ran successfully:

    # check the terraform binary is the correct version
  - name: "terraform core"
    command: "terraform"
    args: ["-v"]
    expectedOutput: ["v0.10.8"]

    # check the rancher-cli binary is the correct version
  - name: "rancher-cli"
    command: "rancher"
    args: ["-v"]
    expectedOutput: ["v0.6.7"]

    # check the docker client is the correct version
  - name: "docker client"
    command: "docker"
    args: ["-v"]
    expectedOutput: ["17.09.1-ce"]

however, if I add something like checking that the HTTP_PROXY env var is set to what I expect

    # check the HTTP_PROXY value
  - name: "HTTP_PROXY"
    command: "echo"
    args: ["$HTTP_PROXY"]
    expectedOutput: ["http://xxx-yyy.zzz.com:80"]

That test FAILS ... it is trying to run Terraform not the test:

NOTE: I did notice the metadataTests section in the docs but I couldn't get these to run (raised a separate issue for that), so treat the above as a simple example of any command I might want to run 'echo' in this case

$ structure-test -test.v -image 300820918606.dkr.ecr.eu-west-1.amazonaws.com/digital-web-mobile/terraform-aws:0.10.8 cmd_tests.yaml
Using driver docker
=== RUN   TestAll
2018/01/21 19:47:07 Running tests for file cmd_tests.yaml
=== RUN   TestAll/Command_Test:_terraform_core
=== RUN   TestAll/Command_Test:_rancher-cli
=== RUN   TestAll/Command_Test:_docker_client
=== RUN   TestAll/Command_Test:_HTTP_PROXY
--- FAIL: TestAll (1.54s)
    --- PASS: TestAll/Command_Test:_terraform_core (0.50s)
        docker_driver.go:72: stdout: Terraform v0.10.8

                Terraform v0.10.8

    --- PASS: TestAll/Command_Test:_rancher-cli (0.37s)
        docker_driver.go:72: stdout: Terraform v0.10.8

                rancher version v0.6.7
    --- PASS: TestAll/Command_Test:_docker_client (0.38s)
        docker_driver.go:72: stdout: Terraform v0.10.8

                Docker version 17.09.1-ce, build 19e2cf6
    --- FAIL: TestAll/Command_Test:_HTTP_PROXY (0.28s)
        docker_driver.go:72: stdout: Terraform initialized in an empty directory!

                The directory has no Terraform configuration files. You may begin working
                with Terraform immediately by creating Terraform configuration files.
                $HTTP_PROXY
        utils.go:29: Expected string 'http://xxx-yyy.zzzs.com:80' not found in output!
        structure_test.go:46: Total tests run: 4
FAIL

Is it possible to run such a test when the image has an ENTRYPOINT (the tests before all worked happily enough) ?

Kind Regards

Fraser.

[nkubala]

Hey @goffinf, thanks for reporting this one. Looks like there are a number of issues here.

First off, the issue with the entrypoint: as it stands right now, running command tests in a container with an entrypoint will always run the entrypoint BEFORE running the command the user is trying to test. I don't think we want to do this, because if we're testing a command in a container we want that to be the only thing that's run. I'll fix this soon.

I know that sometimes entrypoint scripts are used as "setup" in a container before it's ever actually run; in this case, I'd say just call your entrypoint script directly in the setup field of each command test. You'll probably want to do that for some of your tests.

The second issue I see here is that the regexes you're using the match the output for your first three commands are matching the outputs as expected, but they're also matching the prefixed Terraform v0.10.8 that's getting printed as a result of the entrypoint being run. Since these are regexes technically you'd probably want to do something like
expectedOutput: ["$Terraform v0.10.8\n"]
to be explicit. I'll look into making this UX a little more forgiving, but for now I'd change your regexes to match this.

The last issue is that your first 3 commands seemed to run the entrypoint script correctly (they all output Terraform v0.10.8 before running the specified command), but your HTTP_PROXY test seems to fail the entrypoint script? Any idea why this is happening?

Printing out shell variables using $ won't work because substitutions aren't supported currently, but you can use the metadataTest to test environment variables. See my comment on the other issue you opened for that.

[goffinf]

Hey Nick,

thanks for the feedback.

Do you know I saw the stdout: Terraform v0.10.8 for each test and I didn't twig it was running a terraform command that would return the version (apart from the first one where I ask it to do that in the test itself). Actually the entrypoint command (script) runs terraform init with a few options which interesting shouldn't support -v but somehow, because -v is included in the other commands I am running for the first 3 tests, seems to pick that up and emits the terraform version output (wierd). If I change -v in say the second test to -w an error occurs because the entrypoint is using that instead and its invalid. Anyway, if at some future point you intend to update the code to skip the entrypoint or otherwise ensure that only the command in the test is run, that will sort that out.

This is a slightly unusual use case in the sense that the container I am testing is a build tool (terraform) and as such is intended to simply run the terraform command (plan, apply destroy, whatever) and then exit. Perhaps I should have chosen a more main-stream example, but nonetheless it was what I was doing at the time so ... why not.

Anyway, I really like what you have done here and will be showing it to the team later this week. I anticipate the 'groan' when I tell them I am looking for them to follow TDD for containers from now on :-)