Make integration tests working on Windows

[tsedmik]

If I run our Integration tests on Windows, they fail. It looks like some test steps are missing (not implemented for Windows) - for example crc.exe start

This is the output of integration tests run:

make: Entering directory '/cygdrive/c/Jenkins/workspace/crc_pipeline/crc'

powershell.exe : go: downloading github.com/DATA-DOG/godog v0.7.13

At C:\Jenkins\workspace\crc_pipeline@tmp\durable-d39b8f6c\powershellWrapper.ps1:3 char:1

+ & powershell -NoProfile -NonInteractive -ExecutionPolicy Bypass -Comm ...

+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    + CategoryInfo          : NotSpecified: (go: downloading...G/godog v0.7.13:String) [], RemoteException

    + FullyQualifiedErrorId : NativeCommandError

 

go: downloading github.com/code-ready/clicumber v0.0.0-20190503113956-2563aed4ef12


Log successfully started, logging into: C:\Jenkins\workspace\crc_pipeline\crc\test\integration\out\test-results/integration_2019-8-9_08-49-56.log

Running integration test in: C:\Jenkins\workspace\crc_pipeline\crc\test\integration\out\test-run

Working directory set to: C:\Jenkins\workspace\crc_pipeline\crc\test\integration\out\test-run

Deleted CRC home folder C:\Users\CDKQE\.crc.

The powershell instance has been started and will be used for testing.

Obtaining bundle...

Copying bundle from bundle.crcbundle to C:\Jenkins\workspace\crc_pipeline\crc\test\integration\out\test-run.

Using bundle: 

Feature: Basic test

  Checks whether CRC top-level commands behave correctly.


  Scenario: CRC version                   # features\basic.feature:5

    When executing "crc version" succeeds # shell.go:234 -> github.com/code-ready/clicumber/testsuite.ExecuteCommandSucceedsOrFails

    Then stderr should be empty           # shell.go:300 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldBeEmpty

    And stdout should contain "version:"  # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain


  Scenario: CRC help                                # features\basic.feature:11

    When executing "crc --help" succeeds            # shell.go:234 -> github.com/code-ready/clicumber/testsuite.ExecuteCommandSucceedsOrFails

    Then stdout should contain "Usage:"             # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain

    And stdout should contain "Available Commands:" # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain

    And stdout should contain "Flags:"              # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain

    And stdout should contain                       # shell.go:288 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContainContent

      """

      Use "crc [command] --help" for more information about a command.

      """


  Scenario: CRC status                # features\basic.feature:22

    When executing "crc status" fails # shell.go:234 -> github.com/code-ready/clicumber/testsuite.ExecuteCommandSucceedsOrFails

    Then stderr should contain        # shell.go:288 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContainContent

      """

      Machine \"crc\" does not exist. Use \"crc start\" to add a new one.

      """


  Scenario: CRC status check                                                                                       # features\basic.feature:62

    When with up to "10" retries with wait period of "1m" command "crc status" output should not contain "Stopped"

    And stdout should contain "Running"                                                                            # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain


  Scenario: CRC IP check                          # features\basic.feature:67

    When executing "crc ip" succeeds              # shell.go:234 -> github.com/code-ready/clicumber/testsuite.ExecuteCommandSucceedsOrFails

    command 'crc ip', expected to succeed, exited with exit code: 1

    Then stdout should match "\d+\.\d+\.\d+\.\d+" # shell.go:324 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldMatch


  Scenario: CRC forcible stop                                          # features\basic.feature:72

    When executing "crc stop -f"                                       # shell.go:207 -> github.com/code-ready/clicumber/testsuite.ExecuteCommand

    Then stdout should contain "CodeReady Containers instance stopped" # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain

    output did not match. Expected: 'CodeReady Containers instance stopped', Actual: ''


  Scenario: CRC status check                                                                                      # features\basic.feature:77

    When with up to "2" retries with wait period of "1m" command "crc status" output should not contain "Running"

    And stdout should contain "Stopped"                                                                           # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain


  Scenario: CRC delete                                                 # features\basic.feature:82

    When executing "crc delete" succeeds                               # shell.go:234 -> github.com/code-ready/clicumber/testsuite.ExecuteCommandSucceedsOrFails

    command 'crc delete', expected to succeed, exited with exit code: 1

    Then stdout should contain "CodeReady Containers instance deleted" # shell.go:284 -> github.com/code-ready/clicumber/testsuite.CommandReturnShouldContain


Feature: Test configuration settings

  Checks whether CRC `config set` command works as expected in conjunction with `crc setup` and `crc start`.

Deleted CRC instance (if one existed).


--- Failed steps:


  Scenario: CRC IP check # features\basic.feature:67

    When executing "crc ip" succeeds # features\basic.feature:68

      Error: command 'crc ip', expected to succeed, exited with exit code: 1


  Scenario: CRC forcible stop # features\basic.feature:72

    Then stdout should contain "CodeReady Containers instance stopped" # features\basic.feature:74

      Error: output did not match. Expected: 'CodeReady Containers instance stopped', Actual: ''


  Scenario: CRC delete # features\basic.feature:82

    When executing "crc delete" succeeds # features\basic.feature:83

      Error: command 'crc delete', expected to succeed, exited with exit code: 1



8 scenarios (3 passed, 3 failed, 2 undefined)

20 steps (11 passed, 3 failed, 2 undefined, 4 skipped)

1.6200794s


You can implement step definitions for undefined steps with these snippets:


func withUpToRetriesWithWaitPeriodOfCommandOutputShouldNotContain(arg1, arg2, arg3, arg4 string) error {

	return godog.ErrPending

}


func FeatureContext(s *godog.Suite) {

	s.Step(`^with up to "([^"]*)" retries with wait period of "([^"]*)" command "([^"]*)" output should not contain "([^"]*)"$`, withUpToRetriesWithWaitPeriodOfCommandOutputShouldNotContain)

}


FAIL	github.com/code-ready/crc/test/integration	2.254s

make: *** [Makefile:119: integration] Error 1


make: Leaving directory '/cygdrive/c/Jenkins/workspace/crc_pipeline/crc'

script returned exit code 1

[tsedmik]

PR was merged into master branch