Tying into a test suite setup

[findheld-doug]

Hello,

I believe I'm missing a step somewhere. I have the following setup:

version: "1.1"
name: SSL Check
env:
  host: example.com
tests:
  example:
    steps:
      - name: GET request
        http:
          url: https://{{env.host}}
          headers:
            user-agent: custom
          method: GET
          check:
            status: /^20/
            ssl:
              valid: true
              signed: true
              daysUntilExpiration:
                - gte: 31

which executes fine from the command line, but I'm having trouble getting the file to be read by any of the testing suites using the runner.

I have the following in my package.json:

{
  "scripts": {
    "test": "jest"
  },
  "private": "true",
  "name": "infrstructure-tests",
  "version": "1.0.0",
  "description": "A test suite against our infrastructure.",
  "devDependencies": {
    "@stepci/runner": "^1.4.5",
    "jest": "^29.2.1"
  }
}

Just getting the following when I try to run the example:

✗ npm test        

> infrstructure-tests@1.0.0 test
> jest

 FAIL  test/example.com.test.js
  ✕ Check Response Status (1274 ms)

  ● Check Response Status

    expect(received).toBe(expected) // Object.is equality

    Expected: true
    Received: undefined

      4 | test('Check Response Status', async () => {
      5 |   const result = await runFromFile(path.join(__dirname, 'stepci/example.yml'))
    > 6 |   expect(result.passed).toBe(true)
        |                         ^
      7 | })

      at Object.toBe (test/example.com.test.js:6:25)

Test Suites: 1 failed, 1 total
Tests:       1 failed, 1 total
Snapshots:   0 total
Time:        4.561 s, estimated 5 s
Ran all test suites.

[findheld-doug]

oddly enough changing the line:

expect(result.passed).toBe(true)

when changed to:

expect(result.passed == "true")

Causes the tests to pass successfully. This has unblocked me for the moment.

[findheld-doug]

changing the line:

expect(result.passed).toBe(true)
to
expect(result.result.passed).toBe(true)

Solves the problem at least more correctly

[mishushakov]

Hey, I think the docs for testing integration were outdated 😔

The runner returns the following type:
https://github.com/stepci/runner/blob/f4030962c1b3512de286baa48207d6dae30f8bc8/src/index.ts#L59-L68

So your code should be looking like this:

const { result } = await runFromFile(path.join(__dirname, 'stepci/example.yml'))

I'll update the docs, sorry for the inconvenience!

[mishushakov]

Update: I have updated the docs for testing framework integration: https://github.com/stepci/stepci/blob/main/docs/using-test-tools.md

[sebastianwis]

Hey Doug @findheld-doug
Thank you for your contribution here!
It would be amazing to welcome you to our Discord if you haven't joined it yet:
https://discord.com/invite/KqJJzJ3BTu
This is where we keep users up to date with updates, features and bug resolutions.