feat(runner): enable the bail setting

[ihexxa]

Background

By enabling "stop on error" setting, runner stops immediately when it sees an error.

Changes

• Enable the "bail" setting
• Minor style fixes

Ref: INS-4403

[jackkav]

in the CLI and other testing frameworks this is referred to as "bail"

also please avoid abbreviation like "curIteration" where possible.

[ihexxa]

Let me update that part although I just copied them. 🥹

[filfreire]

@ihexxa I've tried

• setting an assertion of a request to fail
• causing a reference error (e.g. non existing variable)
• or adding a non existing environment variable / template tag to the url path

in none of these cases did the bail seem to work, still executing the request that came afterwards.

Is there any other type of errors that are actually meant to make the option trigger?

[ihexxa]

Thanks @filfreire let me check, assertion failure and non-exist variables might not be regarded as "error"s in runner. You could try just throw an error although it is a bit wild.

[notjaywu]

@ihexxa I've tried

• setting an assertion of a request to fail
• causing a reference error (e.g. non existing variable)
• or adding a non existing environment variable / template tag to the url path

in none of these cases did the bail seem to work, still executing the request that came afterwards.

Is there any other type of errors that are actually meant to make the option trigger?

From my point of view, setting an assertion of a request to fail is more like a feature request, some others may want the progress to continue.

[jackkav]

bail or stop-on-error, should exit the runner if at any point a test fails or an unhandled exception occurs, in my mental modal a request is equivalent to a test suite containing 0..n tests. In the cli i need to keep a success flag in the outer loop and return out oif the runner loop early if bail is true. In this PR a throw is probably a more sensible way to bail.

[filfreire]

LGTM

Bail should now work for referenceError thrown - I think this is OK 👍

Assertion and missing Env variable - might not be be a good idea to add now

We found in other clients the behavior is the same.

There is one extra case where an assertion can fail - and what's the behavior of that failure when outside of a test's scope - but it should be ok to leave it out of scope of this PR