Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: pending x:expect should not trigger running SUT #1876

Merged
merged 7 commits into from
Mar 18, 2024

Conversation

galtm
Copy link
Member

@galtm galtm commented Mar 11, 2024

I was doing some experiments with @catch to pinpoint the latest spot where catch="yes" needs to go if you want to avoid a hard error when running a test whose SUT produces an error. It seemed like a bug that a pending x:expect element caused the SUT to run. I expected a pending x:expect element not to influence test behavior at all.

This PR changes the logic for what causes the SUT to run from "this scenario isn't pending and has at least one x:expect child" to "this scenario isn't pending and has at least one x:expect child that also isn't pending."

Example of effect on test-driven development

On the master branch, the first scenario has a hard error, while the second scenario runs to completion.

  <x:scenario label="Function not written yet">
    <x:call function="function-not-written-yet"/>  
    <x:expect label="will someday return 3" select="3"
      pending="not written yet"/>
  </x:scenario>

  <x:scenario label="Function not written yet">
    <x:call function="function-not-written-yet"/>
    <x:pending label="not written yet">
      <x:expect label="will someday return 3" select="3"/>
    </x:pending>
  </x:scenario>

On the branch of this PR, both scenarios run to completion. The first scenario is similar to the catch.xspec addition in this PR.

@galtm galtm added this to the v3.0 milestone Mar 11, 2024
@galtm galtm added the bug label Mar 11, 2024
@galtm galtm requested a review from AirQuick March 11, 2024 18:35
@galtm galtm enabled auto-merge (squash) March 18, 2024 20:34
@galtm galtm merged commit 6a726ab into xspec:master Mar 18, 2024
38 checks passed
@galtm galtm deleted the catch-experiments branch March 18, 2024 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant