Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide an API for custom expectations with diagnostics #11

Open
allevato opened this issue Sep 22, 2023 · 1 comment
Open

Provide an API for custom expectations with diagnostics #11

allevato opened this issue Sep 22, 2023 · 1 comment
Labels
enhancement New feature or request public-api Affects public API

Comments

@allevato
Copy link

Description

Spawned from the discussion in https://forums.swift.org/t/brainstorming-customizing-matchers/67456/11:

When a custom comparison is needed, it was recommended that users could just write whatever comparison they need as a Bool-returning function and pass that into #expect. That's fine from the point of view of keeping the #expect API simple, but expectations will often want to provide better diagnostics than what can be extracted from the arguments.

For example, pretend we didn't have isSuperset(of:) and we wrote our own terrible predicate:

extension Collection where Element: Hashable {
  func isThisASuperset(of smaller: some Collection<Element>) -> Bool {
    let diff = Set(smaller).subtracting(Set(self))
    return diff.isEmpty
  }
}

@Test func foo() {
  let x = [1, 3]
  let y = [1, 2, 4, 5]
  #expect(x.isThisASuperset(of: y))
}

When running this test, the runner produces the following diagnostic:

Expectation failed: (x → [1, 3]).isItASuperset(of: y → [1, 2, 4, 5])

which is helpful! But what I'd really like is something like:

Expectation failed: argument contained [2, 4, 5] which were not in the receiver -- (x → [1, 3]).isItASuperset(of: y → [1, 2, 4, 5])

One way to achieve this would be to create an #expect overload that takes, say, an ExpectationResult instead. That could look like the following:

enum ExpectationResult {
  case success
  case failure(reason: String)
}

and then the predicate becomes:

extension Collection where Element: Hashable {
  func isThisASuperset(of smaller: some Collection<Element>) -> ExpectationResult {
    let diff = Set(smaller).subtracting(Set(self))
    return diff.isEmpty
      ? .success
      : .failure(reason: "argument contained \(diff) which were not in the receiver")
  }
}

This would be a bare minimum API for these kinds of diagnostics, and simple enough for most purposes. It wouldn't be advanced enough to factor in the named of the arguments though; if we wanted the output to be something like this, we'd need something more complex:

Expectation failed: 'y' contained [2, 4, 5] which were not in 'x' -- (x → [1, 3]).isItASuperset(of: y → [1, 2, 4, 5])

Expected behavior

No response

Actual behavior

No response

Steps to reproduce

No response

swift-testing version/commit hash

0.0.0-initial

Swift & OS version (output of swift --version && uname -a)

No response

@allevato allevato added the enhancement New feature or request label Sep 22, 2023
@grynspan
Copy link
Contributor

Tracked internally with rdar://106832903. Using this OSS issue to track going forward.

@grynspan grynspan added the public-api Affects public API label Feb 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request public-api Affects public API
Projects
None yet
Development

No branches or pull requests

2 participants