-
-
Notifications
You must be signed in to change notification settings - Fork 914
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
POC: Result-builder based DSL #1149
Conversation
let group: ExampleGroup | ||
|
||
if let x = specClass as? ResultBuilderQuickSpec.Type { | ||
group = x.spec().rootGroup | ||
} else { | ||
group = ExampleGroup( | ||
description: "root example group", | ||
flags: [:], | ||
isInternalRootExampleGroup: true | ||
) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new DSL builds its structure up internally, without needing global state like the World
.
Still, the World
is where the examples are ultimately queried from, so I just patched this method for now.
This would be helpful with the async stuff I'm doing. It might even make sense to to make async support require this and leave the existing API as-is. I like doing this. I would also like to reduce some boilerplate/extra indentation if possible. I don't really have good ideas there, unfortunately. |
Me too! The outter-most My biggest concern here is compile-time and run-time performance. I don't know how expensive the result builder mechanism is to desugar. Would need some measurement. Follow-up thought: We can remove all the class-related boiler plate, and just have top-level code like: Spec {
describe("foo") {
it("foos") { ... }
}
} And have the framework automatically generate a class in the background, not unlike the way it uses |
This is my main concern with removing too much of the boilerplate. While it would be awesome to remove the boilerplate associated with having to make
Well, you're further ahead than me! I haven't done any research on what we can do with ResultBuilders. I think this approach of removing that outer-most |
I spent some time this weekend working with this. I can see eventually deprecating the current way of building tests in favor of using ResultBuilder. Especially once we remove the integration with An incredibly disappointing issue that I ran into is that statements with variable assignment aren't picked up by private var underlyingValue = 0
private func generateValue() -> Int {
underlyingValue += 1
print("Called `generateValue()`, underlyingValue is now \(underlyingValue)")
return underlyingValue
}
// ...
Spec {
describe_builder("testing functions being called more than once") {
let value = generateValue() // this is only called once, when `Spec` is called
it_builder("calls generate value each time") {
expect(value).to(equal(1))
}
it_builder("calls generate value each time please") {
expect(value).to(equal(2))
}
}
} Which is incredibly annoying that the variable assignment is not picked up by I get why this isn't working how we want it to ( I think this is still worth iterating on, especially if we can solve the issue with variable assignment statements. |
Hey Rachel, I saw your mention on #1177 Sorry, I kind of disappeared off the face of the earth. I think I want to get a bit more involved in this kind of open source stuff, so I'll try to at least respond in a timely manner from now on 😅
That does kind of make sense because variable assignments aren't an expression. They're a statement. Although I can't find anything like a I would love to further iterate on a POC for this. However, I would caution against committing to it as a chosen direction, because I think there's a sizable risk that we end up discovering that result builders are too slow to be reasonable for large specs. SwiftUI does use result builders, and Apple does have every incentive to optimize the feature, but I fear that test specs on average are much larger, more complex structures than most UIs (which are encouraged to be refactored and broken down into small, modular sub-views). There's only one way to find out :) |
Closing this, because I think Apple's official Apple has explored using Result Builders to implement a testing DSL, but it looks like Macros are a better fit here.
From @stmontgomery https://forums.swift.org/t/a-new-approach-to-testing-in-swift/67425/26 |
Agreed. I do think that a form of Quick that's built on top of swift-testing has a future, especially once Quick 6 becomes a thing with required concurrency checks. Additionally, the existing syntax has a place just for legacy and obj-c compatibility. |
This pr is a really rough POC that uses Swift's new Result Builders.
I've found that setup code often needs to use fresh objects for each examples, which requires using
beforeEach
. But to access any of the values initialized inbeforeEach
, they need to be declared in the parent scope, and they must be optional, which is a nuance:This new Result Builder based DSL lets you express the same thing using:
There's a trade-off here: now all code inside the spec acts as if it were in a
beforeEach
, meaning it's not currently possible to nest something inside the Spec and only have it called once. Arguably, such things should be lifted to stored properties above thespec()
func, anyway.What do you guys think?
Related: #908