@Shared State in Tests for consecutive actions is modified prematurely

[Badlazzor]

Description

Consider having a Reducer that has 4 actions:

enum Action {
    case fetchA
    case resultForA
    case fetchB
    case resultForB
}

It also shares the As and Bs in a @Shared [A: IdentifiedArray<B>] State.

The Reducer should do the following:

• gets a fetchA
• calls a networking client async to actually do the work in a .run, sending resultForA when done
• if the results are successful for A sends a fetchB, but before that updates the @Shared state with the As adding empty arrays for each entry (for display reasons)
• calls a networking client async to actually do the work in a .run, sending resultForB when done
• if the results are successful overrides the @Shared state's Bs

The problem I see is when testing and sending fetchA to the TestStore the actions are received as expected, but the state changes are accumulated into the first actions changes in the @Shared state.

Checklist

• I have determined whether this bug is also reproducible in a vanilla SwiftUI project.
• If possible, I've reproduced the issue using the main branch of this package.
• This issue hasn't been addressed in an existing GitHub issue or discussion.

Expected behavior

Sending fetchA to the TestStore would result in the following:

store.receive(.resultForA) {
// only changes for As in the Shared State
}

store.receive(.fetchB)
store.receive(.resultsForB) {
// only changes for Bs in the Shared State
}

Actual behavior

Sending fetchA to the TestStore would result in the following:

store.receive(.resultForA) {
// changes (albeit correct overall) to both As and Bs under them
}

store.receive(.fetchB)
store.receive(.resultsForB) {
// no state change
}

Steps to reproduce

No response

The Composable Architecture version information

1.13.1

Destination operating system

iOS 17.5

Xcode version information

Version 15.4 (15F31d)

Swift Compiler version information

Apple Swift version 5.10 (swiftlang-5.10.0.13 clang-1500.3.9.4)
Target: arm64-apple-darwin23.3.0

[mbrandonw]

Hi @Badlazzor, the behavior you are witnessing is unfortunately expected, though not necessary ideal. I believe this behavior is documented, but I couldn't find it and so I will have to look into that.

But currently the TestStore eagerly executes effect actions when they are received, which means the logic that mutates shared state is happening before one even does store.receive. Before releasing the shared state tools we looked into making TestStore delay executing an effect action until one actually does store.receive, but unfortunately we found that would be a breaking change. There are tests out there that assert on dependencies in a specific way that would be broken by delaying the execution of the effect action.

So, we do plan on fixing this issue, but it will have to wait for 2.0. Since we do not consider this an issue with the library today I am going to convert it to a discussion. Please feel free to continue the conversation over there.

[Badlazzor]

Thanks for the explanation @mbrandonw . Do you have any suggestions on how to test chained events and shared state?
My best idea currently is to have exhaustivity turned off, test for the chain of events and at the last one test for the overall desired state.

[Badlazzor]

Also, I was diligently reading through the shared state docs before starting to write the tests and this possible problem did not occur to me at all before encountering it. As shared state seems like a pretty big win and pull into the library (especially with the persistence), I think adding some examples on testing these scenarios would be great idea.

[mihaho]

To mitigate this issue, I created a new debug configuration named "Testing", set it as the configuration for tests in the scheme, and added a TESTING compiler flag under "Other Swift Flags" for this configuration. Then, in the code, I implemented the following:

#if TESTING
var someProperty: <SOME_TYPE>
#else
@Shared var someProperty: <SOME_TYPE>
#endif

init() {
  #if TESTING
  if NSClassFromString("XCTestCase") == nil { fatalError("Allowed only in tests") } // to be on the safe side
  someProperty = .init(uniqueElements: ...)
  #else
  _someProperty = Shared(wrappedValue: ..., <SOME_PERSISTENCE_KEY>)
  #endif
}

For tests, I use a plain type (<SOME_TYPE>) to test exactly what happens for each action. During normal app execution, it uses the @Shared property wrapper.

This solution is not perfect, but I wanted to share it in case someone finds it helpful for asserting state changes for specific actions. Just a heads-up: if you're working with multiple targets, you'll need to add this configuration and set the flag for each target individually.

[arkulpaKonstantin]

I have run into this issue today and sadly just now found this thread and the related one here 3589.

For our case we could add a TestClock to the test and then use try await testClock.sleep(for: .nanoseconds(1)) in the dependencies. Then between every store.receive we just advance the test clock by the expected amount. This way forces the TestStore to evaluate the effects on after the other.

Would it be possible to have a Exhaustivity option - or similar flag - such that the TestStore does wait on execution until store.receive is called?
This would not break the current tests with the option to opt into before 2.0 arrives.

[mbrandonw]

Hi @arkulpaKonstantin, we are reluctant to do this because of how much work it is to maintain two different modes of TestStore for a temporary amount of time. While it's not optimal to lose out on this bit of exhaustivity in testing, you still do get some test coverage, and we think this may just have to be how things are until 2.0.