Introduce subscription tests #14472
Merged
Introduce subscription tests #14472
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ilmotta
requested review from
erikseppanen,
cammellos,
rasom,
vkjr,
flexsurfer,
qfrank,
yqrashawn,
Parveshdhull,
briansztamfater,
smohamedjavid,
J-Son89,
jo-mut,
OmarBasem,
ibrkhalil and
siddarthkay
Jenkins BuildsClick to see older builds (12)
|
cammellos
approved these changes
Dec 1, 2022
erikseppanen
reviewed
Dec 1, 2022
There was a problem hiding this comment.
While I'm in agreement (and excited) with most of this, since (and I think for good reason) the clojure community commonly advises data over functions over macros (descending order of preference), I'll do my duty here to question the necessity of the macro:
The reasoning here seems a little thin (I might be wrong):
You could use cljs.test/use-fixtures, but it has a limitation that it only works for top-level tests, not for individual cljs.test/testing sections, so you'd end up having to define multiple deftests repeating long subscription names. Or you'd need to copy & paste setup/teardown code inside testing macros. No fun, and again, easy to miss and end-up with flaky tests (no no no...)
- Is having multiple
deftestsreally such a bad thing? It's just longer test names right? Not saying it's ideal, but if we're weighing it against the overhead of a macro, something to consider... - If we were not able to use
use-fixtures(as you mentioned regarding withtestingsections), would just creating a function for setup / teardown (instead ofuse-fixtures) be enough for us to avoid introducing a macro? (example at the bottom of this page: https://clojuredocs.org/clojure.test/use-fixtures)
|
that's awesome, thank you, would be cool to have description of this PR somewhere in the documentation |
flexsurfer
approved these changes
Dec 2, 2022
Hey @erikseppanen, difficult questions to answer. As I was writing the code I could almost hear you saying: "why macros?" Before trying to give my perspective, I'd like to reiterate some of the problems we're trying to solve.
So my mindset when working in this PR was to make things as easy as possible and not as simple as possible. Which is a prioritization that I often prefer in the context of testing. The easier to write tests, the more tests people will write.
Longer test names are not that big deal fore sure, and having multiple
Yes, a function would be enough. As usual, a macro is almost never needed, especially the ones for syntax sugar. Based in your comments, the following example could be a general solution that doesn't require (deftest communities-section-list-test
(testing "builds sections using the first community name char (uppercased)"
(h/setup-subs
(fn []
(swap! rf-db/app-db assoc :communities
{"0x1" {:name "civilized monkeys"}
"0x2" {:name "Civilized rats"}})
(is (= [{:title "C"
:data [{:name "civilized monkeys"}
{:name "Civilized rats"}]}]
(rf/sub [:communities/section-list]))))))
(testing "sorts by section ascending"
(h/setup-subs
(fn []
(swap! rf-db/app-db assoc :communities
{"0x3" {:name "Memorable"}
"0x1" {:name "Civilized monkeys"}})
(is (= [{:title "C" :data [{:name "Civilized monkeys"}]}
{:title "M" :data [{:name "Memorable"}]}]
(rf/sub [:communities/section-list])))))))I like the solution with and without the macro. By using the function approach, we lose some things in the name of simplicity, in exchange for developers to duplicate certain names and remember how/when to use the I'd be just as happy to merge a solution with only the function if that's what's preferred. The important thing is to help all of us test layer-3 subscriptions in a coherent manner. |
ilmotta
force-pushed
the
chore/test-subs
branch
from
90b445f
to
d6d8678
Compare
Sure @flexsurfer, I could document a little bit in the |
vkjr
approved these changes
Dec 2, 2022
ilmotta
force-pushed
the
chore/test-subs
branch
from
d6d8678
to
f08af89
Compare
yqrashawn
approved these changes
Dec 2, 2022
siddarthkay
approved these changes
Dec 2, 2022
ibrkhalil
approved these changes
Dec 3, 2022
qfrank
approved these changes
Dec 5, 2022
smohamedjavid
approved these changes
Dec 5, 2022
erikseppanen
approved these changes
Dec 6, 2022
ilmotta
force-pushed
the
chore/test-subs
branch
from
12ba6a1
to
b4ad964
Compare
cammellos
pushed a commit
that referenced
this pull request
Dec 12, 2022
ilmotta
added a commit
that referenced
this pull request
Oct 5, 2023
This commit shows how to move away from rf/defn https://github.com/status-im/status-mobile/blob/f12c7401d167d7adb81f97bdf9c0902b39ee37bc/src/utils/re_frame.clj#L1-L90 & rf/merge https://github.com/status-im/status-mobile/blob/f12c7401d167d7adb81f97bdf9c0902b39ee37bc/src/utils/re_frame.cljs#L39-L85 and why we should do it. ## Problems Before jumping to solutions, let's understand the problems first, in no order of importance. ### Problem 1: Cyclic dependencies If you ever tried to move event handlers or the functions used inside them to different files in status-mobile, you probably stumbled in cyclic dependency errors. When an event is registered in re-frame, it is globally available for any other place to dispatch. The dispatch mechanism relies on globally unique keywords, the so called event IDs, e.g. :chat/mute-successfully. This means that event namespaces don't need to require other event namespaces, just like you don't need to require subscription namespaces in views. rf/merge increases the likelihood of cyclic dependencies because they force event namespaces to require each other. Although not as common, this happened a few times with devs in the team and it can be a big hassle to fix if you are unlucky. It is a problem we should not have in the first place (at least not as easily). ### Problem 2: We are not linting thousands of lines of code The linter (clj-kondo) is incapable of understanding the rf/defn macro. In theory, we could teach clj-kondo what the macro produces. I tried this, but gave up after a few tries. This is a big deal, clj-kondo can catch many issues and will continue to catch more as it continue to evolve. It's hard to precisely count how many lines are affected, but `find src/ -type f -name 'events.cljs' -exec wc -l {} +` gives us more than 4k LOC. ### Problem 3: Blocking RN's UI thread for too long Re-frame has a routing mechanism to manage events. When an event is dispatched, it is enqueued and scheduled to run some time later (very soon). This process is asynchronous and is optimized in such a way as to balance responsiveness vs the time to empty the queue. >[...] when processing events, one after the other, do ALL the currently queued >events. Don't stop. Don't yield to the browser. Hog that CPU. > >[...] but if any new events are dispatched during this cycle of processing, >don't do them immediately. Leave them queued. > >-- https://github.com/day8/re-frame/blob/master/src/re_frame/router.cljc#L8-L60 Decisions were made (way back in 2017) to reduce the number of registered re-frame events and, more importantly, to coalesce events into bigger ones with the rf/merge pattern. I tried to find evidence of real problems that were trying to be solved, but my understanding is that decisions were largely based on personal architectural preferences. Fast-forward to 2023, and we are in a situation where we have many heavy events that process a LOT of stuff in one go using rf/merge, thus blocking the UI thread longer than we should. See, for example, [status-im2.contexts.profile.login.events/login-existing-profile](https://github.com/status-im/status-mobile/blob/3082605d1e9897da1b1784588aa22fdc65c84823/src/status_im2/contexts/profile/login/events.cljs#L69), [status-im2.contexts.profile.login.events/get-chats-callback](https://github.com/status-im/status-mobile/blob/3082605d1e9897da1b1784588aa22fdc65c84823/src/status_im2/contexts/profile/login/events.cljs#L98), and many others. The following excerpt was generally used to justify the idea that coalescing events would make the app perform better. > We will reduce the the amount of subscription re-computations, as for each > distinct action, :db effect will be produced and swapped into app-db only once > > -- status-im/swarms#31 (comment) This is in fact incorrect. Re-frame, ever since 2015 (so before the original discussions in 2017) uses a concept of batching to process events, which means subscriptions won't re-run after every dispatched event, and thus components won't re-render either. Re-frame is smarter than that. > groups of events queued up will be handled in a batch, one after the other, > without yielding to the browser (previously re-frame yielded to the browser > before every single event). > > -- https://github.com/day8/re-frame/blob/39adca93673f334dc751ee2d99d340b51a9cc6db/docs/releases/2015.md#050--2015-11-5 Here's a practical example you can try in a shadow-cljs :mobile REPL to see the batching behavior in practice. ```clojure ;; A dummy event that toggles between DEBUG and INFO levels. (re-frame/reg-event-fx :dummy-event (fn [{:keys [db]}] {:db (update-in db [:profile/profile :log-level] (fn [level] (if (= "DEBUG" level) "INFO" "DEBUG")))})) (def timer (js/setInterval #(re-frame/dispatch [:dummy-event]) 50)) ;; 1. In component status-im.ui.screens.advanced-settings.views/advanced-settings, ;; add a print call to see when it's re-rendered by Reagent because the ;; subscription :log-level/current-log-level will be affected by our dummy event. ;; ;; 2. Also add a print call to the subscription :log-level/current-log-level to ;; see that the subscription does NOT re-run on every dispatch. ;; Remember to eval this expression to cancel the timer. (js/clearInterval timer) ``` If you run the above timer with 50ms interval, you'll see a fair amount of batching happening. You can infer that's the case because you'll see way less than 20 print statements per second (so way less than 20 recomputations of the subscription, which is the max theoretical limit). When the interval is reduced even more, to say 10ms (to simulate lots of dispatches in a row), sometimes you don't see a single recomputation in a 5s window because re-frame is too busy processing events. This shows just how critical it is to have event handlers finishing as soon as possible to relinquish control back to the UI thread, otherwise responsiveness is affected. It also shows that too many dispatches in a row can be bad, just as big event handlers would block the batch for too long. You see here that dispatching events in succession does NOT cause needless re-computations. Of course there's an overhead of using re-frame.core/dispatch instead of calling a Clojure function, but the trade-off is clearly documented: the more we process in a single event, the less responsive the app may be because re-frame won't be able to relinquish control back to the UI thread. The total time to process the batch increases, but re-frame can't stop in the middle compared to when different dispatches are used. Thus, I believe this rf/merge pattern is harmful as a default practice in an environment such as ours, where it's desirable end-users feel a snappy RN app. I actually firmly believe we can improve the app's responsiveness by not coalescing events by default. We're also preventing status-mobile from taking the most advantage from future improvements in re-frame's scheduler. I can totally see us experimenting with other algorithms in the scheduler to best fit our needs. We should not blindly reduce the number of events as stated here #2634 (comment). Solution: only coalesce events into one pile when it's strictly desirable to atomically update the app db to avoid inconsistencies, otherwise, let the re-frame scheduler do its job by using fx, not rf/merge. When needed, embrace *eventual app db consistency* as a way to achieve lower UI latency, i.e. write fast and short events, intentionally use :dispatch-later or other timing effects to bend the re-frame's scheduler to your will. There's another argument in favor of using something like rf/merge which I would like to deconstruct. rf/merge gives us a way to reuse computations from different events, which is nice. The thing here is that we don't need rf/merge or re-frame to reuse functions across namespaces. rf/merge complects re-frame with the need to reuse transformations. Instead, the solution is as trivial as it gets, reuse app db "transformers" across events by extracting the logic to data store namespaces (src/status_im/data_store). This solution has the added benefit of not causing cyclic dependency errors. ### Problem 4: Clojure's language server doesn't understand code under rf/defn Nowadays, many (if not most) Clojure devs rely on the Clojure Language Server https://github.com/clojure-lsp/clojure-lsp to be more effective. It is an invaluable tool, but it doesn't work well with the macro rf/defn, and it's a constant source of frustration when working in event namespaces. Renaming symbols inside the macro don't work, finding references, jumping to local bindings, etc. Solution: don't use rf/defn, instead use re-frame's reg-event-fx function and clojure-lsp will understand all the code inside event handlers. ### Problem 5: Unit tests for events need to "test the world" Re-frame's author strongly recommends testing events that contain non-trivial data transformations, and we do have many in status-mobile (note: let's not confuse with integration tests in status_im/integration_test.cljs). That, and non-trivial layer-3 subscriptions should be covered too. The reasoning is that if we have a well developed and tested state layer, many UI bugs can be prevented as the software evolves, since the UI is partially or greatly derived from the global state. See re-frame: What to Test? https://github.com/day8/re-frame/blob/39adca93673f334dc751ee2d99d340b51a9cc6db/docs/Testing.md#what-to-test. See PR Introduce subscription tests #14472, where I share more details about re-frame's testing practices. When we use rf/merge, we make unit testing events a perennial source of frustration because too many responsibilities are aggregated in a single event. Unfortunately, we don't have many devs in the team that attempted to write unit tests for events to confirm my claim, but no worries, let's dive into a real example. In a unit test for an event, we want to test that, given a cofx and args, the event handler returns the expected map of effects with the correct values (usually db transformations). Let's assume we need to test the following event. The code is still using the combo rf/defn & rf/merge. ```clojure (rf/defn accept-notification-success {:events [:activity-center.notifications/accept-success]} [{:keys [db] :as cofx} notification-id {:keys [chats]}] (when-let [notification (get-notification db notification-id)] (rf/merge cofx (chat.events/ensure-chats (map data-store.chats/<-rpc chats)) (notifications-reconcile [(assoc notification :read true :accepted true)])))) ``` As you can see, we're "rf/merging" two other functions, namely ensure-chats and notifications-reconcile. In fact, ensure-chats is not registered in re-frame, but it's 99% defined as if it's one because it needs to be "mergeable" according to the rules of rf/merge. Both of these "events" are quite complicated under the hood and should be unit tested on their own. Now here goes the unit test. Don't worry about the details, except for the expected output. ```clojure (deftest accept-notification-success-test (testing "marks notification as accepted and read, then reconciles" (let [notif-1 {:id "0x1" :type types/private-group-chat} notif-2 {:id "0x2" :type types/private-group-chat} notif-2-accepted (assoc notif-2 :accepted true :read true) cofx {:db {:activity-center {:filter {:type types/no-type :status :all} :notifications [notif-2 notif-1]}}} expected {:db {:activity-center {:filter {:type 0 :status :all} :notifications [notif-2-accepted notif-1]} :chats {} :chats-home-list nil} ;; *** HERE *** :dispatch-n [[:activity-center.notifications/fetch-unread-count] [:activity-center.notifications/fetch-pending-contact-requests]]} actual (events/accept-notification-success cofx (:id notif-2) nil)] (is (= expected actual))))) ``` Notice the map has a :dispatch-n effect and other stuff inside of it that are not the responsibility of the event under test to care about. This happens because rf/merge forces the event handler to compute/call everything in one go. And things get MUCH worse when you want to test an event A that uses rf/merge, but A calls other events B and C that also use rf/merge (e.g. event :profile.login/get-chats-callback). At that point you flip the table in horror 😱, but testing events and maintaining them should be trivial. Solution: Use re-frame's `fx` effect. Here's the improved implementation and its accompanying test. ```clojure (defn accept-notification-success [{:keys [db]} [notification-id {:keys [chats]}]] (when-let [notification (get-notification db notification-id)] (let [new-notifications [(assoc notification :read true :accepted true)]] {:fx [[:dispatch [:chat/ensure-chats (map data-store.chats/<-rpc chats)]] [:dispatch [:activity-center.notifications/reconcile new-notifications]]]}))) (re-frame/reg-event-fx :activity-center.notifications/accept-success accept-notification-success) (deftest accept-notification-success-test (testing "marks notification as accepted and read, then reconciles" (let [notif-1 {:id "0x1" :type types/private-group-chat} notif-2 {:id "0x2" :type types/private-group-chat} notif-2-accepted (assoc notif-2 :accepted true :read true) cofx {:db {:activity-center {:filter {:type types/no-type :status :all} :notifications [notif-2 notif-1]}}} ;; *** HERE *** expected {:fx [[:dispatch [:chat/ensure-chats []]] [:dispatch [:activity-center.notifications/reconcile [notif-2-accepted]]]]} actual (events/accept-notification-success cofx [(:id notif-2) nil])] (is (= expected actual))))) ``` Notice how the test expectation is NOT verifying what other events do (it's actually "impossible" now). Using fx completely decouples events and makes testing them a joy again. ### Problem 6: Unordered effects status-mobile still uses the legacy way to describe the effects map, which has the problem that their order is unpredictable. > Prior to v1.1.0, the answer is: no guarantees were provided about ordering. > Actual order is an implementation detail upon which you should not rely. > > -- https://github.com/day8/re-frame/blob/39adca93673f334dc751ee2d99d340b51a9cc6db/docs/Effects.md#order-of-effects > In fact, with v1.1.0 best practice changed to event handlers should only > return two effects :db and :fx, in which case :db was always done first and > then :fx, and within :fx the ordering is sequential. This new approach is more > about making it easier to compose event handlers from many smaller functions, > but more specificity around ordering was a consequence. > > -- https://github.com/day8/re-frame/blob/39adca93673f334dc751ee2d99d340b51a9cc6db/docs/Effects.md#order-of-effects ### Problem 7: Usage of deprecated effect dispatch-n We have 35 usages, the majority in new code using dispatch-n, which has been officially deprecated in favor of multiple dispatch tuples in fx. See https://github.com/day8/re-frame/blob/39adca93673f334dc751ee2d99d340b51a9cc6db/docs/api-builtin-effects.md#L114 ### Problem 8: Complexity 🧙♂️ Have you ever tried to understand and/or explain how rf/merge and rf/defn work? They have their fare share of complexity and have tripped up many contributors. This is not ideal if we want to create a project where contributors can learn re-frame as quickly as possible. Re-frame is already complicated enough to grasp for many, the added abstractions should be valuable enough to justify. Interestingly, rf/merge is a stateful function, and although this is not a problem in practice, it is partially violating re-frame's spirit of only using pure functions inside event handlers. ### Problem 9: Using a wrapping macro rf/defn instead of global interceptors When rf/defn was created inside status-mobile, re-frame didn't have global interceptors yet (which were introduced 3+ years ago). We no longer have this limitation after we upgraded our old re-frame version in PR #15997. Global interceptors are a simple and functional abstraction to specify functions that should run on every event, for example, for debugging during development, logging, etc. This PR already shows this is possible by removing the wrapping function utils.re-frame/register-handler-fx without causing any breakage. ## Conclusion By embracing re-frame's best practices for describing effects https://github.com/day8/re-frame/blob/39adca93673f334dc751ee2d99d340b51a9cc6db/docs/FAQs/BestPractice.md#use-the-fx-effect, we can solve long standing issues that affect every contributor at different levels and bring the following benefits: - Simplify the codebase. - Bring back the DX we all deserve, i.e. Clojure Language Server and clj-kondo fully working in event namespaces. - Greatly facilitate the testability of events. - Give devs more flexibility to make the app more responsive, because the new default would not coalesce events, which in turn, would block the UI thread for shorter periods of time. At least that's the theory, but exceptions will be found. The actions to achieve those benefits are: - Don't use the macro approach, replace rf/defn with re-frame.core/reg-event-fx. - Don't use rf/merge, simply use re-frame's built-in effect :fx. - Don't call event handlers as normal functions, just as we don't directly call subscription handlers. Use re-frame's built-in effect :fx. ## How do we refactor the remainder of the code? Some numbers first: - There are 228 events defined with rf/defn in src/status-im2/. - There are 34 usages of rf/merge in src/status_im2/. ## Resources - Release notes where fx was introduced in re-frame: https://github.com/day8/re-frame/blob/39adca93673f334dc751ee2d99d340b51a9cc6db/docs/releases/2020.md#110-2020-08-24
ilmotta
added a commit
that referenced
this pull request
Jun 14, 2024
Introduces a new macro deftest-event to facilitate writing tests for event handlers. Motivation came from the _problem of having to always extract event handlers as vars in order to test them_. Although the implementation of deftest-sub and deftest-event are similar, deftest-sub is critically important because it guarantees changes in one subscription can be caught by tests from all other related subscriptions in the graph (reference: PR #14472). This is not the case for the new deftest-event macro. deftest-event is essentially a way of make testing events less ceremonial by not requiring event handlers to be extracted to vars. But there are a few other small benefits: - The macro uses re-frame and "finds" the event handler by computing the interceptor chain (except :do-fx), so in a way, the tests are covering a bit more ground. - Slightly easier way to find event tests in the repo since you can just find references to deftest-event. - Possibly slightly easier to maintain by devs because now event tests and sub tests are written in a similar fashion. - Less code diff. Whether an event has a test or not, there's no var to add/remove. - The dispatch function provided by the macro makes reading the tests easier over time. For example, when we read subscription tests, the Act section of the test is always the same (rf/sub [sub-name]). Similarly for events, the Act section is always (dispatch [event-id arg1 arg2]). - Makes the re-frame code look more idiomatic because it's more common to define handlers as anonymous functions. Downside: deftest-sub and deftest-event are relatively complicated macros. Note: The test suite runs just as fast and clj-kondo can lint code within the macro just as well. Before: ```clojure (deftest process-account-from-signal-test (testing "process account from signal" (let [cofx {:db {:wallet {:accounts {}}}} effects (events/process-account-from-signal cofx [raw-account]) expected-effects {:db {:wallet {:accounts {address account}}} :fx [[:dispatch [:wallet/get-wallet-token-for-account address]] [:dispatch [:wallet/request-new-collectibles-for-account-from-signal address]] [:dispatch [:wallet/check-recent-history-for-account address]]]}] (is (match? expected-effects effects))))) ``` After ```clojure (h/deftest-event :wallet/process-account-from-signal [event-id dispatch] (let [expected-effects {:db {:wallet {:accounts {address account}}} :fx [[:dispatch [:wallet/get-wallet-token-for-account address]] [:dispatch [:wallet/request-new-collectibles-for-account-from-signal address]] [:dispatch [:wallet/check-recent-history-for-account address]]]}] (reset! rf-db/app-db {:wallet {:accounts {}}}) (is (match? expected-effects (dispatch [event-id raw-account]))))) ```
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR brings to every contributor's reach the capability to easily test subscriptions in a way that also make sure re-frame's signal graph is validated. It also adds tests to more complex subscriptions to show how well the proposed solution scales to any subscription.
There's a long rationale about why this PR is doing what it's doing, and I hope the explanations that follow can help us get on the same page.
I'm assuming not everyone is familiar with some of the concepts I mention here, and because of that I prefer to more thoroughly explain them. I'm sorry if sometimes this feels pedantic to you.
Motivation
Just a handful of layer-3 subscriptions have tests, and the ones that do are incomplete.
The team is not aligned on how to design layer-3 subscription tests and the perils lurking in the dark mysteries of re-frame. We can see this in some PR threads.
Make more layers of the architecture testable, so that we can start to actually increase the quality of the app over time. In my opinion, at the speed the mobile codebase is growing/churning and the amount of contributors coming in, we need to aggressively improve our testing story.
Quality assurance is not just trusting end-of-the-line QA processes (e.g. manual tests, PR reviews and e2e tests) and expecting engineers to write great code (whatever that means). Quality should be embedded in everything we do.
Subscriptions (layer-3), events, effects and interceptors should be tested and there's nothing preventing us from doing so. They are the underlying structures that make the app work, and if we don't test them we put too much burden on the last layer (view), which is the most complicated to test.
Why should we care? Software engineers test so that systems can evolve with confidence from one snapshot to another, often in highly collaborative environments. Layer-3 subs are one piece of this puzzle.
Main problem
What's all the fuss about testing subscription handlers? After all, they're pure functions.
If you're not yet familiar with or need a refresh on re-frame's signal graph, I highly recommend first reading https://day8.github.io/re-frame/subscriptions/.
According to reframe's testing doc, only layer-3 subscriptions should be tested. I completely agree with that because testing extractors (layer-2) is almost pointless given their simplicity (there's no real logic). Therefore, this PR is only concerned with layer-3 subs.
Re-frame suggests testing subscriptions by extracting anonymous subscription handlers to named functions. This works, but in practice, testing like that has important limitations for subscriptions because it's easy to make tests pass, when in reality the signal graph could be broken 💥
Example problem
Here's a relatively complicated layer-3 subscription:
If we follow re-frame's recommendation to test this subscription, then we should extract the anonymous function, like so.
At this point, the extracted public function is testable, but notice we lose the ability to actually verify the subscription is receiving the correct inputs.
Let's say requirements change, and
:wallet/currencyreturns a different map structure. Well then, instead of breaking, tests would still pass. Now imagine the entire system's subscriptions are tested like this. In that case, every single change to a subscription would require manually hunting its usages and checking if tests need to be updated, otherwise there's no guarantee a regression in the signal graph was not introduced. This is a lot of work many contributors won't do (including myself).A little bit better, but with issues
The solution I found to be the most valuable from previous experiences is to not ignore the subscription inputs, i.e. subscribe to the real subscription and let re-frame do its work.
So as you can see, now the problem is that the test is mixed up with boring stuff to arrange and clean-up state. You could use
cljs.test/use-fixtures, but it has a limitation that it only works for top-level tests, not for individualcljs.test/testingsections, so you'd end up having to define multipledeftests repeating long subscription names. Or you'd need to copy & paste setup/teardown code insidetestingmacros. No fun, and again, easy to miss and end-up with flaky tests (no no no...)Even better solution
Clojure gives us the power of macros to abstract all that plumbing. So here's the final solution, using
test-helpers/deftest-sub. The actual implementation of the macro is quite interesting, see the For the Curious section for more details.Important things:
testingmacro, and they'll be correctly augmented by thedeftest-submacro to reset the subscription cache and restore the app DB state.For the curious
The
deftest-subMacroIf you'd like to learn more about macros, specifically Deep Code Walking Macros (DCWM), check out this excellent blog post https://blog.fogus.me/2013/07/17/an-introduction-to-deep-code-walking-macros-with-clojure. If, on the other hand, you have no idea how macros work, the Clojure for the Brave and True book section on macros is a very good starting point.
Now, if you try to call the
deftest-submacro with invalid arguments, you get a beatiful error generated by expound. This works because I've defined aclojure.spec.alpha/fdeffor the macro. This is what Clojure core does for many macros, and it greatly improves DX.See, I'm not suggesting using clojure.spec everywhere, but using it for macros with instrumentation enabled is definitely a win.
fdefhas zero impact in production performance with instrumentation disabled, and besides this macro is only for testing, so no impact whatsoever.What about the guidelines?
This is an introductory PR to the subject of testing subscriptions, and it's my hope that PRs from now on will be able to be opened with subscription tests whenever needed. I think we can update the guidelines once more people write subscription tests and we get practical feedback.
More references
status: ready