🛠️ Refactoring for testability & adding test coverage

[AdamWagner]

See

• https://github.com/Olivine-Labs/busted
• https://github.com/dbalatero/VimMode.spoon/blob/master/bin/dev-setup
• https://github.com/dbalatero/VimMode.spoon/blob/master/spec/buffer_spec.lua

[AdamWagner]

This is going to be more involved than expected, but this is a good thing (I think).

Challenge #1: Hammerspoon is special

Hammerspoon does not execute in the default lua environment that you can access by running lua on the command line.

The Hammerspoon modules themselves can only be loaded from within Hammerspoon because they rely on the common macOS Frameworks (Foundation, CoreGraphics, etc.) being linked against the application, as well as the Hammerspoon specific framework, LuaSkin. The only lua process which can successfully meet those requirements is the lua process embedded within Hammerspoon itself.
– asmagill src

This means that leading test runners like busted aren't going to work as expected. I think there are only only two ways to deal with this:

1. Add test method to the main stackline module so that tests can be run from the Hammerspoon console
2. Mock out all Hammerspoon methods used by stackline

I'm currently leaning toward no. 1 because no. 2 is a ton of work, and I'd like to leverage some of the more generic Hammerspoon methods in tests (e.g., using hs.geometry to create realistic frame objects from test data).

Challenge #2: Stackline's design violates tons of best practices for testability

I think Stackline's structure is generally sensible (that was the goal, anyway), but it's becoming increasingly clear that it's not very testable.

• Heavy use of globals in constructors make it difficult to control object creation when testing
• More generally, reliance on access to globals (hs, u, and even parts of stackline itself) makes it hard for tests to pass in fakes
• Constructors that "do work" instead of simply assigning dependencies make it difficult to isolate business logic

Much of this critique was inspired by this 11 year-old talk: Design Tech Talk Series Presents: OO Design for Testability. All of the talks in this series can be found at Google's Testing Blog.

---

Much of the development time for the last couple of features (fuzzy frame detection and improved configuration) was spent chasing down collateral damage many levels removed from the actual change. Solving this puzzle & establishing a baseline of test coverage is now my top priority (before new features like displaying window titles and even before trying to solve irritating bugs such as swallowing all click events).

Assuming that a pretty major refactor will be required to make testing possible presents a chicken-egg problem: I don't want to make changes without unit tests b/c it's hard to predict all the points of failure, and I need to make changes in order to add tests. The goal of this refactor is to make future features & changes easy-to-implement (with confidence that the change didn't break something else).

[AdamWagner]

Here's a long overdue update on testing:

Re: Challenge #1: Hammerspoon is special

I found something really interesting that unblocked this challenge: a handful of Hammerspoon modules can be required from a vanilla lua environment! Plus, the modules that work in a vanilla lua environment are the same modules that made mocking untenable (IMO):

1. hs.geometry
2. hs.inspect
3. hs.fnutils
4. hs.watchable

In particular, the ability to require hs.geometry from a vanilla lua environment is the turning point that lead me to mock the rest of the Hammerspoon modules that stackline relies on. These 4 Hammerspoon modules don't rely on C extensions that only load within Hammerspoon's lua env.

Check out the Hammerspoon mocks here https://github.com/AdamWagner/stackline/tree/tests/basic/spec/hammerMocks and feel free to use in personal projects. Also, I've been thinking about breaking this out into its own repo, but haven't done this yet to avoid dealing with git submodules. If anyone is interested in using hammerMocks separately, just let me know and I'll bite the bullet.

Re: Challenge #2: Stackline's design violates tons of best practices for testability

The ability to mock out Hammerspoon modules enabled the use of busted as a test runner, which largely let me sidestep this challenge (for now). Running each test in an isolated environment and using fixtures to hydrate the hammerspoon mocks make testing possible without needing to pass in fake object constructors. So there's no major refactor to dependency injection in this pass.

---

Here's a visual and a coverage report:

Coverage (2020-11-28)

File                                                                 Hits Missed Coverage
-----------------------------------------------------------------------------------------
/Users/adamwagner/.hammerspoon/stackline/stackline/configManager.lua 147  38     79.46%
/Users/adamwagner/.hammerspoon/stackline/stackline/indicator.lua     124  34     78.48%
/Users/adamwagner/.hammerspoon/stackline/stackline/query.lua         53   4      92.98%
/Users/adamwagner/.hammerspoon/stackline/stackline/stack.lua         27   9      75.00%
/Users/adamwagner/.hammerspoon/stackline/stackline/stackline.lua     49   22     69.01%
/Users/adamwagner/.hammerspoon/stackline/stackline/stackmanager.lua  72   4      94.74%
/Users/adamwagner/.hammerspoon/stackline/stackline/window.lua        62   3      95.38%
-----------------------------------------------------------------------------------------
Total                                                                534  114    82.41%