discussion: should the framework ship canonical test fakes (huginn.testing)?

[ChristopherJHart]

Currently, each project defines its own hand-crafted fakes for unit testing jobs (FakeDevice, FakeBroker, FakeResults, FakeContext) in tests/jobs/conftest.py. These fakes are structurally identical across projects since they implement the same core framework interfaces.

Question

Should the Huginn framework ship a huginn.testing module with canonical fake implementations, so that projects and plugins don't independently re-implement the same test doubles?

Considerations

Arguments for framework-provided fakes:

• The fakes are generic — they'd look nearly identical in any Huginn project
• Reduces boilerplate when starting a new project
• The framework knows its own interfaces best, so framework-maintained fakes stay in sync as interfaces evolve
• As the framework disaggregates into plugins (broker plugins, inventory plugins), each plugin would need its own fakes — having a common base in the framework prevents divergence

Arguments for project-defined fakes:

• Projects may need different fakes (e.g., a NETCONF-heavy project needs FakeBroker.edit() while an SSH-only project doesn't)
• Fakes are simple enough that copying them isn't burdensome
• Framework-shipped fakes create a maintenance contract — changes to the fakes become breaking changes for downstream projects

Plugin architecture implications

Once components are disaggregated into separate plugins, a natural ownership boundary emerges:

• Core framework owns fakes for core interfaces (FakeContext, FakeResults, FakeDevice)
• Broker plugins (SSH, NETCONF, HTTP) own fakes for their specific broker interfaces
• Inventory plugins own fakes for testbed/inventory-related interfaces
• Projects own the spec-driven harness (support.py) and any project-specific fixtures

The framework could also define a FakeBroker base or protocol that broker plugins extend with their specific methods, providing a common pattern without the framework needing to know about SSH or NETCONF specifics.

References

• docs/08-unit-testing-automation.md — Pattern 3: Hand-Crafted Fakes