Repository navigation
What evidence should justify recommending a network mock for a PR? #2775
|
I'm experimenting with a local, deterministic QA tool that reads a PR diff and repository evidence to identify what should be verified before merge. One difficult boundary is deciding when a network change actually justifies a mock or fixture. A changed endpoint, error branch, or response type may be relevant, but generating a handler from that signal alone can easily encode the wrong contract or couple the test to implementation details. Before suggesting an MSW handler, what evidence would you consider sufficient? For example:
And when should a tool explicitly stop and say that the repository does not contain enough evidence? I'm interested in the design principle rather than a framework-specific setup. The goal is to make the recommendation conservative and explainable instead of generating a mock for every network-related diff. |
Replies: 3 comments 2 replies
|
Hi. I always suggest going design-first. Have a spec that is the only source of truth for your network (OpenAPI or whatever you fancy). Reasoning about actual contract changes becomes easy since, well, now you have an actual contract to reason about. Anything else is guesswork that's prone to false positives/negatives. |
|
@kettanaito I’ve been exploring this boundary while working on an open-source tool, and I’m wondering whether QA inference and mock generation should be treated as separate stages. Repository evidence may help identify affected behavior and suggest what needs verification, while an authoritative network spec remains the basis for deriving a mock contract. One case I’m still unsure about is when an OpenAPI document defines the endpoint, status codes, and response schema but provides no concrete error examples. Would it be reasonable for a tool to derive only the handler structure and require scenario-specific payloads to be supplied explicitly? Or would you consider even that too speculative and prefer the tool to stop? |
|
Thanks, that clarifies the boundary I was unsure about. 🙇🏻♂️ I had been treating the absence of explicit error response examples as insufficient evidence. Looking at how This gives me a much clearer way to separate QA scenario selection, contract derivation from the specification, and the concrete mock artifact. I really appreciate you taking the time to explain this. 🚀 |
Hi. I always suggest going design-first. Have a spec that is the only source of truth for your network (OpenAPI or whatever you fancy). Reasoning about actual contract changes becomes easy since, well, now you have an actual contract to reason about. Anything else is guesswork that's prone to false positives/negatives.