Feature request
Description the feature
Introduce a lightweight, local execution history mechanism to perform trend analysis and warn users about performance degradation or sudden changes in response sizes.
Goals of this feature:
- Zero-Dependency History Storage: Establish a lightweight, local, and file-based mechanism (e.g., JSON or SQLite) to persist metadata of past runs without requiring external database setups.
- Performance Regression Detection: Implement a simple comparison algorithm to detect significant increases in latency or response size compared to historical running averages.
- Actionable Developer Alerts: Warn developers directly in the CLI and HTML reports when an endpoint's performance degrades beyond a configurable threshold.
- Strict Storage Bounds: Cap the historical log size to prevent infinite disk space consumption on the user's local machine.
Is your feature request related to a problem?
Yes. Currently, every ScanAPI execution is stateless and completely isolated. There is no historical context. If an endpoint becomes 50% slower after a new deployment, ScanAPI will still pass (as long as it doesn't timeout), and the developer might miss a critical performance regression.
Do you have any suggestions on how to add this feature in scanapi ?
- Storage: Create a
HistoryManager class to handle reading/writing to a local, hidden file (e.g., .scanapi_history.json in the project root or user's cache directory). Ensure the file is capped (e.g., keeping only the last 20 runs) to keep it lightweight.
- Logic: During a run, ScanAPI compares the current average response times and success rates against the moving average of the last 5 or 10 runs stored in the history file.
- User Alerts:
- If an endpoint's response time increases by a certain threshold (e.g., >20%), the CLI can print a warning:
⚠️ Warning: GET /users is 25% slower than its 10-run average (250ms vs 200ms).
- Add a CLI flag to disable this behavior (e.g.,
--no-history) or to clear history.
Feature request
Description the feature
Introduce a lightweight, local execution history mechanism to perform trend analysis and warn users about performance degradation or sudden changes in response sizes.
Goals of this feature:
Is your feature request related to a problem?
Yes. Currently, every ScanAPI execution is stateless and completely isolated. There is no historical context. If an endpoint becomes 50% slower after a new deployment, ScanAPI will still pass (as long as it doesn't timeout), and the developer might miss a critical performance regression.
Do you have any suggestions on how to add this feature in scanapi ?
HistoryManagerclass to handle reading/writing to a local, hidden file (e.g.,.scanapi_history.jsonin the project root or user's cache directory). Ensure the file is capped (e.g., keeping only the last 20 runs) to keep it lightweight.⚠️ Warning: GET /users is 25% slower than its 10-run average (250ms vs 200ms).--no-history) or to clear history.