Repository navigation
Software testing_2
This page gathers the test cases designed to validate the user stories of Sprint 2. The idea is to have a clear inventory that any team member can review, execute, and understand, regardless of whether the test is manual or automated.
We took the Sprint 2 backlog and, for each user story, derived one or more test cases based on two things: the happy path that must always work, and at least one alternative path where something goes wrong or the user does something unexpected. This resulted in a total of 15 cases.
Out of those 15, we automated 13 (those that can be executed without human intervention) and left 2 as manual: the end-to-end flow with a real Docker container, and the login test with invalid credentials, which was already executed in Sprint 1 and is maintained as regression.
By the end of the sprint, we passed 14 out of the 15 cases. The only pending one is the end-to-end case (CP-14), which will be executed during the presentation because it requires bringing up the full Docker stack along with Supabase.
Linked to user story US-03. The precondition is that the backend is running and Supabase is accessible. The steps are to send a POST to /api/alerts with a typical Alertmanager payload in firing state, alertname ContainerCrashed, and a label name=app-demo. After that, query the incidents table in Supabase.
The expected result is a 200 response, and that in Supabase there exists an incident with target app-demo, severity high, status detected, container_runtime docker, and source_type container.
This case is automated in test_alert_processor.py::test_process_firing_alert_creates_incident and passes.
Also from US-03. The idea is that when Prometheus triggers the resolution of an alert, the backend should close the associated incidents without creating a new one. The test simulates having an active incident and then sends the same alert with status resolved. The expected result is that the incident ends up with status resolved, the resolved_at field is populated, and the function does not return any incident_id because nothing was created.
Automated and passing.
Alternative case of US-03. If a new alert arrives that is not yet registered in SEVERITY_MAP, the system should not break. It should assign a default severity of medium and still create the incident. The test sends an alert with a made-up alertname and verifies that the incident is created with severity medium.
Automated and passing.
Parameterized case that covers 11 combinations of alerts, including Docker, Podman, and Postgres. We validate that ContainerOOMKilled maps to critical, ContainerCrashed to high, ContainerCPUThrottling to medium, PostgresConnectionsExhausted to critical, PostgresLowCacheHitRatio to medium, and so on for the rest. If someone adds a new alert and forgets to update the map, this test will catch it.
Automated and passing.
Linked to US-06. The test calls query_loki_logs with a container_id and an alert_time, simulating a Loki response with two streams and multiple lines. The expected result is a formatted string with lines ordered chronologically and each with a readable UTC timestamp.
Automated and passing.
Critical alternative case of US-06. The idea is that if Loki is down, the system must not fail when creating an incident: it simply does not attach logs and continues. The test simulates a ConnectionError when calling Loki and verifies that the function returns an empty string without raising an exception.
Automated and passing.
Linked to US-04. When an incident arrives with label container_runtime=docker, the supervisor must route it to the DockerAgent. The test creates an IncidentContext with that label and calls find_agent_for(ctx), expecting the returned agent to be the DockerAgent.
Automated and passing.
Alternative case of US-04, essential to maintain domain separation. If an incident has target postgres/customers and source_type database, the DockerAgent must not match, because it is not its domain. The test verifies that DockerAgent.matches(ctx) returns False in that context.
Automated and passing.
Linked to US-01. Performing a GET to /api/incidents/ with query params such as severity=critical&status=detected should return only incidents that meet both criteria, ordered by descending date. The test verifies that the response is 200 and that the list respects the filters.
Automated and passing.
Alternative case of US-02. If the user sends a POST to /api/incidents/ with a severity not included in the allowed enum (for example ultra-critical), the endpoint must respond with 422 and a Pydantic error detail explaining the issue. The test confirms that validation works.
Automated and passing.
Frontend case, pure function from the ApprovalModal component. It has several scenarios: incident_type oom should return High level, dependency_failure should return Medium, any command containing the word logs should return Low (because it is read-only), and an unknown incident_type should fall back to default Medium.
Automated in getRisk.test.js and passing.
Linked to the HITL (Human In The Loop) approval flow. The test renders the component with a sample incident, writes a comment in the textarea, clicks the Approve button, and verifies that the onApprove function was called with the comment text.
Automated in ApprovalModal.test.jsx and passing.
Alternative case of the approval flow. The user should be able to close the modal without clicking. The test renders the modal and simulates a keyboard event with the Escape key, expecting onClose to be called once.
Automated and passing.
This is the most important manual case because it validates that the entire flow works together. The steps are to bring up the stack with docker compose up -d, start the backend with uvicorn, create a crashing container with docker run -d --name app-demo alpine sh -c "echo BOOM; sleep 2; exit 1", wait about 30 seconds, and open the frontend.
The expected result is that the incident appears in the dashboard in real time (via Supabase Realtime), with status analyzed, an incident_type classified by the agent, and an agent_reasoning populated in markdown. The entire flow should take less than 30 seconds, leaving margin over NFR-02 (15 seconds).
Pending execution during the presentation.
Linked to HU-00. This is a regression test from Sprint 1. Incorrect credentials are entered in the login form, and it is verified that an error message appears and that the user is not redirected to the dashboard.
Executed and passing, with no regressions detected.
So that any team member can confirm that the sprint is properly covered, here is the traceability:
US-01 is covered by CP-09. US-02 by CP-10 plus model tests. US-03 by CP-01, CP-02, CP-03, and CP-04. US-04 by CP-07 and CP-08. US-06 by CP-05 and CP-06. The human approval story by CP-11, CP-12, and CP-13. The full product flow by CP-14. Login (regression) by CP-15.
From the Backend folder, pytest -v runs the 13 automated backend cases plus model tests. From the Frontend folder, npm run test runs the component and pure function tests. A green output is the evidence that they passed.
The full detail (with exact steps and expected results for each case) is in docs/sprint2-pruebas/02-casos-de-prueba.md within the repository.