What E2E smoke tests would be mandatory before trusting an AI-assisted classification workflow?

[kalaba992]

Hi Cypress community,

I’m building an AI-powered customs classification assistant. The production app is private, but I created a sanitized public demo so people can review the flow safely.

Live demo:
https://carinski-ekosistem-demo.pages.dev

Public demo repo:
https://github.com/kalaba992/Carinski-ekosistem-demo

The demo is guest-only and contains no private data, no credentials, no Auth0, no Convex, no OpenAI, and no production infrastructure. It uses synthetic examples and illustrative outputs only.

My question is specifically about Cypress / E2E testing strategy:

For a production-grade app with:

• authenticated login
• user-specific classification history
• protected write paths
• admin-only access
• audit-style classification output
• confidence / warning states
• strict “no fake auth / no mock evidence” release gates

what E2E or smoke tests would you consider mandatory before the system could be trusted?

The highest-value feedback for me would be:

1. Which flows must be covered by E2E?
2. Which flows should be unit/integration tests instead?
3. What auth/session edge cases are commonly missed?
4. What CI gates would you require?
5. What failure mode would be most dangerous to miss?
6. What Cypress anti-patterns should I avoid?

I created a structured testing review template here:
https://github.com/kalaba992/Carinski-ekosistem-demo/issues/new?template=testing_review.yml

Critical feedback is welcome. I’m not asking anyone to build tests for me — I’m trying to understand what experienced Cypress users would consider the minimum trustworthy E2E coverage.

[qunoot-ahmed]

For a regulated workflow like this, I'd keep the Cypress smoke suite focused on business risk rather than AI behavior itself.

My must-have smoke coverage would be:

• Login/logout and session expiry handling
• Protected route access (authenticated vs unauthenticated users)
• User role validation (user vs admin)
• Successful classification flow
• Classification failure/error handling
• Verification that results are stored and visible in history
• Data isolation (User A cannot access User B's records)

For auth/session edge cases, I'd specifically test:

• Session expiration during an active workflow
• Direct navigation to protected URLs
• Back-button behavior after logout
• Multiple users using the same browser session

I'd leave confidence scoring, classification rules, and AI response validation to API/integration tests because those tend to be more deterministic and easier to maintain than E2E tests.

The biggest mistake I see teams make is spending too much E2E effort validating AI output and not enough effort validating permissions, data access, and failure states. Those are usually the areas that create the highest production risk.

For CI, I'd make the smoke suite a mandatory deployment gate and keep it small enough that it runs in a few minutes.

[kalaba992]

Thanks a lot for the detailed answer — this is exactly the kind of practical perspective I was looking for.

I especially agree with your point about keeping E2E smoke tests focused on business risk rather than trying to validate the AI output itself. For this kind of regulated workflow, permissions, access control, failure states, and stored history are probably the areas where a small mistake can cause the biggest damage.

The distinction between E2E smoke tests and API/integration tests for scoring, rules, and response validation also makes a lot of sense.

I appreciate the clear breakdown. This gives me a much better direction for keeping the smoke suite small, mandatory, and useful as a deployment gate.

[oloualex101-jpg]

Hiii

[petriellorichard7-star]

-