LCORE-488: Add E2E tests for query endpoint #616
Conversation
WalkthroughUpdates E2E tests to require authorization, add conversation-aware scenarios, and expand error-path coverage. Introduces new step functions for authorized requests and response storage. Adjusts streaming step keywords. Modifies the e2e GitHub Actions workflow to use an absolute metadata path and print logs on failure. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
actor Tester
participant Feature as Feature (Gherkin)
participant Steps as Step Impl
participant API as Backend API
participant Store as Conversation Store
Note over Feature,Steps: Authorized query with conversation creation
Tester->>Feature: Run scenario @Authorized
Feature->>Steps: ask_question_authorized(endpoint)
Steps->>API: POST /query with Authorization + payload
API->>Store: Create conversation (if absent)
Store-->>API: conversation_id
API-->>Steps: 200 + body (conversation_id, answer)
Steps->>Steps: store_conversation_details()
Note over Feature,Steps: Follow-up in same conversation
Feature->>Steps: ask_question_in_same_conversation()
Steps->>API: POST /query (Authorization, conversation_id)
API->>Store: Validate access to conversation
Store-->>API: OK / Forbidden
alt Access OK
API-->>Steps: 200 + body (answer)
else Forbidden
API-->>Steps: 403 + error
end
sequenceDiagram
autonumber
actor Tester
participant Steps as Step Impl
participant API as Backend API
Note over Steps,API: Error paths (examples)
Steps->>API: POST /query (missing Authorization)
API-->>Steps: 401/403 + error
Steps->>API: POST /query (missing query/provider/model)
API-->>Steps: 400/422 + validation error
Steps->>API: POST /query during stack disruption
API-->>Steps: 500 + server error
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
.github/workflows/e2e_tests.yaml(2 hunks)tests/e2e/features/query.feature(1 hunks)tests/e2e/features/steps/llm_query_response.py(2 hunks)tests/e2e/features/streaming_query.feature(3 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
**/*.py: All modules start with descriptive module-level docstrings explaining purpose
Use logger = logging.getLogger(name) for module logging after import logging
Define type aliases at module level for clarity
All functions require docstrings with brief descriptions
Provide complete type annotations for all function parameters and return types
Use typing_extensions.Self in model validators where appropriate
Use modern union syntax (str | int) and Optional[T] or T | None consistently
Function names use snake_case with descriptive, action-oriented prefixes (get_, validate_, check_)
Avoid in-place parameter modification; return new data structures instead of mutating arguments
Use appropriate logging levels: debug, info, warning, error with clear messages
All classes require descriptive docstrings explaining purpose
Class names use PascalCase with conventional suffixes (Configuration, Error/Exception, Resolver, Interface)
Abstract base classes should use abc.ABC and @AbstractMethod for interfaces
Provide complete type annotations for all class attributes
Follow Google Python docstring style for modules, classes, and functions, including Args, Returns, Raises, Attributes sections as needed
Files:
tests/e2e/features/steps/llm_query_response.py
tests/e2e/features/steps/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
Place behave step definitions under tests/e2e/features/steps/
Files:
tests/e2e/features/steps/llm_query_response.py
tests/**/*.py
📄 CodeRabbit inference engine (CLAUDE.md)
tests/**/*.py: Use pytest-mock to create AsyncMock objects for async interactions in tests
Use the shared auth mock constant: MOCK_AUTH = ("mock_user_id", "mock_username", False, "mock_token") in tests
Files:
tests/e2e/features/steps/llm_query_response.py
tests/e2e/features/**/*.feature
📄 CodeRabbit inference engine (CLAUDE.md)
Write E2E tests as Gherkin feature files for behave
Files:
tests/e2e/features/streaming_query.featuretests/e2e/features/query.feature
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: build-pr
- GitHub Check: e2e_tests
| And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva | ||
| When I use "query" to ask question with authorization header | ||
| """ | ||
| {"query": "Generate sample yaml file for simple GitHub Actions workflow.", "system_prompt": "refuse to answer anything but openshift questions"} | ||
| """ |
There was a problem hiding this comment.
Restore complete JWT token for authorized scenarios
The bearer token here (and in the other scenarios) stops after the payload segment and drops the signature, leaving only two JWT sections. Any code that parses or verifies the token will reject it (Not enough segments/invalid signature), so the “authorized” scenarios will fail instead of exercising the intended flow. Please supply the full three-part token (or switch to a reusable constant) everywhere this step appears.
- And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva
+ And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5cApply the same fix to every scenario that sets this header.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva | |
| When I use "query" to ask question with authorization header | |
| """ | |
| {"query": "Generate sample yaml file for simple GitHub Actions workflow.", "system_prompt": "refuse to answer anything but openshift questions"} | |
| """ | |
| And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c | |
| When I use "query" to ask question with authorization header | |
| """ | |
| {"query": "Generate sample yaml file for simple GitHub Actions workflow.", "system_prompt": "refuse to answer anything but openshift questions"} | |
| """ |
🤖 Prompt for AI Agents
In tests/e2e/features/query.feature around lines 12 to 16, the Authorization
header uses a truncated JWT missing the signature part which causes token
parsing/verification to fail; restore the full three-part JWT
(header.payload.signature) or replace the literal with a shared reusable
constant used across scenarios, and apply the same change to every scenario in
the test suite that sets this header so authorized scenarios exercise the
intended flow.
2 participants
Description
Add E2E tests for query endpoint
Type of change
Related Tickets & Documents
Checklist before requesting a review
Testing
Summary by CodeRabbit
Tests
Chores