Skip to content

fix constant hostname in e2e tests #712

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 23, 2025
Merged

fix constant hostname in e2e tests #712

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions tests/e2e/features/authorized_noop.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ Feature: Authorized endpoint API tests for the noop authentication module

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1

Scenario: Check if the authorized endpoint works fine when user_id and auth header are not provided
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/authorized_noop_token.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ Feature: Authorized endpoint API tests for the noop-with-token authentication mo

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1

Scenario: Check if the authorized endpoint fails when user_id and auth header are not provided
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/conversations.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ Feature: conversations endpoint API tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1


Expand Down
11 changes: 7 additions & 4 deletions tests/e2e/features/environment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
import requests
import subprocess
import time
import os
from behave.model import Scenario, Feature
from behave.runner import Context

Expand All @@ -21,14 +22,16 @@
)


def _fetch_models_from_service(hostname: str = "localhost", port: int = 8080) -> dict:
def _fetch_models_from_service() -> dict:
"""Query /v1/models endpoint and return first LLM model.

Returns:
Dict with model_id and provider_id, or empty dict if unavailable
"""
try:
url = f"http://{hostname}:{port}/v1/models"
host_env = os.getenv("E2E_LSC_HOSTNAME", "localhost")
port_env = os.getenv("E2E_LSC_PORT", "8080")
url = f"http://{host_env}:{port_env}/v1/models"
response = requests.get(url, timeout=5)
response.raise_for_status()
data = response.json()
Expand Down Expand Up @@ -105,7 +108,7 @@ def after_scenario(context: Context, scenario: Scenario) -> None:
"llama-stack",
"curl",
"-f",
"http://localhost:8321/v1/health",
f"http://{context.hostname_llama}:{context.port_llama}/v1/health",
],
capture_output=True,
timeout=5,
Expand Down Expand Up @@ -155,7 +158,7 @@ def after_feature(context: Context, feature: Feature) -> None:
if "Feedback" in feature.tags:
print(context.feedback_conversations)
for conversation_id in context.feedback_conversations:
url = f"http://localhost:8080/v1/conversations/{conversation_id}"
url = f"http://{context.hostname}:{context.port}/v1/conversations/{conversation_id}"
headers = context.auth_headers if hasattr(context, "auth_headers") else {}
response = requests.delete(url, headers=headers)
assert response.status_code == 200, url
2 changes: 0 additions & 2 deletions tests/e2e/features/feedback.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ Feature: feedback endpoint API tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1
And I set the Authorization header to Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6Ikpva

Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/health.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ Feature: REST API tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1


Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/info.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ Feature: Info tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1

Scenario: Check if the OpenAPI endpoint works as expected
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/query.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ Feature: Query endpoint API tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1

Scenario: Check if LLM responds properly to restrictive system prompt to sent question with different system prompt
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/rest_api.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ Feature: REST API tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1

Scenario: Check if the OpenAPI endpoint works as expected
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/smoketests.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@ Feature: Smoke tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1


Expand Down
5 changes: 5 additions & 0 deletions tests/e2e/features/steps/common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,17 @@

from behave import given # pyright: ignore[reportAttributeAccessIssue]
from behave.runner import Context
import os


@given("The service is started locally")
def service_is_started_locally(context: Context) -> None:
"""Check the service status."""
assert context is not None
context.hostname = os.getenv("E2E_LSC_HOSTNAME", "localhost")
context.port = os.getenv("E2E_LSC_PORT", "8080")
context.hostname_llama = os.getenv("E2E_LLAMA_HOSTNAME", "localhost")
context.port_llama = os.getenv("E2E_LLAMA_PORT", "8321")
Comment on lines 8 to +15
Copy link
Contributor

@coderabbitai coderabbitai bot Oct 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Update the docstring and verify port type expectations.

The docstring states "Check the service status" but the function actually configures the context with hostname and port values from environment variables. Additionally, ports are stored as strings, but downstream code may expect integers.

Apply this diff to update the docstring:

 @given("The service is started locally")
 def service_is_started_locally(context: Context) -> None:
-    """Check the service status."""
+    """Configure context with service hostnames and ports from environment variables."""
     assert context is not None

Run the following script to verify whether downstream code expects ports as integers:

🏁 Script executed:

#!/bin/bash
# Description: Check if port values are used in numeric contexts or string interpolation

# Search for usage of context.port and context.port_llama
rg -n -C3 'context\.port' tests/e2e/

Length of output: 12969

Update docstring and convert port strings to integers for type consistency.

The docstring "Check the service status" is misleading—the function actually configures context with hostname and port values from environment variables. Additionally, os.getenv() returns strings, but other code (e.g., set_service_port in common_http.py) sets context.port as an integer, creating a type inconsistency.

Apply this diff:

 @given("The service is started locally")
 def service_is_started_locally(context: Context) -> None:
-    """Check the service status."""
+    """Configure context with service hostnames and ports from environment variables."""
     assert context is not None
     context.hostname = os.getenv("E2E_LSC_HOSTNAME", "localhost")
-    context.port = os.getenv("E2E_LSC_PORT", "8080")
+    context.port = int(os.getenv("E2E_LSC_PORT", "8080"))
     context.hostname_llama = os.getenv("E2E_LLAMA_HOSTNAME", "localhost")
-    context.port_llama = os.getenv("E2E_LLAMA_PORT", "8321")
+    context.port_llama = int(os.getenv("E2E_LLAMA_PORT", "8321"))
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
@given("The service is started locally")
def service_is_started_locally(context: Context) -> None:
"""Check the service status."""
assert context is not None
context.hostname = os.getenv("E2E_LSC_HOSTNAME", "localhost")
context.port = os.getenv("E2E_LSC_PORT", "8080")
context.hostname_llama = os.getenv("E2E_LLAMA_HOSTNAME", "localhost")
context.port_llama = os.getenv("E2E_LLAMA_PORT", "8321")
@given("The service is started locally")
def service_is_started_locally(context: Context) -> None:
"""Configure context with service hostnames and ports from environment variables."""
assert context is not None
context.hostname = os.getenv("E2E_LSC_HOSTNAME", "localhost")
context.port = int(os.getenv("E2E_LSC_PORT", "8080"))
context.hostname_llama = os.getenv("E2E_LLAMA_HOSTNAME", "localhost")
context.port_llama = int(os.getenv("E2E_LLAMA_PORT", "8321"))
🤖 Prompt for AI Agents 
In tests/e2e/features/steps/common.py around lines 8 to 15, the docstring is
misleading and port values are left as strings; update the docstring to describe
that the function configures context hostnames and ports from environment
variables, and convert E2E_LSC_PORT and E2E_LLAMA_PORT to integers (with safe
defaults) before assigning to context.port and context.port_llama so their types
match other code that expects ints.



@given("The system is in default state")
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/features/streaming_query.feature
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,6 @@ Feature: streaming_query endpoint API tests

Background:
Given The service is started locally
And REST API service hostname is localhost
And REST API service port is 8080
And REST API service prefix is /v1


Expand Down
Loading