Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions python/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ Highlights:
- **Pass/fail mapping.** Every pytest outcome (pass, assertion failure, exception, skip, xfail, hard exit) maps to a `TestStatus` and propagates to parent steps and the report. `step.measure(...)` returns a pass/fail boolean without raising, so all measurements land in the report even when one fails; `step.fail_if_measurements_failed()` fails the test at the end without adding assertion noise to `error_info`.
- **Assertion messages as error info.** Assertion failure messages are reported as the step's error info.
- **Git metadata.** Repo, branch, and commit are captured on the report automatically.
- **Terminal output.** The plugin prints a session header with the SDK version and active mode, and an end-of-run `Sift report` panel showing the test case, outcome, step and measurement breakdowns (color-coded), test system/operator, plus a link to the report (online), the saved log and upload command (offline), or a disabled note. Both suppress under `-q`. `SiftClient.app_url` exposes the web-app origin; set `sift_report_url_base` for on-prem or custom deployments. `--sift-open-report` opens the report in a browser at session end.

See the [Pytest Plugin guide](https://github.com/sift-stack/sift/blob/main/python/docs/guides/pytest_plugin/index.md) and the runnable quickstart example for full configuration.

Expand Down
3 changes: 3 additions & 0 deletions python/docs/guides/pytest_plugin/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,8 @@ def sift_client() -> SiftClient:
| `--sift-disabled` | off | Skip Sift entirely. Nothing contacts the API and no log file is written; `step.measure(...)` still evaluates bounds and returns a real pass/fail boolean. Also honored via `SIFT_DISABLED=1`. Supersedes every other flag (disabled wins over offline). |
| `--sift-log-file=<path\|true\|false>` | temp file | Where the JSONL log of create/update calls goes. With a log file set, the plugin spawns an `import-test-result-log --incremental` worker that polls the file and replays entries against Sift while the run is in flight. Pass `false` to disable the file entirely; create/update calls then go straight to the API synchronously during tests. Incompatible with `--sift-offline` since offline mode needs the log file as its sole sink. |
| `--no-sift-git-metadata` | git metadata on | Skip capturing git repo/branch/commit on the report's metadata. |
| `--sift-report-url-base=<origin>` | derived from REST URI | Web-app origin used to build the clickable report link in the terminal footer (e.g. `https://app.siftstack.com`). Set this for on-prem or custom deployments whose API host can't be mapped to a frontend automatically. Also honored via the `SIFT_APP_URL` environment variable. When unset, the link is derived from the REST URI for known Sift hosts. |
| `--sift-open-report` | off | Open the resulting report in a browser at session end. Online mode only; a no-op when the report URL can't be resolved. Intended for local development. |

These can be passed permanently via `addopts`:

Expand All @@ -158,6 +160,7 @@ CLI flags, when passed, override the ini values.
| `sift_module_step` | bool (default `true`) | _(ini-only)_. Opens a parent step for each test module (file). |
| `sift_class_step` | bool (default `true`) | _(ini-only)_. Opens a parent step for each test class, including nested classes. |
| `sift_parametrize_nesting` | bool (default `true`) | _(ini-only)_. Clusters parametrized tests under shared parents (`test_x`, `axis=value`) instead of flat leaves (`test_x[value]`). |
| `sift_open_report` | bool (default `false`) | `--sift-open-report` |

```toml title="pyproject.toml"
[tool.pytest.ini_options]
Expand Down
82 changes: 82 additions & 0 deletions python/docs/guides/pytest_plugin/running_modes.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,88 @@ pytest --sift-log-file=./sift-results.jsonl
Pass both flags and disabled wins: it skips Sift entirely and supersedes every
other setting.

## Terminal output

Each run prints a header with the SDK version and active mode, and an end-of-run
`Sift report` panel summarizing the outcome. Both are suppressed under `-q`. The
panel is color-coded when the terminal supports it (green pass, red
failure/error, yellow skip, cyan link) and plain text otherwise (`--color=no`,
captured output, CI logs).

The section title carries the report name (truncated if long). The `Steps` row
tallies every step in the report by final status, so it counts substeps and the
package/module/class/parametrize grouping steps too — its totals are expected to
exceed pytest's own test count. The `Measurements` row tallies recorded
measurements (`step.measure(...)`) and is omitted when there are none. The
`Test case` and `System` rows echo the report's test case, test system, and
operator.

**Online** shows the report metadata, step and measurement breakdowns, and a
clickable link. The web host is derived from the REST URI for known Sift hosts;
for on-prem or custom deployments set `--sift-report-url-base`
(ini: `sift_report_url_base`, env: `SIFT_APP_URL`). Add `--sift-open-report` to
open the report in a browser at session end.

```text
============================= test session starts ==============================
platform linux -- Python 3.11.8, pytest-8.3.2, pluggy-1.5.0
Sift: sift-stack-py 0.17.0 — online mode
collected 12 items

tests/test_battery.py ........ [ 66%]
tests/test_thermal.py .... [100%]

================ Sift report · pytest tests/ 2026-05-27T22:44:23Z ==============
Test case pytest tests/
Status PASSED online · sift-stack-py 0.17.0
Steps 14 passed
Measurements 42 passed
System ci-runner-7 · cibot
Log file /tmp/sift-a1b2c3.jsonl
Report https://app.siftstack.com/test-results/0193f1a2-7c44-7e5b-9b1a-2f6c0d9e84aa
============================== 12 passed in 3.45s ==============================
```

If the background uploader doesn't finish, the panel still links the report and
flags that it may be incomplete:

```text
================ Sift report · pytest tests/ 2026-05-27T22:44:23Z ==============
Test case pytest tests/
Status FAILED online · sift-stack-py 0.17.0
Steps 11 passed · 2 failed · 1 error
Measurements 40 passed · 3 failed
System ci-runner-7 · cibot
Log file /tmp/sift-a1b2c3.jsonl
Report https://app.siftstack.com/test-results/0193f1a2-7c44-7e5b-9b1a-2f6c0d9e84aa
may be incomplete — finish with: import-test-result-log /tmp/sift-a1b2c3.jsonl
```

When the web host can't be resolved and no override is set, the `Report` row
shows the report id instead of a link.

**Offline** shows the metadata and breakdowns, then the upload command under a
small rule (the log path is part of the command):

```text
================ Sift report · pytest tests/ 2026-05-27T22:44:23Z ==============
Test case pytest tests/
Status PASSED offline · not uploaded
Steps 14 passed
Measurements 42 passed
System ci-runner-7 · cibot
Log file ./run.jsonl
------------------------------ to upload to Sift -------------------------------
>> import-test-result-log ./run.jsonl
```

**Disabled** notes that no report was created:

```text
===================================== Sift =====================================
Sift disabled — no test report created.
```

## Online mode (default)

`report_context` resolves `client_has_connection` at session start. The default
Expand Down
25 changes: 5 additions & 20 deletions python/lib/sift_client/_internal/grpc_transport/transport.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@

from importlib.metadata import PackageNotFoundError, version
from typing import TYPE_CHECKING, Any, TypedDict, cast
from urllib.parse import ParseResult, urlparse

import grpc
import grpc.aio as grpc_aio
Expand All @@ -21,6 +20,7 @@
Metadata,
MetadataInterceptor,
)
from sift_client._internal.urls import parse_host

if TYPE_CHECKING:
from sift_client._internal.grpc_transport._async_interceptors.base import ClientAsyncInterceptor
Expand Down Expand Up @@ -78,7 +78,7 @@ def use_sift_channel(

credentials = get_ssl_credentials(cert_via_openssl)
options = _compute_channel_options(config)
api_uri = _clean_uri(config["uri"], use_ssl)
api_uri = parse_host(config["uri"])
channel = grpc.secure_channel(api_uri, credentials, options)
interceptors = _compute_sift_interceptors(config, metadata)
return grpc.intercept_channel(channel, *interceptors)
Expand All @@ -98,7 +98,7 @@ def use_sift_async_channel(
return _use_insecure_sift_async_channel(config, metadata)

return grpc_aio.secure_channel(
target=_clean_uri(config["uri"], use_ssl),
target=parse_host(config["uri"]),
credentials=get_ssl_credentials(cert_via_openssl),
options=_compute_channel_options(config),
interceptors=_compute_sift_async_interceptors(config, metadata),
Expand All @@ -112,7 +112,7 @@ def _use_insecure_sift_channel(
FOR DEVELOPMENT PURPOSES ONLY
"""
options = _compute_channel_options(config)
api_uri = _clean_uri(config["uri"], False)
api_uri = parse_host(config["uri"])
channel = grpc.insecure_channel(api_uri, options)
interceptors = _compute_sift_interceptors(config, metadata)
return grpc.intercept_channel(channel, *interceptors)
Expand All @@ -125,7 +125,7 @@ def _use_insecure_sift_async_channel(
FOR DEVELOPMENT PURPOSES ONLY
"""
return grpc_aio.insecure_channel(
target=_clean_uri(config["uri"], False),
target=parse_host(config["uri"]),
options=_compute_channel_options(config),
interceptors=_compute_sift_async_interceptors(config, metadata),
)
Expand Down Expand Up @@ -205,21 +205,6 @@ def _metadata_async_interceptor(
return MetadataAsyncInterceptor(md)


def _clean_uri(uri: str, use_ssl: bool) -> str:
"""
This will automatically transform the URI to an acceptable form regardless of whether or not
users included the scheme in the URL or included trailing slashes.
"""

if "http://" in uri or "https://" in uri:
parsed: ParseResult = urlparse(uri)
return parsed.netloc

full_uri = f"https://{uri}" if use_ssl else f"http://{uri}"
parsed_res: ParseResult = urlparse(full_uri)
return parsed_res.netloc


def _compute_user_agent() -> str:
try:
return f"sift_stack_py/{version('sift_stack_py')}"
Expand Down
4 changes: 2 additions & 2 deletions python/lib/sift_client/_internal/rest.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from typing_extensions import NotRequired
from urllib3.util import Retry

from sift_client._internal.grpc_transport.transport import _clean_uri
from sift_client._internal.urls import parse_host

_DEFAULT_REST_RETRY = Retry(total=3, status_forcelist=[500, 502, 503, 504], backoff_factor=1)

Expand All @@ -33,7 +33,7 @@ class SiftRestConfig(TypedDict):
def compute_uri(restconf: SiftRestConfig) -> str:
uri = restconf["uri"]
use_ssl = restconf.get("use_ssl", True)
clean_uri = _clean_uri(uri, use_ssl)
clean_uri = parse_host(uri)

if use_ssl:
return f"https://{clean_uri}"
Expand Down
55 changes: 55 additions & 0 deletions python/lib/sift_client/_internal/urls.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
"""Helpers for turning Sift API endpoints into web-app (frontend) URLs.

The Sift frontend can be hosted on several domains and the backend exposes no
field for its own URL, so the frontend origin is derived client-side from the
API host. This table mirrors the canonical mapping used by the Grafana
datasource (sift-stack/sift-grafana-datasource,
``src/components/sharelink/getFrontendHostnameDefaults.ts``). Hosts outside the
table (on-prem and custom deployments) require an explicit override.
"""

from __future__ import annotations

from urllib.parse import urlparse

# API host (host[:port], no scheme) -> frontend origin (with scheme).
_API_HOST_TO_FRONTEND_ORIGIN: dict[str, str] = {
"api.siftstack.com": "https://app.siftstack.com",
"gov.api.siftstack.com": "https://gov.siftstack.com",
}


def parse_origin(url: str) -> str:
"""Normalize a URL or bare host into a ``scheme://host[:port]`` origin.

Bare hosts (no scheme) are assumed to be ``https``.
"""
candidate = url if "://" in url else f"https://{url}"
parsed = urlparse(candidate)
return f"{parsed.scheme}://{parsed.netloc}".rstrip("/")


def parse_host(url: str) -> str:
"""Extract ``host[:port]`` from a URL or bare host string."""
candidate = url if "://" in url else f"https://{url}"
return urlparse(candidate).netloc


def frontend_origin_for_api(api_base_url: str, override: str | None = None) -> str | None:
"""Return the Sift web-app origin for a given API base URL.

Args:
api_base_url: The REST API base URL (e.g. ``https://api.siftstack.com``).
override: An explicit frontend origin (host or full URL) to use instead
of the derived value. Set this for on-prem or custom deployments
whose API host isn't in the built-in mapping.

Returns:
The frontend origin (e.g. ``https://app.siftstack.com``), or ``None``
when no override is given and the API host isn't recognized.
"""
if override:
return parse_origin(override)
if not api_base_url:
return None
return _API_HOST_TO_FRONTEND_ORIGIN.get(parse_host(api_base_url))
2 changes: 1 addition & 1 deletion python/lib/sift_client/_tests/pytest_plugin/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@

import pytest

_SIFT_ENV_VARS = ("SIFT_API_KEY", "SIFT_GRPC_URI", "SIFT_REST_URI", "SIFT_DISABLED")
_SIFT_ENV_VARS = ("SIFT_API_KEY", "SIFT_GRPC_URI", "SIFT_REST_URI", "SIFT_DISABLED", "SIFT_APP_URL")


@pytest.fixture
Expand Down
Loading
Loading