Repository navigation
[RFC] 日志输出规范 | About service log level and output formats #7753
Replies: 2 comments
Developer Logging GuideThis document defines log level semantics, content boundaries, and implementation rules for every OpenViking runtime (Python, Rust, the C++ engine, TypeScript). It targets contributors, not operators; Related entry points:
1. Level semanticsA record's level is decided by what the record is for, not by how serious it feels.
Additional rules:
Decision orderCommon misjudgements
2. Why not "severity"Severity-based levels drift with each developer's judgement: one person calls a failure Purpose-based levels are objective and decidable:
Each level then maps to exactly one consumer, and alerting needs no second filtering pass. 3. Audit logging (the only legitimate use of
|
| Field | Meaning |
|---|---|
ts |
UTC timestamp, ISO 8601 |
event |
Stable name, e.g. audit.account.create, audit.acl.grant |
result |
success / failure |
actor_id / actor_role / actor_type |
Actor identity; actor_type is user, admin, root, or service |
account_id |
Owning account |
target_type / target_id |
Object acted on, e.g. user, api_key, config |
action |
Action name, matching the last segment of event |
request_id / trace_id |
Correlation identifiers |
source_ip / user_agent |
Entry-point metadata, may be empty |
error_code |
Stable error code on failure; never a stack trace |
3.3 Output form
Audit needs two sinks, not one:
- Structured event via
openviking.observability.events.try_publish_event(), projected byusage_audit. This is the queryable, retention-managed primary channel. - Log copy on the dedicated
openviking.auditlogger atWARNING. This is the side-channel copy for log collection, SIEM, and offline forensics.
The log copy uses a fixed, splittable key-value layout, not prose:
audit_logger.warning(
"audit event=%s actor=%s role=%s account=%s target=%s/%s result=%s reason=%s",
"acl.grant",
actor_id,
actor_role,
account_id,
"user",
target_user_id,
"success",
"",
extra={"request_id": request_id, "trace_id": trace_id},
)Constraints:
- Audit after execution and record the actual outcome; write both success and failure, never success only.
- A failed audit write is itself
ERRORand must not roll back the business transaction. - Audit records must never be swallowed by a business-layer
except Exception. - Never put plaintext keys, passwords, tokens, or full request bodies in an audit message.
3.4 Anti-pattern
# Wrong: an account migration logged at INFO, invisible to the audit pipeline
logger.info("Migrated API key for user %s in account %s", user_id, account_id)
# Right
audit_log(
event="api_key.migrate",
result="success",
actor_type="system",
account_id=account_id,
target_type="api_key",
target_id=user_id,
)3.5 ROOT invocations
ROOT is the highest-privilege identity. Under api_key auth mode, every call made under a ROOT identity must leave one WARNING audit record, even when the call is read-only and even when it has no business impact.
Rationale: when ROOT is misused or leaked, the only way to reconstruct what happened is the trail. A read-only call can still read across tenants, so it is no less dangerous than a write. A ROOT call to an ordinary business endpoint would normally be INFO; here WARNING wins.
Trigger
Both conditions must hold:
- The auth mode is
api_key(AuthMode.API_KEY); - The resolved
identity.role == Role.ROOT, that is, the caller holds the root API key (server/api_keys/key_mgr.py:131).
Non-triggering cases
| Case | Reason |
|---|---|
dev mode (auth/plugins/dev.py:41) |
No authentication; every request is synthesized as ROOT and is not a real superuser |
trusted mode (auth/plugins/trusted.py:183) |
ROOT is injected by the gateway (admin paths, the /health control probe) and does not represent a real actor |
| Custom auth modes | Semantics are unknown, so nothing is recorded by default; a mode that needs audit should call log_audit() in its own plugin |
Unauthenticated routes (/health, /ready, /metrics) |
Never produce a ROOT identity |
| Internal ROOT contexts (TTL, semantic processor, viking_fs, system_assets, pack_service, core) | Do not pass through the auth entry point and therefore never trigger |
Where to implement
_build_request_context() (server/auth/__init__.py:113) is the only choke point that sees identity.role, the auth mode, and request.state at once. Do not scatter this across the 38 Role.ROOT references in openviking/server (18 of which are equality checks).
Prefer "mark on entry, emit on completion" so one record carries both the caller and the outcome:
# _build_request_context(): mark only, do not emit
if ctx.role == Role.ROOT and _auth_mode(request) == AuthMode.API_KEY.value:
request.state.root_audit = {
"actor_id": identity.user_id or "root-key",
"account_id": identity.account_id,
}# On request completion: emit exactly one record
if (root_audit := getattr(request.state, "root_audit", None)) is not None:
audit_logger.warning(
"audit event=root.invoke actor=%s role=root route=%s method=%s "
"status=%d result=%s request_id=%s",
root_audit["actor_id"],
route,
method,
status_code,
"success" if status_code < 400 else "failure",
request_id,
)_auth_mode() already exists (server/auth/__init__.py:32); do not read the config again. Note that it falls back to api_key when no config is available, so unit tests must inject the auth mode explicitly or their requests will be treated as real ROOT.
Required fields
event=root.invoke, actor_id, actor_type=root, account_id, http_method, http_route, http_status_code, result, request_id, trace_id, source_ip, user_agent.
- The root API key has no account or user; write
root-keyasactor_id, notdefault/default. - Record failures too, with
result=failureand anerror_code. - Mark high-risk actions (ROOT reading plaintext keys, cross-account reads or writes) with
sensitive=trueso they can be alerted separately.
Constraints
- One record per request. Do not emit once on entry and again on completion.
- No sampling. Sampling an audit trail is giving it up. If ROOT polling (Studio task polling, SDK health checks) makes the volume large, reduce the polling frequency or use a non-ROOT identity; do not drop samples server-side.
- Content is still governed by section 4: no request bodies, no raw path parameters, no context content.
4. Log security rules
Log files, trace attributes, and audit storage are three independent egress paths; a leak in any one of them is a leak. This section governs fields.
4.1 Allowlist, not denylist
Deny by default. Only the "Allowed" column in 4.2 may be emitted verbatim; every other field must be transformed (hash, truncate, redact, relativize) or reduced to an aggregate (length, count, duration, boolean). When classification is unclear, treat the field as forbidden.
4.2 Field classification
| Class | Fields | Handling |
|---|---|---|
| Allowed | account_id, user_id, request_id, trace_id, span_id, operation, http_method, http_route, http_status_code, duration_ms, role, actor_type, task_id, task_type, task_status, task_stage, retry counts, queue depth, error_code |
Emit verbatim |
| Allowed after transformation | Local paths, viking:// URIs, resource_id, exception messages |
Apply 4.4–4.8 |
| Forbidden | peer_id, keys and credentials, context content, absolute paths, full request/response bodies, model input and output |
Never emit |
Task metadata (task_id, task_type, status, stage, durations, retry and error counts, queue depth) may be logged directly; it is the most useful troubleshooting dimension. Task meta is the exception: log it wholesale only after confirming its values contain no URIs, paths, or secrets; otherwise log key names without values.
resource_id may be logged only as an opaque deployment label. If it comes from an environment variable and could be a path or URL, treat it under 4.7.
4.3 The existing injection baseline
_get_log_context() currently injects only trace_id, span_id, request_id, operation, account_id, and user_id; RootSpanAttributes.to_log_fields() deliberately excludes url_path and url_query. That is the existing baseline for this section, and any newly injected field is subject to 4.2.
However, ObservabilityContext.extra is merged into log fields unconditionally, so extra is not exempt.
4.4 peer_id
Never emit it to logs, trace attributes, or audit fields, including four indirect forms:
- URIs such as
viking://user/{user_id}/peers/{peer_id}/...embed the peer id; RequestContext.actor_peer_id;- Object serialization (
%r,%s,asdict(),model_dump()); - Resource locators inside task
meta.
# Wrong: the full URI exposes the peer
logger.info("rebuild failed uri=%s", "viking://user/u1/peers/p9/memories")
# Right: truncate to the user level, keep only a correlation hash for the peer
logger.info("rebuild failed user=%s peer=%s", user_id, short_hash(peer_id))When the same peer must be correlated across log lines, use an irreversible short hash (sha256(peer_id)[:8]). The hash is meaningful only within one deployment; do not reuse conclusions across environments.
4.5 Keys and credentials
The following are forbidden, together with any derived form that allows reconstruction (prefix, suffix, length combination):
- API keys, user keys,
submission_token,submission_hash - OAuth access/refresh tokens, JWT, OIDC assertions
- Cookies,
Authorization, signature headers, pre-signed URLs - Credentials inside database or Redis connection strings
- Encryption keys and plaintext entries managed by
ov privacy
The single exception: when diagnostic output must show a key, use mask_api_key() (last four characters only) behind an explicit diagnostic switch.
repr=False (RequestContext.api_key, TaskRecord.auth) only blocks repr(); it does not block asdict(), model_dump(), or __dict__. The rule is therefore never log a whole object, rather than relying on repr=False.
4.6 Context content
"Context content" means any text produced by a user, a model, or an external resource that has not been reduced to an equivalent summary:
- prompts, system prompts, messages, tool call arguments and results
- session bodies, memory bodies, summary bodies, rule bodies, skill bodies
- document, web, and code bodies or fragments; retrieved chunk text
- user input and model output
Allowed substitutes: character count, token count, item count, language, file type, sha256[:8], hit/miss.
The existing log_payloads switch is the counter-example: models/rerank/jev_rerank.py:191 and service/rule_jev.py:76 log the complete request and response payload at WARNING when enabled, writing queries and document bodies to disk. Direction: default off (already the default), demote to DEBUG when enabled, mark payload=true, apply redaction, and permit it only in troubleshooting environments.
4.7 Paths
Absolute paths are forbidden. They expose the deployment directory, the host user name, and structures such as workspace/{account_id}/{user_id}/peers/{peer_id}/ — leaking the peer id together with the tenant layout.
Rules:
- Log only a relative path inside the workspace, a bare file name, or an extension;
- Never log user-controlled paths verbatim (zip entry names, markdown image paths, upload file names, WebDAV paths); they may contain
../, absolute paths, or peer segments; - For correlation, use
relative path + sha256[:8]; - Treat
viking://URIs like local paths and handle the peer segment per 4.4; - The same applies inside exception messages: run
sanitize_error()first.
34 log sites currently print file_path, path_str, or resolved_path directly (e.g. parse/parsers/markdown.py:845, parse/parsers/text_encoding.py:74) and need remediation.
4.8 Exceptions and error messages
Exception strings often embed the offending value (InvalidArgumentError("bad uri: ...")), so logger.warning("...: %s", e) is itself an exfiltration channel.
- By default emit
type(exc).__name__plus a message passed throughsanitize_error(); - Emit a full stack trace only at
ERRORand only for local troubleshooting, after redaction; - Never emit
repr(exc.args).
4.9 All three egress paths
| Path | Existing protection | Requirement |
|---|---|---|
| Log file/stdout | No central entry point | Follow this section; add log_safety.py |
| OTel trace attributes | to_log_fields() already excludes url_path and url_query |
New span attributes obey this section too; keep url.path only when needed |
| Audit storage | sanitize_public_http_error(), sanitize_error(), SENSITIVE_TASK_KEYS |
Keep as is; classify new fields the same way |
4.10 Suggested implementation and checks
- Add
openviking/observability/log_safety.pywithshort_hash(),safe_uri_for_log(),safe_path_for_log(), andsafe_log_fields(). repr=Falseis a fallback, not a line of defence.- CI: AST-scan logger calls for forbidden identifiers (
peer_id,api_key,token,prompt,content,messages, absolute-path variables); warn first, then block. - Tests: unit-test the redaction helpers; add one failure case each for audit and
log_payloads.
4.11 Security review checklist
- Every new log field is in the 4.2 allowlist or has been transformed per 4.4–4.8.
- No
peer_id, and no full URI that embeds a peer segment. - No keys, credentials, tokens, cookies, or connection strings.
- No prompts, messages, bodies, chunks, or model output.
- No absolute paths; user-controlled paths are transformed.
- No
%r/%s/asdict()/model_dump()of a whole object. -
extrafields are equally compliant.
5. Fallback, retry, and 4xx rejection logging (INFO)
The reader of INFO is an operator: they need to know the system is struggling on its own, or that a caller was correctly rejected, without being paged. Every fallback record states which transition, which attempt, and what the outcome was.
5.1 Retries and backoff
logger.info(
"retry operation=%s attempt=%d/%d delay=%.2fs error=%s",
operation_name,
attempt + 1,
retry_limit,
delay,
type(exc).__name__,
)- One
INFOper retry, neverWARNING. - After retries are exhausted, emit one
ERROR— exactly one. - Aggregate batch retries; do not log one line per record.
- Keep only the exception type and a short message during retries; leave the stack trace to the final
ERROR.
5.2 Degradation and fallbacks
logger.info(
"vlm unavailable; using empty summary uri=%s file_type=%s",
file_path,
file_type,
)- Always record which fallback fired and what the degraded behaviour is; otherwise operators cannot distinguish "by design" from "silently degraded".
- When degradation makes the result incomplete (empty summary, missing fields), mark it explicitly, e.g.
degraded=true, so downstream can filter. - Never log a fallback at
WARNING; that turnsWARNINGback into a "worth noting" bin.
5.3 Circuit breakers and state transitions
INFO records every state-machine transition: CLOSED → OPEN, OPEN → HALF_OPEN, HALF_OPEN → CLOSED, plus primary/backup switch and failback. openviking/utils/circuit_breaker.py already follows this and can serve as the reference.
5.4 Startup, shutdown, migration
Process start/stop, one-off data migrations, cache warmup, and background task lifecycle are all INFO. Do not flood single-request lifecycles with INFO.
5.5 4xx business rejections
Bad client arguments, missing resources, and unauthorized requests are always INFO:
- Not a fault, so never
ERROR; otherwise caller misuse floods alerting; - It reveals a real caller problem (SDK misuse, missing permission, mistyped URI) and must stay diagnosable, so never
DEBUG; - It scales with requests and stays bounded, so it does not make
INFOnoisy.
logger.info(
"request rejected status=%d error_code=%s route=%s",
400,
code,
http_route,
)Constraints:
- One rejection record per request; do not log in both the service layer and the router.
- No
ERRORand no stack trace for a 4xx. - The security side of 401/403 (authentication failure, authorization attempt) additionally produces an audit
WARNINGper 3.1; ordinary 4xx (bad arguments, 404) never enters audit. - The response log and the audit record serve different consumers and may coexist, but one log line must not pretend to be both.
6. Business process logging (DEBUG)
The default outlet. This is the business process of the success path; rejected requests belong to INFO per 5.5. Requirements:
- Record the basis for a decision, not vacuous progress.
"start to process"/"done"carry no information. - Summarize large objects: URI, counts, lengths, hashes, durations. Never whole documents, whole prompts, or whole response bodies.
- Guard hot paths before building the message:
if logger.isEnabledFor(logging.DEBUG):
logger.debug("retrieval candidates=%s", summarize(candidates))- Intermediates go to
DEBUGwith a size cap (2 KB recommended); truncate and marktruncated=true.
7. Current state of the repository
Numbers come from the current main, scoped to openviking/, openviking_cli/, and openviking_gateway/, excluding tests/, benchmark/, and bot/.
7.1 Level distribution
| Package | DEBUG | INFO | WARNING | ERROR | EXCEPTION | CRITICAL |
|---|---|---|---|---|---|---|
openviking/ |
266 | 385 | 471 | 247 | 88 | 4 |
openviking_cli/ |
5 | 6 | 17 | 2 | 0 | 0 |
openviking_gateway/ |
1 | 0 | 5 | 0 | 8 | 0 |
| Total | 272 | 391 | 493 | 249 | 96 | 4 |
WARNING is the most used level: 26% more than INFO and 81% more than DEBUG. That is the opposite of "WARNING is audit only".
7.2 What WARNING actually contains
Of 493 WARNING calls:
- ~190 mention failure semantics (
fail,error,unavailable,invalid,unable,exception) - ~41 mention fallback semantics (
fallback,retry,degrade,skip,ignor,disabled,missing) - about 7 match audit semantics by keyword (
account,permission,role,acl,api_key,revoke,grant), and those are all failure paths (e.g. "trusted identity batch registration failed; will retry"), not audit events
About 98% of WARNING usage should move down to INFO or DEBUG under this policy, while the audit events that belong at WARNING mostly do not exist yet.
Representative anti-patterns:
| Location | Current | Should be |
|---|---|---|
openviking/utils/model_retry.py:491 |
WARNING after a retryable error, with backoff |
INFO |
openviking/models/embedder/base.py:422 |
WARNING for a slow call (≥3s) |
INFO |
openviking/server/account_stores/redis_mysql.py:485 |
WARNING when the lock is unavailable and MySQL is read |
INFO |
openviking/server/account_stores/file.py:248 |
WARNING when the account store reload fails and retries |
INFO |
openviking/storage/queuefs/semantic_processor.py:1357 |
WARNING when VLM is unavailable and an empty summary is used |
INFO |
openviking/observability/http_observability_middleware.py:557 |
WARNING for a metrics write failure |
INFO (ERROR on first occurrence) |
openviking/server/request_id.py:77 |
WARNING for a regenerated X-Request-ID |
INFO |
7.3 Audit gaps
| Module | Log calls | Note |
|---|---|---|
openviking/server/api_keys/key_mgr.py |
0 | Account creation, user registration, key regeneration, role setting, and deletion are entirely unlogged |
openviking/server/routers/acl.py |
0 | ACL get/set/delete/grant/revoke are entirely unlogged |
openviking/server/routers/admin.py |
6 | create_account and delete_account have no audit; all 6 are cleanup-phase exceptions |
openviking/server/routers/system.py |
2 | Identity resolution failure at WARNING; it is a fallback |
The problem is not merely "WARNING is used in the wrong places" — the audit capability itself is missing. That is the highest-priority item.
7.4 Implementation inconsistencies
- The default level suppresses
INFO:LogConfig.leveldefaults toWARNING(openviking_cli/utils/config/log_config.py:12) while the fallback in_load_log_config()isINFO(openviking_cli/utils/logger.py:684). Once fallbacks and retries move toINFO, default deployments will not see them at all. - Non-lazy formatting: enabling ruff's
Grules reports 480 issues — 449G004(f-string logging), 27G201(.error(..., exc_info=True)instead of.exception()), and 4G001.pyproject.tomlcurrently selects onlyE/W/F/I/C/B, so CI does not catch them. - Two logging stacks: the codebase is stdlib
loggingplusget_logger(), butopenviking/telemetry/tracer.pystill usesloguru..pr_agent.tomlalso claims a "loguru pattern used throughout the codebase", which is inaccurate. - Per-runtime configuration: Python stdlib, the C++ engine via spdlog (
openviking/utils/agfs_utils.py:216,openviking/storage/vectordb/utils/logging_init.py:58), Rust tracing, and OTel log export each read the samelog.levelwithout a unified validation of level mapping or format conversion. - Duplicate access logs:
openviking.observability.httplogs one completion record per request (openviking/server/request_id.py:86) anduvicorn.accesslogs another; atlog.level=INFOa single request produces two lines. print()remains: 334print(calls in library code, mostly on the startup path inopenviking/server/bootstrap.py, bypassing level control and redaction.- Plain text only:
_build_standard_handler()installs a singlelogging.Formatter(format_string)with no JSON option;request_idis prepended into the message body (openviking_cli/utils/logger.py:489), so collectors cannot split on fields. - No loop-level aggregation:
local_collection.py(18WARNINGcalls) andsemantic_processor.py(11) log inside per-item paths, so a batch failure floods the log.
8. Additional rules beyond the level mapping
The level mapping alone is not enough to land this policy. The following rules cover the gaps.
8.1 Log once
Log once where the failure originates, then raise. An upper layer that wraps the exception must not log it again. A middle layer either logs and returns an error object, or raises untouched — never both.
# Wrong: the lower layer already logged; the upper layer logs the same stack again
except Exception:
logger.exception("compile tag validation failed")
raise OpenVikingError(...) from exc
# Right: raise in the middle, log at the boundary
except (ConfigPatchError, ValueError) as exc:
raise InvalidArgumentError(str(exc)) from exc8.2 Lazy formatting
Use %s placeholders; never f-strings or .format():
# Wrong
logger.warning(f"Failed to resolve identity: {e}")
# Right
logger.info("Failed to resolve identity: %s", e)Three reasons: no formatting cost when the level is disabled; the original template stays available to extra and aggregation tooling; f-strings tend to interpolate sensitive values into exception text.
8.3 Prefer structured fields
Put dimensions that need search, aggregation, or alerting in extra, not only in the message:
logger.info(
"HTTP request completed method=%s route=%s status=%d duration_ms=%.1f",
method, route, status_code, duration_ms,
extra={"http_method": method, "http_route": route,
"http_status_code": status_code, "duration_ms": duration_ms},
)Reserved field names: request_id, trace_id, account_id, user_id, operation, route, duration_ms, result, degraded.
8.4 Sensitive data
Field-level rules live in section 4, which is the single source of truth. One implementation note: use mask_api_key() from openviking/server/api_keys/key_mgr.py when a key must be shown, sanitize_public_http_error() from openviking/observability/http_error_context.py for error context, and sanitize_error() from openviking/service/task_domain.py for exception text. The same applies to audit records.
8.5 Noise control
- Loops and batches: one summary plus
DEBUGdetail, withtotal/failed/skippedin the summary. - De-duplicate repeated warnings within a request: first occurrence
ERROR, subsequent onesDEBUG. - Tune third-party logger levels inside
get_loggerinitialization; do not make callerssetLevelthemselves. - High-frequency paths (per request, per chunk, per candidate) default to
DEBUGand guard withisEnabledFor.
8.6 Record size
Keep a message under 2 KB and structured field values under 1 KB. Truncate and mark truncated=true with original_length=<n>. Never %r a large object or a full exception context.
8.7 Message style
- English only, so cross-team greps and English docs line up.
- Lowercase start, no trailing period, no "Oops" or "Something went wrong".
- Module prefix as
[module], e.g.[ingest],[AUTH],[FeishuAccessor]; the repository already does this, keep it. - Keep templates stable and put only variables in placeholders, otherwise aggregation explodes into discrete values.
8.8 What not to log
- No function enter/exit tracing; use tracing or a profiler.
- No logging for its own sake: every record answers at least one of "what happened", "who is affected", "what to look at next".
- Do not let a log line be the only evidence of a business decision; record important branches in metrics or the event bus.
8.9 Tests
- Assert levels and structured fields with
caplog; do not assert full message text. - Cover audit paths explicitly, at least success and failure.
- Do not mock-assert that a logger was called.
8.10 Cross-runtime consistency
Rust, C++, and TS logging must map levels identically and share the same log.level source. Convert format differences (%l in spdlog vs %(levelname)s in Python) in the initialization layer, not per call site.
9. Rollout
| Priority | Item | Scope | Note |
|---|---|---|---|
| P0 | Add audit events (accounts, membership, roles, ACL, API keys, config, deletion) | New code, no reclassification | Covers the 7.3 gaps; highest value, lowest risk |
| P0 | Demote log_payloads full payloads from WARNING to DEBUG and force redaction |
2 sites (jev_rerank.py, rule_jev.py) |
Enabling the switch currently writes queries and bodies at WARNING, violating 4.6 |
| P0 | ROOT invocation audit: mark in _build_request_context, emit one WARNING on completion |
One choke point | api_key mode only; dev/trusted and internal contexts never trigger, see 3.5 |
| P0 | Decide the default log level | One field plus docs | If WARNING stays, document that fallbacks/retries are invisible by default |
| P1 | Add the openviking.audit logger and a log_audit() helper |
New module | Centralizes fields and dual writes |
| P1 | Enable ruff G rules |
CI | Scope existing violations with per-file-ignores, then burn down 480 |
| P1 | Add log_safety.py (short_hash, safe_uri_for_log, safe_path_for_log, safe_log_fields) |
New module | Single entry point for the transformations in 4.4–4.8 |
| P2 | Demote WARNING per 7.2 |
~190 sites | Batch by module, one regression test per batch |
| P2 | Aggregate loop logging | ~30 files | Summary plus detail |
| P2 | Remediate path logging (34 sites) and indirect peer_id leakage |
~34 sites | Relative path plus short hash |
| P3 | De-duplicate access logs | 2 sites | Decide owner: uvicorn.access or observability.http |
| P3 | print() → get_logger() |
334 sites | Start with server/bootstrap.py |
| P3 | Remove loguru |
1 file | Fix the .pr_agent.toml description too |
| P4 | Optional JSON formatter | 1 handler | For field-based collection |
PR checklist
- Every new log line matches the level semantics in section 1; no fallback or retry at
WARNING. - Security-sensitive actions go through
log_audit(), record both outcomes, and include all fields. - No f-string or
.format()logging. - No logging inside
exceptfollowed by a re-raise (double stack traces). - No keys, tokens, bodies, or full payloads in logs.
- No
peer_id, no full URI embedding a peer segment, no absolute paths (4.4, 4.7). - Endpoints reachable by ROOT are confirmed to hit the 3.5 audit rather than only
INFO. - Walked the security checklist in 4.11.
- Loops and batches aggregate instead of flooding.
- Messages are English, module-prefixed, and template-stable.
10. Quick reference
Audit (permissions/accounts/keys/config/deletion) → WARNING openviking.audit
Failure that needs a human → ERROR
Retries, backoff, fallback, circuit breaker, failback → INFO
4xx rejections (arguments/missing/unauthorized) → INFO
Any ROOT call in api_key mode → WARNING openviking.audit
Business process, intermediates, success detail → DEBUG
Process cannot continue → CRITICAL
Allowed: account_id, user_id, request_id, trace_id, task_*, durations, counts, error codes
Transform: local path → relative path; URI → strip peer segment; exception → type + sanitize_error
Never emit: peer_id, keys and credentials, context content, absolute paths, full payloads
from openviking_cli.utils.logger import get_logger
logger = get_logger(__name__)
logger.debug("retrieval candidates=%d level=%s", len(candidates), level) # process
logger.info("retry operation=embed attempt=%d/%d delay=%.2fs", n, limit, d) # fallback
logger.error("ingest task failed after retries uri=%s", uri, exc_info=True) # needs a human|
Good RFC, can't agree more. |
Uh oh!
There was an error while loading. Please reload this page.
开发者日志输出说明
本文件规定 OpenViking 各运行时(Python、Rust、C++ 引擎、TypeScript)的日志级别语义、内容边界和落地方式。它面向提交代码的开发者,不是给最终用户的部署说明;
log.level、log.format、log.output等配置项仍以配置指南为准。相关入口:
openviking_cli/utils/logger.py(get_logger、reconfigure_logging、OTel 桥接)openviking_cli/utils/config/log_config.pyopenviking/observability/events.py、openviking/observability/usage_audit/1. 级别语义
一条日志应该落在哪一级,只取决于这条记录的作用,不取决于「看起来严不严重」。
CRITICALERRORINFO)、重试中的单次失败、已降级成功、性能偏慢、脏数据跳过WARNINGapi_key模式下 ROOT 身份的任何调用(见 3.5)INFODEBUG补充约定:
ERROR和CRITICAL都不允许在正常控制流里出现。如果一次失败能通过重试、降级或前端提示解决,它就不是ERROR。WARNING是受控级别,语义边界由本文件第 3 节锁定。任何「我觉得值得注意」的记录都不能进WARNING,那是INFO。INFO承载「系统自己扛住了」的信号。兜底、重试、退避、熔断的状态变化必须可观测,但不需要人立刻处理。DEBUG是默认的业务过程出口。写业务代码时,先假设它属于DEBUG,再判断是否命中第 3 节的审计清单或第 5 节的兜底与拒绝清单。INFO:不升级为ERROR(不是故障),也不下沉到DEBUG(否则默认级别下看不到调用方误用)。INFO或DEBUG,但只要是api_key模式下的 ROOT 身份发起,就按WARNING留审计(见 3.5)。dev 与 trusted 模式不计。判定顺序
容易误判的边界
INFOINFO响应日志 + 审计WARNINGapi_key模式下 ROOT 身份调用任意接口(含只读)WARNINGINFO(记录重试)ERROR(只记一条)INFOINFO+ 明确字段标记,不升级为WARNINGINFOERRORINFO(需要长期观测时)或DEBUGINFO,首次ERRORINFOINFO汇总 + 明细DEBUGINFOX-Request-ID非法、已自动重建INFO2. 为什么不是按「严重程度」分级
按严重程度分级会让级别随开发者主观判断漂移:同一类失败,A 觉得是
WARNING,B 觉得是ERROR。本仓库当前就呈现了这个结果——WARNING成了使用最多的级别(见第 7 节)。按作用分级后,级别是可判定的客观属性:
ERROR,可以直接接告警。WARNING,可以直接接审计管道。INFO,可以直接接容量/稳定性看板。DEBUG,只在排障时打开。这样每个级别都能对应一条独立的消费链路,不需要在告警侧再做二次过滤。
3. 审计类日志(
WARNING的唯一合法用途)3.1 必须覆盖的事件
以下动作发生时必须有审计记录。当前仓库在多数位置缺失(见 7.3),新增这些记录属于纯增量改动。
账号与成员
set_role)reset_memory_template一类动作)权限
grant_acl/revoke_acl/delete_acl)密钥与凭证
认证结果
需要注意:这一组是审计记录,与请求返回 4xx 时的
INFO响应日志不是同一条。响应日志记「这次请求被拒了」,审计记录记「谁在什么时候尝试了什么」。两者的 logger、字段、消费者都不同,允许并存;但普通 4xx(参数错误、资源不存在)只记INFO,不进审计。配置与数据
3.2 必填字段
一条审计记录至少包含(字段取值同样受第 4 节约束,
peer_id、密钥、正文不得进入审计):tseventaudit.account.create、audit.acl.grantresultsuccess/failureactor_id/actor_role/actor_typeactor_type取user、admin、root、serviceaccount_idtarget_type/target_iduser、api_key、configactionevent末段一致request_id/trace_idsource_ip/user_agenterror_code3.3 输出形式
审计需要双写,不能只有其中一路:
openviking.observability.events.try_publish_event()发出,由usage_audit投影持久化。这是可查询、可保留期管理的主通道。openviking.audit,级别WARNING。这是给日志采集、SIEM、离线取证用的旁路副本。日志副本的消息格式固定为可切分的键值序列,不要写成散文:
约束:
ERROR,并且不能回滚业务事务。except Exception静默吞掉。3.4 反例
3.5 ROOT 调用
ROOT 是最高权限身份。在
api_key鉴权模式下,任何以 ROOT 身份发生的调用都必须留一条WARNING审计,即使这次调用只读、即使它对业务毫无影响。理由:ROOT 被误用或泄露时,事后唯一能还原的就是轨迹;只读调用同样可能跨租户读取数据,其危害不比写入小。ROOT 调用普通业务接口时本该落
INFO,此时以WARNING为准。触发条件
两个条件同时成立才记录:
api_key(AuthMode.API_KEY);identity.role == Role.ROOT,即持有 root API key(server/api_keys/key_mgr.py:131)。不触发的情况
dev模式(auth/plugins/dev.py:41)trusted模式(auth/plugins/trusted.py:183)/healthcontrol probe),不代表真实操作者log_audit()/health、/ready、/metrics)实现位置
_build_request_context()(server/auth/__init__.py:113)是唯一同时能看到identity.role、认证模式和request.state的收口点。不要把它散进 38 处Role.ROOT引用里(其中 18 处是等值判断)。推荐「入口打标、完成输出」,保证一条记录里既有调用方也有结果:
_auth_mode()已有实现(server/auth/__init__.py:32),不要自己再读一次 config。注意它在拿不到 config 时会回落到api_key,单元测试里要显式注入认证模式,否则测试请求会被当成真实 ROOT。必填字段
event=root.invoke、actor_id、actor_type=root、account_id、http_method、http_route、http_status_code、result、request_id、trace_id、source_ip、user_agent。actor_id写root-key,不要写成default/default。result=failure并带error_code。sensitive=true,便于单独告警。约束
4. 日志安全规则
日志文件、trace 属性、审计存储是三条独立出口,任一条泄露都算泄露。本节按字段规定什么能出来。
4.1 白名单,而不是黑名单
默认拒绝。只有 4.2 表中「允许」一栏的字段可以原样输出;其余字段必须转换(哈希、截断、脱敏、取相对路径),或降级为聚合量(长度、计数、耗时、布尔)。判断不了归属时按禁止处理。
4.2 字段分级
account_id、user_id、request_id、trace_id、span_id、operation、http_method、http_route、http_status_code、duration_ms、role、actor_type、task_id、task_type、task_status、task_stage、重试次数、队列深度、error_codeviking://URI、resource_id、异常消息peer_id、密钥与凭证、上下文内容、绝对路径、完整请求/响应体、模型输入输出任务元信息(
task_id、task_type、状态、阶段、耗时、重试与错误计数、队列深度)可以直接打印,这是排障最常用的维度。任务的meta是例外:只有在确认其值不含 URI、路径、密钥后才可整体打印,否则只打印键名,不打印值。resource_id只允许作为不透明部署标签打印;一旦它来自环境变量且可能是路径或 URL,按 4.7 处理。4.3 已有的注入基线
_get_log_context()目前只注入trace_id、span_id、request_id、operation、account_id、user_id;RootSpanAttributes.to_log_fields()也刻意排除了url_path与url_query。这是本节的既有实现基线,新增注入字段同样受 4.2 约束。但
ObservabilityContext.extra会无条件合并进日志字段,因此extra不是法外之地。4.4
peer_id禁止出现在日志、trace 属性、审计字段中,包括四种间接形式:
viking://user/{user_id}/peers/{peer_id}/...一类 URI 内嵌 peer_id;RequestContext.actor_peer_id;%r、%s、asdict()、model_dump());meta中的资源定位信息。替代写法:
需要跨日志关联同一 peer 时,用不可逆短哈希(
sha256(peer_id)[:8])。哈希只在同一部署内具备关联意义,不要在跨环境分析中复用结论。4.5 密钥与凭证
以下值一律禁止,且禁止任何足以还原的派生形式(前缀、后缀、长度组合):
submission_token、submission_hashAuthorization、签名头、预签名 URLov privacy管理的明文条目唯一例外:诊断输出必须展示 key 时使用
mask_api_key()(仅露末端 4 位),且仅在显式诊断开关下启用。repr=False(RequestContext.api_key、TaskRecord.auth)只挡repr(),挡不住asdict()、model_dump()、__dict__。因此规则是禁止把整个对象塞进日志,而不是依赖repr=False。4.6 上下文内容
「上下文内容」指任何由用户、模型或外部资源产生、且未被等价剪裁的文本:
允许输出的替代量:字符数、token 数、条目数、语言、文件类型、
sha256[:8]、是否命中。现有
log_payloads是反例:models/rerank/jev_rerank.py:191、service/rule_jev.py:76在开关打开时用WARNING打印完整请求与响应 payload,query 与文档正文全部落盘。整改方向:默认关闭(当前已是默认)、开启时降为DEBUG、强制标注payload=true并经过脱敏,且只在排障环境启用。4.7 路径
禁止打印绝对路径。它泄露部署目录、主机用户名,以及
workspace/{account_id}/{user_id}/peers/{peer_id}/这样的目录结构——等于把 peer_id 和租户布局一起暴露。规则:
../、绝对路径或 peers 段;相对路径 + sha256[:8];viking://URI 与本地路径同等对待,peer 段按 4.4 处理;sanitize_error()再输出。当前有 34 处日志直接打印
file_path/path_str/resolved_path一类变量(如parse/parsers/markdown.py:845、parse/parsers/text_encoding.py:74),属于待整改范围。4.8 异常与错误消息
异常字符串经常内嵌触发值(
InvalidArgumentError("bad uri: ...")),所以logger.warning("...: %s", e)本身就是一条泄露通道。type(exc).__name__加经sanitize_error()处理的消息;ERROR且本地排障需要时输出,同样经过脱敏;repr(exc.args)。4.9 三个出口同时成立
log_safety.pyto_log_fields()已排除url_path、url_queryurl.path只在确有需要时保留sanitize_public_http_error()、sanitize_error()、SENSITIVE_TASK_KEYS4.10 建议的实现与检查
openviking/observability/log_safety.py:short_hash()、safe_uri_for_log()、safe_path_for_log()、safe_log_fields()。repr=False只作兜底,不能当防线。peer_id、api_key、token、prompt、content、messages、绝对路径变量)先告警,稳定后阻断。log_payloads路径各补一条失败用例。4.11 安全评审清单
peer_id,也没有打印会内嵌 peer 段的完整 URI。%r/%s/asdict()/model_dump()打印整个对象。extra字段同样合规。5. 兜底、重试与 4xx 拒绝(
INFO)INFO的读者是运维和值班同学:他们需要知道「系统正在自己挣扎」或者「调用方被正常拒绝」,但不需要被叫醒。兜底类记录必须满足三个条件:有状态迁移、有次数、有结果。5.1 重试与退避
INFO,不要用WARNING。ERROR,且只打一条。5.2 降级与兜底
degraded=true一类的标记,让下游能筛。WARNING,否则WARNING会重新变成「需要注意」垃圾桶。5.3 熔断与状态迁移
INFO记录状态机的每一次跃迁:CLOSED → OPEN、OPEN → HALF_OPEN、HALF_OPEN → CLOSED,以及主备切换与回切。仓库里openviking/utils/circuit_breaker.py已经是这个写法,可以当作范例。5.4 启动、关闭、迁移
进程启动/关闭、一次性数据迁移、缓存预热、后台任务的启停,统一
INFO。单次请求的生命周期不要用INFO铺满。5.5 4xx 业务拒绝
客户端参数错误、资源不存在、未授权等 4xx 结果固定记
INFO。理由有三:ERROR,否则告警通道会被调用方误用淹没;DEBUG;INFO噪声。约束:
ERROR,也不写堆栈。WARNING,见 3.1;普通 4xx(参数错误、404)不进审计。6. 业务过程日志(
DEBUG)默认出口。这里放的是成功路径上的业务过程;被拒绝的请求走 5.5 的
INFO。写作要求:"start to process"/"done"这类没有价值。truncated=true)。7. 现状盘点
以下数据取自当前
main(统计范围openviking/、openviking_cli/、openviking_gateway/,不含tests/、benchmark/、bot/)。7.1 级别分布
openviking/openviking_cli/openviking_gateway/WARNING是使用最多的级别,比INFO多 26%,比DEBUG多 81%。这与「WARNING仅限审计」的目标完全相反。7.2
WARNING的实际内容493 条
WARNING中:fail/error/unavailable/invalid/unable/exception等失败语义:约 190 条fallback/retry/degrade/skip/ignor/disabled/missing等兜底语义:约 41 条account/permission/role/acl/api_key/revoke/grant等审计语义:按关键词匹配约 7 条,且这几条都是失败路径(如「受信身份批量注册失败,将重试」),不是审计事件也就是说,
WARNING里约 98% 的内容按新规范应该下沉到INFO或DEBUG,而真正该在WARNING的审计事件基本不存在。典型反例(按新规范应降级):
openviking/utils/model_retry.py:491WARNING,附退避时间INFOopenviking/models/embedder/base.py:422WARNINGINFOopenviking/server/account_stores/redis_mysql.py:485WARNINGINFOopenviking/server/account_stores/file.py:248WARNINGINFOopenviking/storage/queuefs/semantic_processor.py:1357WARNINGINFOopenviking/observability/http_observability_middleware.py:557WARNINGINFO(首次可ERROR)openviking/server/request_id.py:77X-Request-ID非法已重建WARNINGINFO7.3 审计缺口
openviking/server/api_keys/key_mgr.pyopenviking/server/routers/acl.pyopenviking/server/routers/admin.pycreate_account、delete_account无审计;6 条全部是清理阶段的异常openviking/server/routers/system.pyWARNING,属兜底结论:不是「
WARNING用错了地方」这么简单,而是审计能力本身缺失。这是优先级最高的一项。7.4 实现层面的不一致
INFO:LogConfig.level默认WARNING(openviking_cli/utils/config/log_config.py:12),而_load_log_config()的兜底值是INFO(openviking_cli/utils/logger.py:684)。按新规范把兜底/重试放进INFO后,默认部署将完全看不到这些信号。G规则可扫出 480 处,其中G004(f-string 日志)449 处、G201(.error(..., exc_info=True)应为.exception())27 处、G0014 处。当前pyproject.toml只启用了E/W/F/I/C/B,这些问题不会被 CI 拦住。logging+get_logger(),但openviking/telemetry/tracer.py仍用loguru。.pr_agent.toml里也写着「loguru pattern used throughout the codebase」,与实际不符。openviking/utils/agfs_utils.py:216、openviking/storage/vectordb/utils/logging_init.py:58)、Rust tracing、OTel 日志导出各自读同一份log.level,但格式转换与级别映射没有统一校验。openviking.observability.http对每个请求打一条完成日志(openviking/server/request_id.py:86),uvicorn.access也会打一条;当log.level=INFO时同一请求出现两条。print()残留:库代码中 334 处print(,主要集中在openviking/server/bootstrap.py的启动路径,绕过级别控制与脱敏。_build_standard_handler()只挂了logging.Formatter(format_string),没有 JSON 选项;request_id是拼进消息体的(openviking_cli/utils/logger.py:489),采集侧无法按字段切分。local_collection.py(18 条WARNING)、semantic_processor.py(11 条)等在逐条处理场景中直接打日志,批量失败会刷屏。8. 补充规范(原原则之外需要补齐的部分)
原原则只定义了「级别 → 用途」的映射,落地还需要下面这些约定。
8.1 一条日志只记一次
在底层
except里记一次,然后向上抛;上层如果重新包装异常,不要再记一遍。中间层要么记日志并返回错误对象,要么纯抛出,不要既记又抛。8.2 惰性格式化
统一用
%s占位,禁止 f-string 和.format():原因有三:级别未开启时不产生格式化开销;
logging的extra/聚合工具能拿到原始模板;f-string 在异常信息里容易把敏感内容带进日志。8.3 结构化字段优先
需要被检索、聚合、告警的维度放
extra,不要只拼进消息:约定字段名:
request_id、trace_id、account_id、user_id、operation、route、duration_ms、result、degraded。8.4 敏感信息
字段级规则见第 4 节,那里是唯一权威口径。这条只补充一句实现细节:需要展示密钥时用
openviking/server/api_keys/key_mgr.py的mask_api_key(),错误上下文用openviking/observability/http_error_context.py的sanitize_public_http_error()做裁剪与脱敏,异常文本用openviking/service/task_domain.py的sanitize_error()。审计落盘同样适用。8.5 噪声控制
DEBUG,汇总里带total/failed/skipped。ERROR、后续降级为DEBUG。get_logger初始化时统一收敛,不要靠调用方自己setLevel。DEBUG,且先isEnabledFor。8.6 单条日志体积
单条消息建议 ≤ 2 KB,结构化字段值 ≤ 1 KB。超长时截断并标注
truncated=true、original_length=<n>。禁止直接%r一个大对象或整个异常上下文。8.7 文案风格
[模块名],如[ingest]、[AUTH]、[FeishuAccessor];仓库已有此习惯,继续沿用。8.8 不记录的东西
8.9 测试
caplog断言级别与结构化字段,不要断言完整文案。8.10 多运行时一致性
新增 Rust/C++/TS 日志时,级别映射必须与本文一致,并且复用同一个
log.level配置源。格式差异(spdlog 的%l与 Python 的%(levelname)s)在初始化层转换,不要在业务代码里各写一套。9. 落地路线
log_payloads的完整 payload 输出从WARNING降为DEBUG并强制脱敏jev_rerank.py、rule_jev.py)_build_request_context打标 + 完成阶段输出一条WARNINGapi_key模式;dev/trusted 与进程内构造天然不计,见 3.5WARNING默认值,必须在文档中说明「兜底/重试默认不可见」openviking.auditlogger 与log_audit()帮助函数G规则per-file-ignores圈定存量,再逐步清零 480 处log_safety.py(short_hash、safe_uri_for_log、safe_path_for_log、safe_log_fields)WARNING下沉到INFO/DEBUGpeer_id间接泄露uvicorn.access还是observability.http承担print()→get_logger()server/bootstrap.pyloguru残留.pr_agent.toml描述PR 评审清单
提交前自查:
WARNING。log_audit(),成功与失败都记录,字段齐全。.format()日志。except里既记日志又向上抛,造成重复堆栈。peer_id、内嵌 peer 段的完整 URI、绝对路径(第 4.4、4.7 节)。INFO。10. 快速参考
All reactions