Skip to content

feat: MBA-44 LLM trace 경계 및 표시 보강 - #66

Merged
HyungminYoon1 merged 11 commits into
devfrom
feature/mba-44
Jun 27, 2026
Merged

feat: MBA-44 LLM trace 경계 및 표시 보강#66
HyungminYoon1 merged 11 commits into
devfrom
feature/mba-44

Conversation

@HyungminYoon1

Copy link
Copy Markdown
Contributor

변경 사항

  • workflow run LLM trace 조회 API에 workflow read guard와 whitelist 응답 경계를 적용했습니다.
  • LLM usage logging에서 workflow/run context를 공통 helper로 검증하고, invalid/missing/mismatch context는 저장 전에 차단했습니다.
  • legacy workflow의 organization_id 결손은 organization_id=NULL로 usage 저장을 허용하도록 호환성을 유지했습니다.
  • llm_usage_logs.latency_ms column명을 ORM/DB/schema/document에서 일관되게 정리하고, 기존 오타 column을 rename하는 migration을 추가했습니다.
  • permission.denied audit metadata에 policy/resource/required permission context를 보강했습니다.
  • workflow run detail의 LLM token analysis UI를 trace endpoint 기준으로 연결하고, 0-token failed trace와 0ms latency 표시를 보완했습니다.
  • 관련 정책 결정 로그와 API/data model 문서를 갱신했습니다.

관련 이슈

Closes #59

변경 유형

  • 버그 수정
  • 새로운 기능
  • 리팩토링
  • 문서 수정
  • 기타

테스트

  • 로컬에서 테스트 완료
  • 기존 테스트 통과 확인

검증한 명령:

  • python -m py_compile apps/shared/services/llm_usage_context.py apps/gateway/services/llm_service.py apps/workflow_engine/services/llm_service.py apps/shared/schemas/llm.py apps/shared/db/models/llm.py apps/shared/alembic/versions/f8a9b0c1d2e3_rename_llm_usage_latency_ms.py
  • python -m pytest apps/gateway/tests/api/test_permission_helpers.py apps/gateway/tests/api/test_workflow_llm_traces_api.py apps/gateway/tests/services/test_llm_usage_log_context.py apps/shared/tests/test_llm_usage_schema.py apps/shared/tests/test_audit_actions.py apps/log_system/tests/test_workflow_execute_audit.py -q
  • python -m pytest apps/gateway/tests/services/test_team_service_permissions.py apps/gateway/tests/services/test_llm_service_permissions.py -q
  • npm test -- LogTokenAnalysis.test.tsx --run
  • npx eslint app/features/workflow/components/logs/detail-components/LogTokenAnalysis.tsx app/features/workflow/components/logs/detail-components/LogTokenAnalysis.test.tsx
  • rg '\batency_ms\b' apps docs -S
  • git diff --check

참고: npx tsc --noEmit은 기존 app/features/workflow/store/useWorkflowStore.test.ts 타입 오류 3건으로 실패하며, 이번 변경 파일에서는 새 타입 오류가 확인되지 않았습니다.

스크린샷 (UI 변경 시)

  • 별도 스크린샷 없음. Component test로 LLM trace 우선 표시, loading, 0-token failed trace, 노드별 bar width 기준을 검증했습니다.

@linear-code

linear-code Bot commented Jun 27, 2026

Copy link
Copy Markdown

MBA-44

@yoonki1207

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 767f631d9e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +117 to +119
setLlmTraceLoading(true);
setLlmTraceError(false);
const response = await workflowApi.getWorkflowRunLlmTraces(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reset trace state when switching runs

When the user opens another run after viewing one that had LLM traces, this effect starts the new request but leaves the previous llmTraces array in state until the request resolves. Because the summary immediately treats any non-empty llmTraces as authoritative, the new run can temporarily show the previous run's token totals, model breakdown, and costs; on slow trace requests this displays misleading observability data. Clear the trace state at the start of the effect or associate trace rows with the run id before using them.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 fixed

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7e08bfb. LogDetail now clears llmTraces at the start of each run-specific trace request, and LogDetail.test.tsx covers switching runs while the next trace request is still pending so stale trace totals are not displayed.

@HyungminYoon1
HyungminYoon1 merged commit cde421f into dev Jun 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BE][Data][FIX] Audit/LLM trace API foundation

2 participants