feat: add comprehensive logging across scan pipeline#2
Closed
Conversation
Added structured logging to improve observability and debugging: - MalwareBazaar: hash lookups, API responses, matches - Configuration: signature/YARA rule loading with counts - YARA scanning: rule matches with severity and paths - Profile building: metadata extraction and loader detection - Scan pipeline: stage transitions, verdicts, overrides - HTTP endpoints: upload/scan request tracking All logging uses structured fields (upload_id, scan_id, etc.) for consistent querying and analysis. Nightshift-Task: logging-audit Nightshift-Ref: https://github.com/marcus/nightshift
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Added structured logging throughout the Jarspect codebase to significantly improve observability and debugging capabilities. This audit identified critical gaps in logging across external services, configuration loading, and scan pipeline stages.
Changes
MalwareBazaar integration (
src/malwarebazaar.rs)Configuration loading (
src/lib.rs)YARA scanning (
src/analysis/yara.rs)Profile building (
src/profile.rs)Scan pipeline (
src/scan.rs)HTTP endpoints (
src/main.rs)Structured Fields
All logging uses consistent structured fields for queryability:
upload_id: Upload identifierscan_id: Scan operation identifiersha256: File hashjar_size_bytes: File sizefile_count,class_count: Archive statisticsresult,confidence,risk_score: Verdict detailsduration_ms: Operation timingfamily,tags_count: MalwareBazaar metadatarule_id,severity,pack: YARA rule detailsLog Level Guidelines
info: High-level operations, external service calls, scan outcomeswarn: Retryable failures with backoff, graceful degradationdebug: Detailed progress, individual detector/YARA runstrace: Very detailed diagnostics (future use)Documentation
Created
LOGGING_AUDIT.mddocumenting: