Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,9 @@ jobs:
- name: Test
run: swift test

- name: Validate Chronicle contract fixtures
run: python3 scripts/mock_chronicle.py --self-test Tests/Fixtures/chronicle

- name: Lint Swift formatting
run: xcrun swift-format lint --strict --recursive Sources Tests Package.swift

Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/package-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ jobs:
- name: Record checksums
run: |
set -euo pipefail
shasum -a 256 "dist/EvalOps agentd.app/Contents/MacOS/agentd" "dist/agentd.zip" | tee dist/SHA256SUMS
shasum -a 256 "dist/EvalOps agentd.app/Contents/MacOS/agentd" "dist/agentd.zip" "dist/update-channel.json" | tee dist/SHA256SUMS
codesign -dvvv --entitlements :- "dist/EvalOps agentd.app" > dist/codesign.txt 2>&1
spctl -a -t exec -vv "dist/EvalOps agentd.app" > dist/spctl.txt 2>&1

Expand All @@ -110,6 +110,7 @@ jobs:
dist/EvalOps agentd.app
dist/agentd.zip
dist/SHA256SUMS
dist/update-channel.json
dist/codesign.txt
dist/spctl.txt
if-no-files-found: error
36 changes: 29 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ This is the desktop component of the work tracked in

- Captures the active display via `ScreenCaptureKit` at an adaptive 0.2–1 fps;
input idle time drops cadence to `idleFps` and activity restores
`captureFps`.
`captureFps`. Frames include display id, scale, and main-display metadata for
multi-display diagnostics.
- Reads `(bundleId, windowTitle, documentPath)` per frame via the Accessibility
API and `NSWorkspace`.
- Runs Apple Vision OCR on-device.
Expand All @@ -22,6 +23,9 @@ This is the desktop component of the work tracked in
Stripe markers — match → frame dropped, never partial-redacted.
- Per-app allow/deny list and per-path deny list.
- Window-title pause patterns (Zoom, FaceTime, 1Password…).
- Scheduled pause windows from managed policy pause capture for meetings,
interviews, private/focus blocks, or Platform-driven policy windows; manual
pause always wins over automatic resume.
- Secret scanning covers OCR text, window titles, and document paths before a
frame is batched.
- OCR text is scrubbed at full length, then capped to `maxOcrTextChars`
Expand All @@ -44,14 +48,15 @@ This is the desktop component of the work tracked in
without requiring an app restart. Local hard-deny safety rails remain
fail-closed even when a remote policy allows a bundle or path.
- Menu-bar UI: pause/resume (`⌃⌥⌘P`), flush now (`⌃⌥⌘F`), reveal batches dir,
quit.
diagnostics report (`⌃⌥⌘D`), delete queued batches, launch-at-login, quit.

## Build

```
swift build
swift run agentd # foreground; menu-bar item appears
swift test
python3 scripts/mock_chronicle.py --self-test Tests/Fixtures/chronicle
scripts/package_app.sh # release .app bundle with hardened runtime signing
scripts/permission_smoke.sh --no-launch # generate permission-smoke evidence template
```
Expand Down Expand Up @@ -87,6 +92,9 @@ version/checksum/codesign evidence in `dist/permission-smoke-report.md`, and
opens the app unless `--no-launch` is supplied. Use it for the hardware-backed
Screen Recording and Accessibility permission smoke.

Launch-at-login uses native `SMAppService.mainApp` from the menu bar; agentd
does not install LaunchAgent plists.

## Configuration

agentd reads and writes `~/.evalops/agentd/config.json`. Important defaults:
Expand All @@ -95,6 +103,9 @@ agentd reads and writes `~/.evalops/agentd/config.json`. Important defaults:
- `captureFps: 1.0`
- `idleFps: 0.2`
- `idleThresholdSeconds: 60`
- `adaptiveOcrMinChars: 1024`
- `adaptiveOcrBackpressureThreshold: 8`
- `adaptiveOcrBacklogBytes: 67108864`
- `batchIntervalSeconds: 30`
- `maxFramesPerBatch: 24`
- `maxOcrTextChars: 4096`
Expand Down Expand Up @@ -166,23 +177,31 @@ Secret Broker mode, and local permission preflight metadata. Every 30 seconds it
calls `Heartbeat` with pending in-memory frames plus local fallback batch count
and bytes. `RegisterDevice` and `Heartbeat` responses may include a
`CapturePolicy`; agentd applies allowlist, denylist, path-deny, pause-window,
batch interval, and max-frame settings at runtime. Server `PAUSED` capture mode
stops capture until a later policy resumes it, while manual user pause still
wins locally.
scheduled pause windows, batch interval, and max-frame settings at runtime.
Server `PAUSED` capture mode stops capture until a later policy resumes it.
Manual user pause wins over scheduled pause, and scheduled pause wins over
server policy pause for visible menu/diagnostic state.

Encrypted local batches use the `.agentdbatch` extension. The encryption key is
created or loaded from Keychain service `dev.evalops.agentd.local-batch-key`,
accounted by `deviceId`, and is never written to `config.json` or the batch
directory. Retention sweeps apply to both plaintext `.json` batches and
encrypted `.agentdbatch` batches.

Diagnostics reports are written under `~/.evalops/agentd/diagnostics/` with
`0o600` permissions. They summarize permissions, policy, queue pressure, local
batches, and last submit health without OCR text or raw payloads.

`scripts/mock_chronicle.py` provides a strict local mock Chronicle and Secret
Broker harness. CI validates the golden fixtures in `Tests/Fixtures/chronicle`
so request-shape drift is explicit until generated `chronicle.v1` Swift types
are available.

## What's next

- Consume generated `chronicle.v1` Swift types when the platform SDK publishes
them
([evalops/platform#1078](https://github.com/evalops/platform/issues/1078)).
- Calendar / Zoom auto-pause via NATS subject
`chronicle.policy.pause` (siphon-fed).
- Hardware-backed permission-flow smoke test for Screen Recording and
Accessibility prompts.

Expand All @@ -192,6 +211,9 @@ encrypted `.agentdbatch` batches.
Sources/agentd/
main.swift # NSApplication + AppController boot
ChronicleControl.swift # RegisterDevice/Heartbeat + policy response client
Diagnostics.swift # Redacted local report generation
PauseState.swift # Manual/scheduled/policy pause precedence
LaunchAtLoginController.swift # Native SMAppService login item toggle
Config.swift # ~/.evalops/agentd/config.json
CaptureService.swift # SCStream pipeline
WindowContext.swift # AX + NSWorkspace probe
Expand Down
29 changes: 27 additions & 2 deletions Sources/agentd/CaptureService.swift
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ struct CapturedFrame: Sendable {
let timestamp: Date
let cgImage: CGImage
let displayId: CGDirectDisplayID
let displayScale: Double?
let mainDisplay: Bool
}

actor CaptureService: NSObject {
Expand Down Expand Up @@ -55,6 +57,8 @@ actor CaptureService: NSObject {
let output = FrameOutput(
ciContext: ciContext,
displayId: display.displayID,
displayScale: Self.displayScale(display.displayID),
mainDisplay: display.displayID == CGMainDisplayID(),
onFrame: onFrame,
onFrameDropped: onFrameDropped
)
Expand Down Expand Up @@ -91,20 +95,35 @@ actor CaptureService: NSObject {
"capture fps update failed: \(error.localizedDescription, privacy: .public)")
}
}

private static func displayScale(_ displayId: CGDirectDisplayID) -> Double? {
let pixelWidth = CGDisplayPixelsWide(displayId)
guard pixelWidth > 0 else { return nil }
let boundsWidth = CGDisplayBounds(displayId).width
guard boundsWidth > 0 else { return nil }
return Double(pixelWidth) / Double(boundsWidth)
}
}

private final class FrameOutput: NSObject, SCStreamOutput, @unchecked Sendable {
let ciContext: CIContext
let displayId: CGDirectDisplayID
let displayScale: Double?
let mainDisplay: Bool
let dispatcher: BufferedFrameDispatcher

init(
ciContext: CIContext, displayId: CGDirectDisplayID,
ciContext: CIContext,
displayId: CGDirectDisplayID,
displayScale: Double?,
mainDisplay: Bool,
onFrame: @escaping @Sendable (CapturedFrame) async -> Void,
onFrameDropped: @escaping @Sendable () async -> Void
) {
self.ciContext = ciContext
self.displayId = displayId
self.displayScale = displayScale
self.mainDisplay = mainDisplay
self.dispatcher = BufferedFrameDispatcher(onFrame: onFrame, onDropped: onFrameDropped)
}

Expand All @@ -117,7 +136,13 @@ private final class FrameOutput: NSObject, SCStreamOutput, @unchecked Sendable {
else { return }
let ci = CIImage(cvPixelBuffer: pixelBuffer)
guard let cg = ciContext.createCGImage(ci, from: ci.extent) else { return }
let frame = CapturedFrame(timestamp: Date(), cgImage: cg, displayId: displayId)
let frame = CapturedFrame(
timestamp: Date(),
cgImage: cg,
displayId: displayId,
displayScale: displayScale,
mainDisplay: mainDisplay
)
dispatcher.yield(frame)
}
}
Expand Down
42 changes: 41 additions & 1 deletion Sources/agentd/ChronicleControl.swift
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ struct CapturePolicy: Sendable, Codable, Equatable {
var cloudConsolidationTier: String
var minBatchIntervalSeconds: Int
var maxFramesPerBatch: Int
var scheduledPauseWindows: [ScheduledPauseWindow]
var sourcePolicyRef: String

init(
Expand All @@ -45,6 +46,7 @@ struct CapturePolicy: Sendable, Codable, Equatable {
cloudConsolidationTier: String = "",
minBatchIntervalSeconds: Int = 0,
maxFramesPerBatch: Int = 0,
scheduledPauseWindows: [ScheduledPauseWindow] = [],
sourcePolicyRef: String = ""
) {
self.policyVersion = policyVersion
Expand All @@ -57,6 +59,7 @@ struct CapturePolicy: Sendable, Codable, Equatable {
self.cloudConsolidationTier = cloudConsolidationTier
self.minBatchIntervalSeconds = minBatchIntervalSeconds
self.maxFramesPerBatch = maxFramesPerBatch
self.scheduledPauseWindows = scheduledPauseWindows
self.sourcePolicyRef = sourcePolicyRef
}

Expand All @@ -71,6 +74,7 @@ struct CapturePolicy: Sendable, Codable, Equatable {
case cloudConsolidationTier
case minBatchIntervalSeconds
case maxFramesPerBatch
case scheduledPauseWindows
case sourcePolicyRef
}

Expand Down Expand Up @@ -100,11 +104,22 @@ struct CapturePolicy: Sendable, Codable, Equatable {
forKey: .minBatchIntervalSeconds
) ?? 0,
maxFramesPerBatch: try container.decodeIfPresent(Int.self, forKey: .maxFramesPerBatch) ?? 0,
scheduledPauseWindows: try container.decodeIfPresent(
[ScheduledPauseWindow].self,
forKey: .scheduledPauseWindows
) ?? [],
sourcePolicyRef: try container.decodeIfPresent(String.self, forKey: .sourcePolicyRef) ?? ""
)
}
}

struct ScheduledPauseWindow: Sendable, Codable, Equatable {
var id: String
var reason: String
var startsAt: Date
var endsAt: Date
}

struct ChronicleDevice: Sendable, Codable, Equatable {
var deviceId: String
var organizationId: String
Expand Down Expand Up @@ -162,12 +177,32 @@ struct HeartbeatRequest: Sendable, Codable, Equatable {
var organizationId: String
var pendingFrameCount: Int
var pendingBytes: Int64
var paused: Bool
var pauseReason: String?

enum CodingKeys: String, CodingKey {
case deviceId
case organizationId
case pendingFrameCount
case pendingBytes
case paused
case pauseReason
}

init(
deviceId: String,
organizationId: String,
pendingFrameCount: Int,
pendingBytes: Int64,
paused: Bool = false,
pauseReason: String? = nil
) {
self.deviceId = deviceId
self.organizationId = organizationId
self.pendingFrameCount = pendingFrameCount
self.pendingBytes = pendingBytes
self.paused = paused
self.pauseReason = pauseReason
}

func encode(to encoder: Encoder) throws {
Expand All @@ -176,6 +211,8 @@ struct HeartbeatRequest: Sendable, Codable, Equatable {
try container.encode(organizationId, forKey: .organizationId)
try container.encode(pendingFrameCount, forKey: .pendingFrameCount)
try container.encode(String(pendingBytes), forKey: .pendingBytes)
try container.encode(paused, forKey: .paused)
try container.encodeIfPresent(pauseReason, forKey: .pauseReason)
}
}

Expand Down Expand Up @@ -283,7 +320,9 @@ actor ChronicleControlClient {
if responseBody.isEmpty {
return try JSONDecoder().decode(responseType, from: Data("{}".utf8))
}
return try JSONDecoder().decode(responseType, from: responseBody)
let decoder = JSONDecoder()
decoder.dateDecodingStrategy = .iso8601
return try decoder.decode(responseType, from: responseBody)
}
}

Expand Down Expand Up @@ -313,6 +352,7 @@ enum ChronicleControlError: Error, LocalizedError, Equatable {

func encodeChronicleControlRequest<T: Encodable>(_ request: T) throws -> Data {
let enc = JSONEncoder()
enc.dateEncodingStrategy = .iso8601
enc.outputFormatting = [.sortedKeys]
return try enc.encode(request)
}
Loading