fix: use events.jsonl from copilot session-state for log parsing#24028
fix: use events.jsonl from copilot session-state for log parsing#24028
Conversation
- log_parser_bootstrap.cjs: recursively find events.jsonl in directory; use it when found (preferred structured format), fall back to .log/.txt - logs_parsing_core.go (findAgentLogFile): prefer events.jsonl over debug .log files in both the flattened directory walk and recursive fallback; stop walking immediately when events.jsonl is found - logs_filtering_test.go: add tests for events.jsonl discovery and priority Agent-Logs-Url: https://github.com/github/gh-aw/sessions/9c5a58b8-3cc3-4712-8e2f-d4c24ba3213b Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates Copilot log discovery/parsing to prefer the structured events.jsonl session-state output (copied into artifacts) so gh aw logs and the JS parser bootstrap can correctly parse Copilot runs.
Changes:
- JS log parser bootstrap: recursively discovers and reads
events.jsonlwhen directory inputs are supported, otherwise falls back to concatenating.log/.txt. - Go log discovery (
findAgentLogFile): prefersevents.jsonlover Copilot debug.logfiles in both flattened and recursive search paths. - Adds Go unit tests covering
events.jsonldiscovery incopilot-session-state/<uuid>/and prioritization overprocess-*.log.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pkg/cli/logs_parsing_core.go | Prefer events.jsonl when locating Copilot logs (flattened + recursive fallback). |
| pkg/cli/logs_filtering_test.go | Adds tests validating events.jsonl discovery and precedence. |
| actions/setup/js/log_parser_bootstrap.cjs | Adds recursive events.jsonl discovery and uses it as primary parsing input for directory-based Copilot logs. |
Comments suppressed due to low confidence (1)
pkg/cli/logs_parsing_core.go:223
- Same issue as the flattened-dir walk: the recursive fallback returns the first
events.jsonlencountered. With multiple session UUID directories present undercopilot-session-state/, this can pick an older session and ignore the intended one. Prefer choosing the newest/most-relevantevents.jsonlrather than early-exiting on the first match.
// Fallback: search recursively in logDir for events.jsonl, session*.log or process*.log files
// This handles cases where the artifact structure is different than expected
// Note: Copilot changed from session-*.log to process-*.log naming convention
logsParsingCoreLog.Printf("Searching recursively in %s for events.jsonl, session*.log or process*.log files", logDir)
var foundEventsJsonl, foundLogFile string
_ = filepath.Walk(logDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info == nil {
return nil
}
fileName := info.Name()
if !info.IsDir() {
if fileName == "events.jsonl" && foundEventsJsonl == "" {
foundEventsJsonl = path
logsParsingCoreLog.Printf("Found events.jsonl via recursive search: %s", path)
return errors.New("stop") // sentinel to stop walking early
} else if (strings.HasPrefix(fileName, "session") || strings.HasPrefix(fileName, "process")) && strings.HasSuffix(fileName, ".log") && foundLogFile == "" {
foundLogFile = path
logsParsingCoreLog.Printf("Found Copilot log file via recursive search: %s", path)
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // Prefer events.jsonl (structured Copilot session format) over debug .log files. | ||
| // Walk the full tree: stop immediately when events.jsonl is found (preferred), | ||
| // but keep walking after a .log match in case events.jsonl appears later. | ||
| var foundEventsJsonl, foundLogFile string | ||
| _ = filepath.Walk(flattenedDir, func(path string, info os.FileInfo, err error) error { | ||
| if err != nil || info == nil { | ||
| return nil | ||
| } | ||
| if !info.IsDir() && strings.HasSuffix(info.Name(), ".log") && foundFile == "" { | ||
| foundFile = path | ||
| logsParsingCoreLog.Printf("Found session log file: %s", path) | ||
| return errors.New("stop") // sentinel to stop walking early | ||
| if !info.IsDir() { | ||
| if info.Name() == "events.jsonl" && foundEventsJsonl == "" { | ||
| foundEventsJsonl = path | ||
| logsParsingCoreLog.Printf("Found events.jsonl file: %s", path) | ||
| return errors.New("stop") // sentinel to stop walking early | ||
| } else if strings.HasSuffix(info.Name(), ".log") && foundLogFile == "" { |
There was a problem hiding this comment.
findAgentLogFile returns the first events.jsonl hit by filepath.Walk. Because copy_copilot_session_state.sh copies the entire ~/.copilot/session-state/ tree (potentially multiple UUID subdirs), this can select a stale session deterministically (lexicographically smallest path) rather than the most recent session, causing the wrong run to be parsed. Consider selecting the best candidate (e.g., newest events.jsonl by ModTime, or newest UUID dir) instead of returning on the first match.
This issue also appears on line 205 of the same file.
See below for a potential fix:
// Walk the full tree and select the newest candidate by modification time,
// preferring events.jsonl over .log files.
var bestEventsJsonl string
var bestEventsModTime int64
var bestLogFile string
var bestLogModTime int64
_ = filepath.Walk(flattenedDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info == nil {
return nil
}
if !info.IsDir() {
modTime := info.ModTime().UnixNano()
if info.Name() == "events.jsonl" {
if bestEventsJsonl == "" || modTime > bestEventsModTime {
bestEventsJsonl = path
bestEventsModTime = modTime
logsParsingCoreLog.Printf("Found events.jsonl file candidate: %s", path)
}
} else if strings.HasSuffix(info.Name(), ".log") {
if bestLogFile == "" || modTime > bestLogModTime {
bestLogFile = path
bestLogModTime = modTime
logsParsingCoreLog.Printf("Found session log file candidate: %s", path)
}
}
}
return nil
})
if bestEventsJsonl != "" {
return bestEventsJsonl, true
}
if bestLogFile != "" {
return bestLogFile, true
| * Recursively searches a directory tree for the first events.jsonl file. | ||
| * This file is written by the Copilot CLI and contains structured session events. | ||
| * @param {string} dirPath - Directory to search | ||
| * @returns {string|null} Absolute path to events.jsonl, or null if not found | ||
| */ | ||
| function findEventsJsonlRecursive(dirPath) { | ||
| try { | ||
| const entries = fs.readdirSync(dirPath, { withFileTypes: true }); | ||
| for (const entry of entries) { | ||
| const fullPath = path.join(dirPath, entry.name); | ||
| if (entry.isDirectory()) { | ||
| const found = findEventsJsonlRecursive(fullPath); | ||
| if (found) return found; | ||
| } else if (entry.name === "events.jsonl") { | ||
| return fullPath; | ||
| } | ||
| } | ||
| } catch (e) { | ||
| // Ignore read errors (e.g. permission denied on subdirectories) | ||
| } | ||
| return null; |
There was a problem hiding this comment.
findEventsJsonlRecursive returns the first events.jsonl found depth-first. Since the Copilot CLI session-state directory can contain multiple UUID subdirectories (and copy_copilot_session_state.sh copies them all), this can select an older session depending on directory ordering. Consider scanning all candidates and picking the newest events.jsonl (by mtime) or otherwise disambiguating which session should be parsed.
| * Recursively searches a directory tree for the first events.jsonl file. | |
| * This file is written by the Copilot CLI and contains structured session events. | |
| * @param {string} dirPath - Directory to search | |
| * @returns {string|null} Absolute path to events.jsonl, or null if not found | |
| */ | |
| function findEventsJsonlRecursive(dirPath) { | |
| try { | |
| const entries = fs.readdirSync(dirPath, { withFileTypes: true }); | |
| for (const entry of entries) { | |
| const fullPath = path.join(dirPath, entry.name); | |
| if (entry.isDirectory()) { | |
| const found = findEventsJsonlRecursive(fullPath); | |
| if (found) return found; | |
| } else if (entry.name === "events.jsonl") { | |
| return fullPath; | |
| } | |
| } | |
| } catch (e) { | |
| // Ignore read errors (e.g. permission denied on subdirectories) | |
| } | |
| return null; | |
| * Recursively searches a directory tree for the newest events.jsonl file. | |
| * This file is written by the Copilot CLI and contains structured session events. | |
| * @param {string} dirPath - Directory to search | |
| * @returns {string|null} Absolute path to the newest events.jsonl, or null if not found | |
| */ | |
| function findEventsJsonlRecursive(dirPath) { | |
| /** @type {string|null} */ | |
| let newestPath = null; | |
| let newestMtimeMs = -1; | |
| /** | |
| * @param {string} currentDir | |
| */ | |
| function walk(currentDir) { | |
| try { | |
| const entries = fs.readdirSync(currentDir, { withFileTypes: true }); | |
| for (const entry of entries) { | |
| const fullPath = path.join(currentDir, entry.name); | |
| if (entry.isDirectory()) { | |
| walk(fullPath); | |
| } else if (entry.name === "events.jsonl") { | |
| try { | |
| const stat = fs.statSync(fullPath); | |
| const mtimeMs = stat.mtimeMs; | |
| if (mtimeMs > newestMtimeMs) { | |
| newestMtimeMs = mtimeMs; | |
| newestPath = fullPath; | |
| } | |
| } catch (e) { | |
| // Ignore stat errors for individual files | |
| } | |
| } | |
| } | |
| } catch (e) { | |
| // Ignore read errors (e.g. permission denied on subdirectories) | |
| } | |
| } | |
| walk(dirPath); | |
| return newestPath; |
| if !strings.HasSuffix(found, "events.jsonl") { | ||
| t.Errorf("Expected events.jsonl to be preferred over .log, but found %s", found) |
There was a problem hiding this comment.
This subtest only asserts that the chosen file ends with events.jsonl. To make the preference behavior robust (and avoid passing if some other events.jsonl is introduced in the fixture), assert the exact expected path (found == eventsJsonl) like the previous subtest does.
| if !strings.HasSuffix(found, "events.jsonl") { | |
| t.Errorf("Expected events.jsonl to be preferred over .log, but found %s", found) | |
| if found != eventsJsonl { | |
| t.Errorf("Expected events.jsonl (%s) to be preferred over .log, but found %s", eventsJsonl, found) |
The Copilot CLI writes structured session events to
~/.copilot/session-state/<uuid>/events.jsonl, copied to/tmp/gh-aw/sandbox/agent/logs/copilot-session-state/<uuid>/events.jsonlbycopy_copilot_session_state.sh. Neither the JS parser bootstrap norgh aw logswere picking it up — the bootstrap only read top-level.log/.txtfiles, andfindAgentLogFileonly matchedsession*.log/process*.logpatterns.Changes
log_parser_bootstrap.cjs: AddedfindEventsJsonlRecursive()to walk the directory tree forevents.jsonl. WhensupportsDirectories=true, use it as primary content if found; fall back to.log/.txtconcatenation otherwise.logs_parsing_core.go(findAgentLogFile): Both the flattened-dir walker and the recursive fallback now findevents.jsonland prefer it over debug.logfiles. Walk stops immediately onevents.jsonlhit (early exit sentinel preserved).logs_filtering_test.go: Two new sub-tests —events.jsonldiscovered incopilot-session-state/<uuid>/subdirectory, andevents.jsonltakes priority when aprocess-*.logalso exists.