Skip to content

fix: use events.jsonl from copilot session-state for log parsing#24028

Merged
pelikhan merged 1 commit intomainfrom
copilot/update-js-parser-logs
Apr 2, 2026
Merged

fix: use events.jsonl from copilot session-state for log parsing#24028
pelikhan merged 1 commit intomainfrom
copilot/update-js-parser-logs

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 2, 2026

The Copilot CLI writes structured session events to ~/.copilot/session-state/<uuid>/events.jsonl, copied to /tmp/gh-aw/sandbox/agent/logs/copilot-session-state/<uuid>/events.jsonl by copy_copilot_session_state.sh. Neither the JS parser bootstrap nor gh aw logs were picking it up — the bootstrap only read top-level .log/.txt files, and findAgentLogFile only matched session*.log/process*.log patterns.

Changes

  • log_parser_bootstrap.cjs: Added findEventsJsonlRecursive() to walk the directory tree for events.jsonl. When supportsDirectories=true, use it as primary content if found; fall back to .log/.txt concatenation otherwise.

  • logs_parsing_core.go (findAgentLogFile): Both the flattened-dir walker and the recursive fallback now find events.jsonl and prefer it over debug .log files. Walk stops immediately on events.jsonl hit (early exit sentinel preserved).

  • logs_filtering_test.go: Two new sub-tests — events.jsonl discovered in copilot-session-state/<uuid>/ subdirectory, and events.jsonl takes priority when a process-*.log also exists.

- log_parser_bootstrap.cjs: recursively find events.jsonl in directory;
  use it when found (preferred structured format), fall back to .log/.txt
- logs_parsing_core.go (findAgentLogFile): prefer events.jsonl over debug
  .log files in both the flattened directory walk and recursive fallback;
  stop walking immediately when events.jsonl is found
- logs_filtering_test.go: add tests for events.jsonl discovery and priority

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/9c5a58b8-3cc3-4712-8e2f-d4c24ba3213b

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review April 2, 2026 05:35
Copilot AI review requested due to automatic review settings April 2, 2026 05:35
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Copilot log discovery/parsing to prefer the structured events.jsonl session-state output (copied into artifacts) so gh aw logs and the JS parser bootstrap can correctly parse Copilot runs.

Changes:

  • JS log parser bootstrap: recursively discovers and reads events.jsonl when directory inputs are supported, otherwise falls back to concatenating .log/.txt.
  • Go log discovery (findAgentLogFile): prefers events.jsonl over Copilot debug .log files in both flattened and recursive search paths.
  • Adds Go unit tests covering events.jsonl discovery in copilot-session-state/<uuid>/ and prioritization over process-*.log.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
pkg/cli/logs_parsing_core.go Prefer events.jsonl when locating Copilot logs (flattened + recursive fallback).
pkg/cli/logs_filtering_test.go Adds tests validating events.jsonl discovery and precedence.
actions/setup/js/log_parser_bootstrap.cjs Adds recursive events.jsonl discovery and uses it as primary parsing input for directory-based Copilot logs.
Comments suppressed due to low confidence (1)

pkg/cli/logs_parsing_core.go:223

  • Same issue as the flattened-dir walk: the recursive fallback returns the first events.jsonl encountered. With multiple session UUID directories present under copilot-session-state/, this can pick an older session and ignore the intended one. Prefer choosing the newest/most-relevant events.jsonl rather than early-exiting on the first match.
		// Fallback: search recursively in logDir for events.jsonl, session*.log or process*.log files
		// This handles cases where the artifact structure is different than expected
		// Note: Copilot changed from session-*.log to process-*.log naming convention
		logsParsingCoreLog.Printf("Searching recursively in %s for events.jsonl, session*.log or process*.log files", logDir)
		var foundEventsJsonl, foundLogFile string
		_ = filepath.Walk(logDir, func(path string, info os.FileInfo, err error) error {
			if err != nil || info == nil {
				return nil
			}
			fileName := info.Name()
			if !info.IsDir() {
				if fileName == "events.jsonl" && foundEventsJsonl == "" {
					foundEventsJsonl = path
					logsParsingCoreLog.Printf("Found events.jsonl via recursive search: %s", path)
					return errors.New("stop") // sentinel to stop walking early
				} else if (strings.HasPrefix(fileName, "session") || strings.HasPrefix(fileName, "process")) && strings.HasSuffix(fileName, ".log") && foundLogFile == "" {
					foundLogFile = path
					logsParsingCoreLog.Printf("Found Copilot log file via recursive search: %s", path)
				}

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +176 to +189
// Prefer events.jsonl (structured Copilot session format) over debug .log files.
// Walk the full tree: stop immediately when events.jsonl is found (preferred),
// but keep walking after a .log match in case events.jsonl appears later.
var foundEventsJsonl, foundLogFile string
_ = filepath.Walk(flattenedDir, func(path string, info os.FileInfo, err error) error {
if err != nil || info == nil {
return nil
}
if !info.IsDir() && strings.HasSuffix(info.Name(), ".log") && foundFile == "" {
foundFile = path
logsParsingCoreLog.Printf("Found session log file: %s", path)
return errors.New("stop") // sentinel to stop walking early
if !info.IsDir() {
if info.Name() == "events.jsonl" && foundEventsJsonl == "" {
foundEventsJsonl = path
logsParsingCoreLog.Printf("Found events.jsonl file: %s", path)
return errors.New("stop") // sentinel to stop walking early
} else if strings.HasSuffix(info.Name(), ".log") && foundLogFile == "" {
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

findAgentLogFile returns the first events.jsonl hit by filepath.Walk. Because copy_copilot_session_state.sh copies the entire ~/.copilot/session-state/ tree (potentially multiple UUID subdirs), this can select a stale session deterministically (lexicographically smallest path) rather than the most recent session, causing the wrong run to be parsed. Consider selecting the best candidate (e.g., newest events.jsonl by ModTime, or newest UUID dir) instead of returning on the first match.

This issue also appears on line 205 of the same file.

See below for a potential fix:

				// Walk the full tree and select the newest candidate by modification time,
				// preferring events.jsonl over .log files.
				var bestEventsJsonl string
				var bestEventsModTime int64
				var bestLogFile string
				var bestLogModTime int64
				_ = filepath.Walk(flattenedDir, func(path string, info os.FileInfo, err error) error {
					if err != nil || info == nil {
						return nil
					}
					if !info.IsDir() {
						modTime := info.ModTime().UnixNano()
						if info.Name() == "events.jsonl" {
							if bestEventsJsonl == "" || modTime > bestEventsModTime {
								bestEventsJsonl = path
								bestEventsModTime = modTime
								logsParsingCoreLog.Printf("Found events.jsonl file candidate: %s", path)
							}
						} else if strings.HasSuffix(info.Name(), ".log") {
							if bestLogFile == "" || modTime > bestLogModTime {
								bestLogFile = path
								bestLogModTime = modTime
								logsParsingCoreLog.Printf("Found session log file candidate: %s", path)
							}
						}
					}
					return nil
				})
				if bestEventsJsonl != "" {
					return bestEventsJsonl, true
				}
				if bestLogFile != "" {
					return bestLogFile, true

Copilot uses AI. Check for mistakes.
Comment on lines +25 to +45
* Recursively searches a directory tree for the first events.jsonl file.
* This file is written by the Copilot CLI and contains structured session events.
* @param {string} dirPath - Directory to search
* @returns {string|null} Absolute path to events.jsonl, or null if not found
*/
function findEventsJsonlRecursive(dirPath) {
try {
const entries = fs.readdirSync(dirPath, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(dirPath, entry.name);
if (entry.isDirectory()) {
const found = findEventsJsonlRecursive(fullPath);
if (found) return found;
} else if (entry.name === "events.jsonl") {
return fullPath;
}
}
} catch (e) {
// Ignore read errors (e.g. permission denied on subdirectories)
}
return null;
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

findEventsJsonlRecursive returns the first events.jsonl found depth-first. Since the Copilot CLI session-state directory can contain multiple UUID subdirectories (and copy_copilot_session_state.sh copies them all), this can select an older session depending on directory ordering. Consider scanning all candidates and picking the newest events.jsonl (by mtime) or otherwise disambiguating which session should be parsed.

Suggested change
* Recursively searches a directory tree for the first events.jsonl file.
* This file is written by the Copilot CLI and contains structured session events.
* @param {string} dirPath - Directory to search
* @returns {string|null} Absolute path to events.jsonl, or null if not found
*/
function findEventsJsonlRecursive(dirPath) {
try {
const entries = fs.readdirSync(dirPath, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(dirPath, entry.name);
if (entry.isDirectory()) {
const found = findEventsJsonlRecursive(fullPath);
if (found) return found;
} else if (entry.name === "events.jsonl") {
return fullPath;
}
}
} catch (e) {
// Ignore read errors (e.g. permission denied on subdirectories)
}
return null;
* Recursively searches a directory tree for the newest events.jsonl file.
* This file is written by the Copilot CLI and contains structured session events.
* @param {string} dirPath - Directory to search
* @returns {string|null} Absolute path to the newest events.jsonl, or null if not found
*/
function findEventsJsonlRecursive(dirPath) {
/** @type {string|null} */
let newestPath = null;
let newestMtimeMs = -1;
/**
* @param {string} currentDir
*/
function walk(currentDir) {
try {
const entries = fs.readdirSync(currentDir, { withFileTypes: true });
for (const entry of entries) {
const fullPath = path.join(currentDir, entry.name);
if (entry.isDirectory()) {
walk(fullPath);
} else if (entry.name === "events.jsonl") {
try {
const stat = fs.statSync(fullPath);
const mtimeMs = stat.mtimeMs;
if (mtimeMs > newestMtimeMs) {
newestMtimeMs = mtimeMs;
newestPath = fullPath;
}
} catch (e) {
// Ignore stat errors for individual files
}
}
}
} catch (e) {
// Ignore read errors (e.g. permission denied on subdirectories)
}
}
walk(dirPath);
return newestPath;

Copilot uses AI. Check for mistakes.
Comment on lines +441 to +442
if !strings.HasSuffix(found, "events.jsonl") {
t.Errorf("Expected events.jsonl to be preferred over .log, but found %s", found)
Copy link

Copilot AI Apr 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This subtest only asserts that the chosen file ends with events.jsonl. To make the preference behavior robust (and avoid passing if some other events.jsonl is introduced in the fixture), assert the exact expected path (found == eventsJsonl) like the previous subtest does.

Suggested change
if !strings.HasSuffix(found, "events.jsonl") {
t.Errorf("Expected events.jsonl to be preferred over .log, but found %s", found)
if found != eventsJsonl {
t.Errorf("Expected events.jsonl (%s) to be preferred over .log, but found %s", eventsJsonl, found)

Copilot uses AI. Check for mistakes.
@pelikhan pelikhan merged commit 012aaac into main Apr 2, 2026
145 of 154 checks passed
@pelikhan pelikhan deleted the copilot/update-js-parser-logs branch April 2, 2026 05:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants