Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh run view --log does not display logs when job names are beyond a certain length #7642

Closed
Tracked by #7643
williammartin opened this issue Jun 29, 2023 · 6 comments · Fixed by #8684
Closed
Tracked by #7643
Labels
bug Something isn't working gh-run relating to the gh run command help wanted Contributions welcome p2 Affects more than a few users but doesn't prevent core functions

Comments

@williammartin
Copy link
Member

Describe the bug

This is a spinoff from #5011

Given a job in a workflow with a long name, gh run view --log will not display logs for that name.

Steps to reproduce the behavior

Here is a workflow definition that includes a job with a long name. Note that I accidentally left the step name as duplicate but it's not relevant.

Running gh run view 5415184943 --log produces only the logs for the shortjob:

shortjob        Set up job      2023-06-29T18:15:54.4601154Z Current runner version: '2.305.0'
shortjob        Set up job      2023-06-29T18:15:54.4659852Z ##[group]Operating System
shortjob        Set up job      2023-06-29T18:15:54.4660607Z Ubuntu
shortjob        Set up job      2023-06-29T18:15:54.4660922Z 22.04.2
shortjob        Set up job      2023-06-29T18:15:54.4661172Z LTS
shortjob        Set up job      2023-06-29T18:15:54.4661557Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4661856Z ##[group]Runner Image
shortjob        Set up job      2023-06-29T18:15:54.4662253Z Image: ubuntu-22.04
shortjob        Set up job      2023-06-29T18:15:54.4662611Z Version: 20230625.1.0
shortjob        Set up job      2023-06-29T18:15:54.4663193Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20230625.1.0/images/linux/Ubuntu2204-Readme.md
shortjob        Set up job      2023-06-29T18:15:54.4663834Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20230625.1.0
shortjob        Set up job      2023-06-29T18:15:54.4664318Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4664681Z ##[group]Runner Image Provisioner
shortjob        Set up job      2023-06-29T18:15:54.4664992Z 2.0.238.1
shortjob        Set up job      2023-06-29T18:15:54.4665334Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4666082Z ##[group]GITHUB_TOKEN Permissions
shortjob        Set up job      2023-06-29T18:15:54.4666664Z Contents: read
shortjob        Set up job      2023-06-29T18:15:54.4666956Z Metadata: read
shortjob        Set up job      2023-06-29T18:15:54.4667704Z Packages: read
shortjob        Set up job      2023-06-29T18:15:54.4668042Z ##[endgroup]
shortjob        Set up job      2023-06-29T18:15:54.4671985Z Secret source: Actions
shortjob        Set up job      2023-06-29T18:15:54.4672585Z Prepare workflow directory
shortjob        Set up job      2023-06-29T18:15:54.5458534Z Prepare all required actions
shortjob        Set up job      2023-06-29T18:15:54.5759678Z Complete job name: shortjob
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.6997865Z ##[group]Run echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.6998447Z echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.7617853Z shell: /usr/bin/bash -e {0}
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.7618322Z ##[endgroup]
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2023-06-29T18:15:54.8244988Z You should not see me
shortjob        Complete job    2023-06-29T18:15:54.8549991Z Cleaning up orphan processes

Expected vs actual behavior

I would expect to see logs as in the Web UI for the job with the long name.

Why does this happen?

As described in my comment when the .zip file is produced, file paths are truncated due to intrinsic length limitations of .zip files. Our regexp matching logic does not apply the same behaviour, resulting in us not matching file names to their jobs.

See the truncated Loreum Ipsum dir name below:

➜  run-log-5415184943-1688062544 ls -lah
total 32
-rw-r--r--  1 williammartin  staff   2.2K Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types (1).txt
-rw-r--r--  1 williammartin  staff   452B Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types.txt
-rw-r--r--  1 williammartin  staff   2.1K Jun 29 18:25 1_shortjob.txt
-rw-r--r--  1 williammartin  staff   454B Jun 29 18:25 2_shortjob.txt
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 shortjob

Notes

Related issues: #7640, #7641

@williammartin williammartin added the bug Something isn't working label Jun 29, 2023
@williammartin williammartin added the needs-triage needs to be reviewed label Jun 29, 2023
@samcoe samcoe added p2 Affects more than a few users but doesn't prevent core functions and removed needs-triage needs to be reviewed labels Jun 29, 2023
@williammartin williammartin added gh-run relating to the gh run command core This issue is not accepting PRs from outside contributors labels Oct 3, 2023
@shayn-orca
Copy link
Contributor

This is reproducing for me as well. Any plans to apply a fix? Any good workarounds in the meanwhile?

Thanks

@williammartin
Copy link
Member Author

Hey @shayn-orca,

Hadn't prioritised anything around this because it was just theoretical and no one had actually run into it yet.

No workaround comes to mind other than reducing the length of your job names.

I don't think a fix for this would be particularly complicated, just truncating some names based on length. Would you be interested in opening a PR if I pointed you in the right direction?

@shayn-orca
Copy link
Contributor

Sure @williammartin - please send me some directions, I'd love to contribute to cli/cli, I use it all the time :)

@williammartin
Copy link
Member Author

Well, the code I believe we're interested in starts about here:

func attachRunLog(rlz *zip.Reader, jobs []shared.Job) {
for i, job := range jobs {
for j, step := range job.Steps {
re := logFilenameRegexp(job, step)
for _, file := range rlz.File {
if re.MatchString(file.Name) {
jobs[i].Steps[j].Log = file
break
}
}
}
}
}

This function is is looping over files in the downloaded zip and comparing it to the workflow metadata. So for each step in each job, it tries to match the name of a file in the zip via regex. The thing is that the function that produces the regex doesn't account for the server side truncation:

sanitizedJobName := strings.ReplaceAll(job.Name, "/", "")
re := fmt.Sprintf(`%s\/%d_.*\.txt`, regexp.QuoteMeta(sanitizedJobName), step.Number)
return regexp.MustCompile(re)

This looks at the name of the job in the workflow, replaces any / to make composite actions work, and then produces a regex to match against. This regex needs to be updated to account for the truncation, which looks to be 89 characters on the dir name.

Below you will find an example output of the run log after unzip:

➜  run-log-5415184943-1688062544 ls -lah
total 32
-rw-r--r--  1 williammartin  staff   2.2K Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types (1).txt
-rw-r--r--  1 williammartin  staff   452B Jun 29 18:25 1_Lorem Ipsum is simply dummy text of the printing and types.txt
-rw-r--r--  1 williammartin  staff   2.1K Jun 29 18:25 1_shortjob.txt
-rw-r--r--  1 williammartin  staff   454B Jun 29 18:25 2_shortjob.txt
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has
drwxr-xr-x  5 williammartin  staff   160B Jun 29 20:25 shortjob

The regex is kind of confusing so let's look closer:

fmt.Sprintf(`%s\/%d_.*\.txt`, regexp.QuoteMeta(sanitizedJobName), step.Number)

This is going to produce a regex that looks like:

shortjob\/1_.*\.txt

This doesn't seem to match anything right? Well, inside a zip there is just a flat file structure and / is used to indicate directory hierarchy. So the zip entry this matches will actually be like shortjob/1_<stepname> but since the number makes the step unique it's not required to match that last portion, hence the .*.

Ok so anyway, I would bet that if you added a line:

func logFilenameRegexp(job shared.Job, step shared.Step) *regexp.Regexp {
	sanitizedJobName := strings.ReplaceAll(job.Name, "/", "")
	// sanitizedJobName = truncateString(sanitizedJobName, 89)
	re := fmt.Sprintf(`%s\/%d_.*\.txt`, regexp.QuoteMeta(sanitizedJobName), step.Number)
	return regexp.MustCompile(re)
}

That things would start working for you again. The question is, does the server do truncation before or after / (I'm going to bet after, but we can confirm after proving our theory).

Finally, we'll need to add a test for this. It looks like we have a test that opens up a fixture and checks how attachRunLog works, so we should add a file in there with a truncated filename as compared to a new entry in the table driven test that has a long job name:

rlz, err := zip.OpenReader("./fixtures/run_log.zip")

Hope that makes sense and if not please let me know!

@williammartin williammartin added help wanted Contributions welcome and removed core This issue is not accepting PRs from outside contributors labels Feb 2, 2024
@shayn-orca
Copy link
Contributor

Hey @williammartin, check out #8684 please :) We've tested manually and added a UT and it seems to work OK.

@williammartin
Copy link
Member Author

Using the same workflow definition as above, we now get logs from the long job name:

shortjob        Set up job      2024-05-06T09:50:14.1206542Z Current runner version: '2.316.0'
shortjob        Set up job      2024-05-06T09:50:14.1230364Z ##[group]Operating System
shortjob        Set up job      2024-05-06T09:50:14.1231104Z Ubuntu
shortjob        Set up job      2024-05-06T09:50:14.1231498Z 22.04.4
shortjob        Set up job      2024-05-06T09:50:14.1231862Z LTS
shortjob        Set up job      2024-05-06T09:50:14.1232253Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1232669Z ##[group]Runner Image
shortjob        Set up job      2024-05-06T09:50:14.1233110Z Image: ubuntu-22.04
shortjob        Set up job      2024-05-06T09:50:14.1233548Z Version: 20240422.1.0
shortjob        Set up job      2024-05-06T09:50:14.1234563Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240422.1/images/ubuntu/Ubuntu2204-Readme.md
shortjob        Set up job      2024-05-06T09:50:14.1236013Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240422.1
shortjob        Set up job      2024-05-06T09:50:14.1236916Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1237361Z ##[group]Runner Image Provisioner
shortjob        Set up job      2024-05-06T09:50:14.1237854Z 2.0.369.1
shortjob        Set up job      2024-05-06T09:50:14.1238245Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1239254Z ##[group]GITHUB_TOKEN Permissions
shortjob        Set up job      2024-05-06T09:50:14.1240771Z Contents: read
shortjob        Set up job      2024-05-06T09:50:14.1241352Z Metadata: read
shortjob        Set up job      2024-05-06T09:50:14.1241764Z Packages: read
shortjob        Set up job      2024-05-06T09:50:14.1242457Z ##[endgroup]
shortjob        Set up job      2024-05-06T09:50:14.1245399Z Secret source: Actions
shortjob        Set up job      2024-05-06T09:50:14.1246114Z Prepare workflow directory
shortjob        Set up job      2024-05-06T09:50:14.1862882Z Prepare all required actions
shortjob        Set up job      2024-05-06T09:50:14.2091965Z Complete job name: shortjob
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3122787Z ##[group]Run echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3123535Z echo "You should not see me"
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3682154Z shell: /usr/bin/bash -e {0}
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.3682755Z ##[endgroup]
shortjob        Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been the industry's standard dummy text ever since the 1500s  2024-05-06T09:50:14.4167284Z You should not see me
shortjob        Complete job    2024-05-06T09:50:14.4439332Z Cleaning up orphan processes
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3250337Z Current runner version: '2.316.0'
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3275062Z ##[group]Operating System
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3275970Z Ubuntu
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3276328Z 22.04.4
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3276760Z LTS
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3277061Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3277456Z ##[group]Runner Image
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3277968Z Image: ubuntu-22.04
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3278330Z Version: 20240422.1.0
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3279325Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240422.1/images/ubuntu/Ubuntu2204-Readme.md
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3280815Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240422.1
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3281645Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3282063Z ##[group]Runner Image Provisioner
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3282604Z 2.0.369.1
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3282925Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3283870Z ##[group]GITHUB_TOKEN Permissions
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3285501Z Contents: read
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3285935Z Metadata: read
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3286331Z Packages: read
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3287065Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3290362Z Secret source: Actions
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3290923Z Prepare workflow directory
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.3913572Z Prepare all required actions
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Set up job      2024-05-06T09:50:15.4153383Z Complete job name: Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5244563Z ##[group]Run echo "You should not see me"
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5245295Z echo "You should not see me"
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5851988Z shell: /usr/bin/bash -e {0}
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.5852558Z ##[endgroup]
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    duplicate       2024-05-06T09:50:15.6258664Z You should not see me
Lorem Ipsum is simply dummy text of the printing and typesetting industry Lorem Ipsum has been th...    Complete job    2024-05-06T09:50:15.6533322Z Cleaning up orphan processes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working gh-run relating to the gh run command help wanted Contributions welcome p2 Affects more than a few users but doesn't prevent core functions
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants
@williammartin @samcoe @shayn-orca and others