Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: avoid Unexpected end of JSON input when response body is empty #648

Merged
merged 4 commits into from
Nov 9, 2023

Conversation

tsusdere
Copy link
Contributor

@tsusdere tsusdere commented Oct 27, 2023

Resolves #649

Before the change?

  • Customer seems to be having issues with deploying their pages site in GHES and the error via the output from the debug appears to be caused by octokit. Seems when we are getting the response data we might run into an issue where the body is empty and hence we can't parse it using response.json() causing the error to be raised.
##[debug]Evaluating condition for step: 'Deploy to GitHub Pages'
2##[debug]Evaluating: success()
3##[debug]Evaluating success:
4##[debug]=> true
5##[debug]Result: true
6##[debug]Starting: Deploy to GitHub Pages
7##[debug]Loading inputs
8##[debug]Evaluating: github.token
9##[debug]Evaluating Index:
10##[debug]..Evaluating github:
11##[debug]..=> Object
12##[debug]..Evaluating String:
13##[debug]..=> 'token'
14##[debug]=> '***'
15##[debug]Result: '***'
16##[debug]Loading env
17
Run actions/deploy-pages@v1.2.9
18  with:
19    emit_telemetry: false
20    token: ***
21    timeout: 600000
22    error_count: 10
23    reporting_interval: 5000
24    artifact_name: github-pages
25    preview: false
26##[debug]all variables are set
27##[debug]all variables are set
28##[debug]ID token url is https://gitenterprise.xilinx.com/_services/pipelines/eRhS9LBtNTGin3wENmb093bx6wzbQ19LimyrpKukvyqgYmqa0d/00000000-0000-0000-0000-000000000000/_apis/distributedtask/hubs/Actions/plans/399052e0-ae3b-4207-bd94-d869210658b3/jobs/e07742bd-189a-5079-918b-43f8b2f94b89/idtoken?api-version=2.0
29::add-mask::***
30##[debug]Actor: nilsw
31##[debug]Action ID: deployment
32##[debug]Actions Workflow Run ID: 98423
33##[debug]all variables are set
34Artifact exchange URL: https://gitenterprise.xilinx.com/_services/pipelines/eRhS9LBtNTGin3wENmb093bx6wzbQ19LimyrpKukvyqgYmqa0d/_apis/pipelines/workflows/98423/artifacts?api-version=6.0-preview
35##[debug]{"count":1,"value":[{"containerId":240179,"size":10240,"signedContent":null,"fileContainerResourceUrl":"https://gitenterprise.xilinx.com/_services/pipelines/eRhS9LBtNTGin3wENmb093bx6wzbQ19LimyrpKukvyqgYmqa0d/_apis/resources/Containers/240179","type":"actions_storage","name":"github-pages","url":"https://gitenterprise.xilinx.com/_services/pipelines/eRhS9LBtNTGin3wENmb093bx6wzbQ19LimyrpKukvyqgYmqa0d/_apis/pipelines/1/runs/4/artifacts?artifactName=github-pages","expiresOn":"2023-10-26T19:25:29.6207174Z","items":null}]}
36Creating Pages deployment with payload:
37{
38 "artifact_url": "https://gitenterprise.xilinx.com/_services/pipelines/eRhS9LBtNTGin3wENmb093bx6wzbQ19LimyrpKukvyqgYmqa0d/_apis/pipelines/1/runs/4/artifacts?artifactName=github-pages&%24expand=SignedContent",
39 "pages_build_version": "0fbfbe3039ef54252a8d6af01dec3d3020463a67",
40 "oidc_token": "***"
41}
42
Error:
Creating Pages deployment failed
43
Error:
HttpError: invalid json response body at https://gitenterprise.xilinx.com/api/v3/repos/security/sfdb-debug/pages/deployment reason: Unexpected end of JSON input
44    at /scratch/ghe-runners/2/_work/_actions/actions/deploy-pages/v1.2.9/webpack:/deploy-pages/node_modules/@octokit/request/dist-node/index.js:108:1
45    at processTicksAndRejections (node:internal/process/task_queues:96:5)
46    at createPagesDeployment (/scratch/ghe-runners/2/_work/_actions/actions/deploy-pages/v1.2.9/webpack:/deploy-pages/src/api-client.js:116:1)
47    at Deployment.create (/scratch/ghe-runners/2/_work/_actions/actions/deploy-pages/v1.2.9/webpack:/deploy-pages/src/deployment.js:59:1)
48    at main (/scratch/ghe-runners/2/_work/_actions/actions/deploy-pages/v1.2.9/webpack:/deploy-pages/src/index.js:30:1)
49
Error:
HttpError: invalid json response body at https://gitenterprise.xilinx.com/api/v3/repos/security/sfdb-debug/pages/deployment reason: Unexpected end of JSON input
50##[debug]Node Action run completed with exit code 1
51##[debug]Finishing: Deploy to GitHub Pages

After the change?

  • Currently, wrapped the response in a try catch block. If we fail to parse the response to json we fall back to text and hence we are able to return an empty body as opposed to raising an error. However, we might need to investigate what is causing this as the GitHub API should not return an empty body.

Pull request checklist

  • Tests for the changes have been added (for bug fixes / features)
  • Docs have been reviewed and added / updated if needed (for bug fixes / features)

Does this introduce a breaking change?

Please see our docs on breaking changes to help!

  • Yes
  • No

@ghost ghost added this to Inbox in JS Oct 27, 2023
@wolfy1339 wolfy1339 added the Type: Bug Something isn't working as documented label Oct 27, 2023
@ghost ghost moved this from Inbox to Bugs in JS Oct 27, 2023
@gr2m
Copy link
Contributor

gr2m commented Oct 27, 2023

The tests are failing because coverage fell below 100%

------------------------|---------|----------|---------|---------|-------------------
File                    | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
------------------------|---------|----------|---------|---------|-------------------
All files               |   98.78 |      100 |     100 |   98.73 |                   
 fetch-wrapper.ts       |   98.38 |      100 |     100 |    98.3 | 162               
 get-buffer-response.ts |     100 |      100 |     100 |     100 |                   
 index.ts               |     100 |      100 |     100 |     100 |                   
 version.ts             |     100 |      100 |     100 |     100 |                   
 with-defaults.ts       |     100 |      100 |     100 |     100 |                   
------------------------|---------|----------|---------|---------|-------------------
Jest: "global" coverage threshold for statements (100%) not met: 98.78%
Jest: "global" coverage threshold for lines (100%) not met: 98.73%

After running npm test, you can open the coverage report to see where coverage is lacking

open coverage/lcov-report/index.html
image

I think in this particular case, we can add a

// istanbul ignore next

try {
return response.json();
} catch (e) {
return response.text();
Copy link
Contributor

@gr2m gr2m Oct 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

deleted suggestion, see below

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

given that the await could easily get lost in a future refactoring, I think we should add a proper test for this case after all, instead of ignoring the case, to avoid future regression. Especially given that this is happening very sporadically, we wouldn't realize the regression for a longer time.

// using .text(), but this should be investigated since if this were
// to occur in the GitHub API it really should not return an empty body.
try {
return response.json();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would likely need to use an await here in order for the try to actually catch anything since the error is presumably in the async phase of .json() rather than the sync execution.

Or do an "old-fashioned" promise usage like:

return response.json().catch(() => response.text())

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes! good catch!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should cover this with a test after all instead of ignoring the line for coverage, this is too easy to miss.

@gr2m
Copy link
Contributor

gr2m commented Oct 27, 2023

I tried reproducing the "body used already for" error with native fetch but to no avail

import { inspect } from "node:util";
const response = await fetch("https://httpbin.org/status/204");
const textResult = await response.json().catch(() => response.text());
console.log(inspect(textResult));

I'm looking into it

@gr2m
Copy link
Contributor

gr2m commented Oct 27, 2023

My guess is that it's a problem with node-fetch, and fetch-mock which we use for mocking is using node-fetch internally.

The native fetch in Node is still experimental, and many companies turn it off as e.g. nock does not support mocking the native fetch yet (nock/nock#2397).

I do not love it but we could add yet another catch and fallback to an empty string

response.json().catch(() => response.text()).catch(() => '')

We should update the comment to explain why

@gr2m
Copy link
Contributor

gr2m commented Oct 27, 2023

I pushed my change. The suggested alternative is to only use .text(), and then try to parse it with JSON.parse(). Which might actually be a good approach and result and more readable code:

-    return response
-      .json()
-      // In the event that we get an empty response body we fallback to
-      // using .text(), but this should be investigated since if this were
-      // to occur in the GitHub API it really should not return an empty body.
-      .catch(() => response.text())
-      // `node-fetch` is throwing a "body used already for" error if `.text()` is run
-      // after a failed .json(). To account for that we fallback to an empty string
-      .catch(() => "");
+    const responseText = await response.text();
+
+    // Account for the possibility that GitHub returns an empty body
+    // or invalid JSON in an error response
+    if (!responseText.length) {
+      return "";
+    }
+
+    try {
+      return JSON.parse(responseText);
+    } catch (error) {
+      return responseText;
+    }

Personally, I like the current version with the .catch() chaining better

Copy link
Contributor

@gr2m gr2m left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, here is an alternative approach, but I like the current one better:
#648 (comment)

@wolfy1339 wolfy1339 changed the title fix: Unexpected end of JSON input fix: avoid Unexpected end of JSON input when response body is empty Nov 9, 2023
@wolfy1339 wolfy1339 merged commit 819cc3f into octokit:main Nov 9, 2023
7 checks passed
Copy link

github-actions bot commented Nov 9, 2023

🎉 This PR is included in version 8.1.5 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
released Type: Bug Something isn't working as documented
Projects
Archived in project
JS
  
Bugs
Development

Successfully merging this pull request may close these issues.

[BUG]: Unexpected end of JSON input
4 participants