-
Notifications
You must be signed in to change notification settings - Fork 5.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 Bug Report: Github Discovery error #15940
Comments
Hey 👋 Not seen this reported before, but I think we're going to need a little more information on what's going on and what the actual error is. It looks like it's something to do with the Github App credentials or something. Would it be possible for you to try and debug this a little more to get us some more detailed logging information. Possibly by digging in It could be some incompatibility between some of the octokit libraries, or something different it's hard to tell with these error messages. |
Thanks @benjdlambert . I've managed to narrow it down after an exhaustive check. I hope this is right as what looked like could be a network issue but I'm a little sceptical as why is the newer version producing this. Log produced in @octokit/request package.
I've done a search for @octokit/request package being used in the project, not sure where @lerna is but the request log is coming from the v6.2.3
UPDATE: Just fyi, I'm behind a corporate proxy. Have tested being outside the proxy and it works but just not in and it seems to be pointing to the fetch module. |
I think that the error is here:
The proxy is getting in the way. Looks like is not respecting the proxies. Have you looked at these docs? https://github.com/backstage/backstage/blob/master/contrib/docs/tutorials/help-im-behind-a-corporate-proxy.md |
Hi @benjdlambert , I have and have set up the package global-agent/bootstrap as per doc in previous version prior to the upgrade and that worked. I'll need to keep digging to see why its not respecting the proxy in this newer version. |
Hmm yeah that's interesting, I wonder if they've changed how they make requests from |
Just a quick update, I've downgraded Node 18 to Node 16 (specifically 16.13) and have been presented with a different error. It seems the response for the call to Github API and it returned a parsing error when processing with response.json().
Wondering if its todo with the socket hang up ECONNRESET since the response size is literally 0 bytes. Additionally, same request done via Postman with the same credentials returned the expected response flawlessly. |
Just to confirm - are you still behind the corporate proxy here, as the ECONNRESET might hint at? |
Hi there! Yes, I'm still behind corporate proxy and that is not going anywhere I'm afraid. I'm still digging into some of the issues and testing a different envs and it appears to have mixed results. The ECONNRESET is bizarre as when I was debugging, the error fails when it tries to perform response.json() . However, when I write my own stream buffer reader, it successfully produced the json payload as expected coming from github but then ends with the ECONNRESET. The error message is terribly misleading. I have one question, does this change might have any impact for Backstage configured backend auth with Github Apps? We never needed to use PAT tokens since we used Github App instead. UPDATE: Here's the response I get on the UI side of catalog-import page.
|
OK I think i'm starting to understand what's going on here. The bump to Then using this information, I had a look at the Looks like there might be a workaround here at the bottom, but it's not great. I think once we have the backend system we can maybe have the ability to have a fetch service that can be used everywhere that can add these settings so we wont need the global fetch anymore. I'd look at trying to use that workaround to see if it works for you 🤞 and report back. |
To be clear also, I think you will need both |
Hi there! Apologies for late response to this. I've given the undici solution a go. It works on my local dev which is fantastic! Unfortunately, when I promote it through my envs, it still fails with the same error. I've pulled the docker image and run as my local container and it doesn't work. 😭 However, I've managed to progress through but only by running on Node16 instead of Node18. At least this buys me further time (a couple of months maybe?) to locate a fix for now and unblocked the upgrade/development of my org's Backstage instance.
|
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. |
For the interest of closure to this issue. I've finally managed to resolve it. Not by using undici package but a setting in Node.
This turns off the experimental fetch api in native node18 and uses the dependency package fetch module instead. This in-turns respects the proxy setting as per documentation. Avoid using experimental feature, I'd say. 😸 Source: node-fetch/node-fetch#1566 |
This also fixed my GitHub Discovery failures - however, the scaffolder was failing to publish:
I fixed both the scaffolder and discovery, and also removed the need for the no-experimental-fetch flag with NodeJS 20, by implementing
packages/backend/src/index.ts:
|
This got me a more informative error, but same overall problem.
|
I noticed Node20 has reached LTS. Would upgrading to that resolve the issue? |
Can confirm the issue is also no longer a problem for me after reverting the earlier changes and implementing the suggestion above. |
@darrenyung Thanks! I made this to follow up #22658 |
📜 Description
Problem started when upgrading from 1.7.2 to 1.8.3 and still persists when upgrading to 1.10.1
I've made the changes as advised regarding the naming from GitHub to Github but when the app starts on my local, I get the errors coming from my config for the Github Discovery feature.
I haven't change anything from the config nor the Github App itself as it was working prior to 1.8.3. Did I miss something in the upgrade?
👍 Expected behavior
Github Discovery working and able to pull org data from public Github
👎 Actual Behavior with Screenshots
In the console, error reported the following
In the UI, Catalog Import returns the following error
👟 Reproduction steps
yarn dev
During runtime,
📃 Provide the context for the Bug.
No response
🖥️ Your Environment
OS: Darwin 22.2.0 - darwin/x64
node: v18.12.1
yarn: 1.22.19
cli: 0.22.1 (installed)
backstage: 1.10.1
Dependencies:
@backstage/app-defaults 1.1.0
@backstage/backend-app-api 0.3.1
@backstage/backend-common 0.18.1
@backstage/backend-plugin-api 0.3.1
@backstage/backend-tasks 0.4.2
@backstage/backend-test-utils 0.1.33
@backstage/catalog-client 1.3.0
@backstage/catalog-model 1.1.5
@backstage/cli-common 0.1.11
@backstage/cli 0.22.1
@backstage/config-loader 1.1.8
@backstage/config 1.0.6
@backstage/core-app-api 1.4.0
@backstage/core-components 0.11.2, 0.12.3
@backstage/core-plugin-api 1.3.0
@backstage/dev-utils 1.0.11
@backstage/errors 1.1.4
@backstage/integration-aws-node 0.1.1
@backstage/integration-react 1.1.9
@backstage/integration 1.4.2
@backstage/plugin-api-docs 0.8.14
@backstage/plugin-app-backend 0.3.41
@backstage/plugin-auth-backend 0.17.4
@backstage/plugin-auth-node 0.2.10
@backstage/plugin-badges-backend 0.1.35
@backstage/plugin-badges 0.2.38
@backstage/plugin-bazaar-backend 0.2.4
@backstage/plugin-bazaar 0.2.3
@backstage/plugin-catalog-backend-module-github 0.2.4
@backstage/plugin-catalog-backend 1.7.1
@backstage/plugin-catalog-common 1.0.10
@backstage/plugin-catalog-graph 0.2.26
@backstage/plugin-catalog-import 0.9.4
@backstage/plugin-catalog-node 1.3.2
@backstage/plugin-catalog-react 1.2.4
@backstage/plugin-catalog 1.7.2
@backstage/plugin-cost-insights-common 0.1.1
@backstage/plugin-cost-insights 0.12.3
@backstage/plugin-events-node 0.2.2
@backstage/plugin-explore-common 0.0.1
@backstage/plugin-explore-react 0.0.25
@backstage/plugin-explore 0.3.45
@backstage/plugin-github-actions 0.5.14
@backstage/plugin-github-issues 0.2.3
@backstage/plugin-github-pull-requests-board 0.1.8
@backstage/plugin-home 0.4.30
@backstage/plugin-kubernetes-backend 0.9.2
@backstage/plugin-kubernetes-common 0.5.1
@backstage/plugin-kubernetes 0.7.7
@backstage/plugin-org 0.6.4
@backstage/plugin-permission-common 0.7.3
@backstage/plugin-permission-node 0.7.4
@backstage/plugin-permission-react 0.4.9
@backstage/plugin-proxy-backend 0.2.35
@backstage/plugin-scaffolder-backend 1.10.1
@backstage/plugin-scaffolder-common 1.2.4
@backstage/plugin-scaffolder-react 1.0.1
@backstage/plugin-scaffolder 1.10.1
@backstage/plugin-search-backend-module-elasticsearch 1.1.2
@backstage/plugin-search-backend-module-pg 0.5.2
@backstage/plugin-search-backend-node 1.1.2
@backstage/plugin-search-backend 1.2.2
@backstage/plugin-search-common 1.2.1
@backstage/plugin-search-react 1.4.0
@backstage/plugin-search 1.0.7
@backstage/plugin-shortcuts 0.3.6
@backstage/plugin-sonarqube-backend 0.1.6
@backstage/plugin-sonarqube-react 0.1.1
@backstage/plugin-sonarqube 0.6.2
@backstage/plugin-stack-overflow 0.1.10
@backstage/plugin-tech-insights-backend-module-jsonfc 0.1.25
@backstage/plugin-tech-insights-backend 0.5.7
@backstage/plugin-tech-insights-common 0.2.9
@backstage/plugin-tech-insights-node 0.3.9
@backstage/plugin-tech-insights 0.3.6
@backstage/plugin-tech-radar 0.6.0
@backstage/plugin-techdocs-addons-test-utils 1.0.9
@backstage/plugin-techdocs-backend 1.5.2
@backstage/plugin-techdocs-module-addons-contrib 1.0.9
@backstage/plugin-techdocs-node 1.4.5
@backstage/plugin-techdocs-react 1.1.2
@backstage/plugin-techdocs 1.4.3
@backstage/plugin-todo-backend 0.1.38
@backstage/plugin-todo 0.2.16
@backstage/plugin-user-settings 0.6.2
@backstage/release-manifests 0.0.8
@backstage/test-utils 1.2.4
@backstage/theme 0.2.16
@backstage/types 1.0.2
@backstage/version-bridge 1.0.3
👀 Have you spent some time to check if this bug has been raised before?
🏢 Have you read the Code of Conduct?
Are you willing to submit PR?
None
The text was updated successfully, but these errors were encountered: