Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to retryableHttpClient for GitHub AuthN API Client + More Logs #995

Merged

Conversation

yilmi
Copy link
Contributor

@yilmi yilmi commented Jan 4, 2023

Hi,

This pull request is meant to fix issue #994 and adds the following:

  • Making GitHub authenticated API Client use the retryableHttpClientTimeout() by default
  • Adding the repo URL to the log
    • "scanning git repo complete path: repoPath, time: scanTime"
    • "scanning git repo complete Repo: repoUrl path: repoPath, time: scanTime"
  • Adding a logger to all the httpClient supporting one
  • Fixing minor documentation issue (missing flags)

This has already helped recover from requests failures as this dev build log output can show:

{
    "level": "debug",
    "msg": "[DEBUG] GET https://api.github.com/repos/ORG/REPO: retrying in 1s (3 left)",
    "name": "RetryableHttpClientTimeout",
    "time": "2023-01-03T10:43:17Z"
}
{
    "level": "debug",
    "msg": "[DEBUG] GET https://api.github.com/orgs/ORG/repos?per_page=100 (status: 502): retrying in 1s (3 left)",
    "name": "RetryableHttpClientTimeout",
    "time": "2023-01-05T07:57:26Z"
}

{
    "level": "debug",
    "msg": "[DEBUG] GET https://keychecker.trufflesecurity.com/fingerprint/<FINGERPRINT> (status: 503): retrying in 0s (3 left)",
    "name": "RetryableHttpClient",
    "time": "2023-01-06T07:44:31Z"
}

This comes from the client - https://github.com/hashicorp/go-retryablehttp/blob/master/client.go#L690

-- EDIT: Added correct log output examples

@yilmi yilmi requested review from a team as code owners January 4, 2023 22:59
Copy link
Contributor

@dustin-decker dustin-decker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the contribution!

README.md Outdated Show resolved Hide resolved
Copy link
Contributor

@dustin-decker dustin-decker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@dustin-decker dustin-decker merged commit d720c0c into trufflesecurity:main Jan 9, 2023
@yilmi yilmi deleted the retryable-gh-requests-with-logs branch January 9, 2023 18:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants