Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: Bump go-github to v59 in Github plugin #16797

Merged
merged 2 commits into from
Feb 21, 2024

Conversation

AshCorr
Copy link
Contributor

@AshCorr AshCorr commented Feb 21, 2024

Summary

Bump go-github to v59 in the Github source plugin.

This allows Cloudquery to take advantage of some of the new Github APIs, specifically the SBOM API in #16796

I've tested this PR in our Cloudquery environment and did not notice any issues arising from this bump, although to be fair we don't collect every table due to resource and time constraints.

Looking at the Release notes for go-github I don't see any breaking changes particularly concerning apart from a change of timestamp types in v50

@cq-bot
Copy link
Contributor

cq-bot commented Feb 21, 2024

This PR has the following changes to source plugin(s) tables:

  • ⚠️ BREAKING CHANGE: Table github_billing_action: column type changed from int64 to float64 for included_minutes
  • ⚠️ BREAKING CHANGE: Table github_billing_action: column type changed from int64 to float64 for total_minutes_used
  • ⚠️ BREAKING CHANGE: Table github_billing_package: column type changed from int64 to float64 for included_gigabytes_bandwidth
  • ⚠️ BREAKING CHANGE: Table github_billing_storage: column type changed from int64 to float64 for estimated_storage_for_month
  • Table github_issues: column added with name draft and type bool
  • Table github_organization_dependabot_alerts: column added with name auto_dismissed_at and type timestamp[us, tz=UTC]
  • Table github_organization_dependabot_alerts: column added with name repository and type json
  • Table github_organizations: column added with name secret_scanning_validity_checks_enabled and type bool
  • Table github_repositories: column added with name custom_properties and type json
  • Table github_repositories: column added with name web_commit_signoff_required and type bool
  • Table github_repository_dependabot_alerts: column added with name auto_dismissed_at and type timestamp[us, tz=UTC]
  • Table github_repository_dependabot_alerts: column added with name repository and type json
  • Table github_repository_keys: column added with name added_by and type utf8
  • Table github_repository_keys: column added with name last_used and type timestamp[us, tz=UTC]
  • Table github_team_repositories: column added with name custom_properties and type json
  • Table github_team_repositories: column added with name web_commit_signoff_required and type bool
  • Table github_workflow_jobs: column added with name created_at and type timestamp[us, tz=UTC]
  • Table github_workflow_jobs: column added with name head_branch and type utf8
  • Table github_workflow_jobs: column added with name workflow_name and type utf8
  • Table github_workflow_runs: column added with name display_title and type utf8
  • Table github_workflow_runs: column added with name referenced_workflows and type json
  • Table github_workflow_runs: column added with name triggering_actor and type json

@AshCorr AshCorr changed the title Bump go-github to v59 in Github plugin feat: Bump go-github to v59 in Github plugin Feb 21, 2024
@AshCorr AshCorr marked this pull request as ready for review February 21, 2024 18:23
@AshCorr AshCorr requested a review from a team as a code owner February 21, 2024 18:23
@AshCorr AshCorr requested review from maaarcelino and removed request for a team February 21, 2024 18:23
@@ -233,7 +233,7 @@ func githubClientForHTTPClient(httpClient *http.Client, logger zerolog.Logger, e
}

if enterpriseSettings != nil {
return github.NewEnterpriseClient(enterpriseSettings.BaseURL, enterpriseSettings.UploadURL, rateLimiter)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NewEnterpriseClient has been deprecated and throws errors when running make lint, the replacement is to use WithEnterpriseURLs

@erezrokah erezrokah requested review from erezrokah and removed request for maaarcelino February 21, 2024 19:40
@erezrokah erezrokah self-assigned this Feb 21, 2024
@erezrokah erezrokah changed the title feat: Bump go-github to v59 in Github plugin feat!: Bump go-github to v59 in Github plugin Feb 21, 2024
Copy link
Contributor

@erezrokah erezrokah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great @AshCorr 💯 Thanks for the PR.
The timestamp changes are not breaking for us as we already know to resolve GitHub timestamp representation as an arrow timestamp

timestamp := github.Timestamp{}

the changes to float are indeed schema breaking (as we don't migrate changes in data types at the moment), so I've updated the PR title from feat: to feat!: to signal our automation to release this under a major version bump

@erezrokah erezrokah added the automerge Automatically merge once required checks pass label Feb 21, 2024
@kodiakhq kodiakhq bot merged commit 5b89ef6 into cloudquery:main Feb 21, 2024
26 checks passed
kodiakhq bot pushed a commit that referenced this pull request Feb 21, 2024
…16796)



#### Summary

Add a `github_repository_sboms` table to sync data from [Github's Software Bill Of Materials (SBOM) API](https://docs.github.com/en/rest/dependency-graph/sboms?apiVersion=2022-11-28)

This API can tell us which dependencies(NPM, Maven, etc) a repository has. With the Snyk plugin currently out of commission this provides a crucial way of gathering insights of which dependencies are being used across a Github organization.

This PR includes commits from #16797 . I've seperated it into 2 PRs to reduce the cognitive load on reviewers, you may want to review this PR starting from [7d92c1d](7d92c1d)

<img width="2034" alt="image" src="https://github.com/cloudquery/cloudquery/assets/21217225/c8da58e7-fb88-427e-8839-c3c920aa6a60">


<!--
Explain what problem this PR addresses
-->

<!--
kodiakhq bot pushed a commit that referenced this pull request Feb 21, 2024
🤖 I have created a release *beep* *boop*
---


## [8.0.0](plugins-source-github-v7.7.1...plugins-source-github-v8.0.0) (2024-02-21)


### ⚠ BREAKING CHANGES

* Bump `go-github` to v59 in Github plugin ([#16797](#16797))

### This Release has the Following Changes to Tables
- Table `github_billing_action`: column type changed from `int64` to `float64` for `included_minutes` (:warning: breaking)
- Table `github_billing_action`: column type changed from `int64` to `float64` for `total_minutes_used` (:warning: breaking)
- Table `github_billing_package`: column type changed from `int64` to `float64` for `included_gigabytes_bandwidth` (:warning: breaking)
- Table `github_billing_storage`: column type changed from `int64` to `float64` for `estimated_storage_for_month` (:warning: breaking)
- Table `github_issues`: column added with name `draft` and type `bool`
- Table `github_organization_dependabot_alerts`: column added with name `auto_dismissed_at` and type `timestamp[us, tz=UTC]`
- Table `github_organization_dependabot_alerts`: column added with name `repository` and type `json`
- Table `github_organizations`: column added with name `secret_scanning_validity_checks_enabled` and type `bool`
- Table `github_repositories`: column added with name `custom_properties` and type `json`
- Table `github_repositories`: column added with name `web_commit_signoff_required` and type `bool`
- Table `github_repository_dependabot_alerts`: column added with name `auto_dismissed_at` and type `timestamp[us, tz=UTC]`
- Table `github_repository_dependabot_alerts`: column added with name `repository` and type `json`
- Table `github_repository_keys`: column added with name `added_by` and type `utf8`
- Table `github_repository_keys`: column added with name `last_used` and type `timestamp[us, tz=UTC]`
- Table `github_repository_sboms` was added
- Table `github_team_repositories`: column added with name `custom_properties` and type `json`
- Table `github_team_repositories`: column added with name `web_commit_signoff_required` and type `bool`
- Table `github_workflow_jobs`: column added with name `created_at` and type `timestamp[us, tz=UTC]`
- Table `github_workflow_jobs`: column added with name `head_branch` and type `utf8`
- Table `github_workflow_jobs`: column added with name `workflow_name` and type `utf8`
- Table `github_workflow_runs`: column added with name `display_title` and type `utf8`
- Table `github_workflow_runs`: column added with name `referenced_workflows` and type `json`
- Table `github_workflow_runs`: column added with name `triggering_actor` and type `json`

### Features

* Add `github_repository_sboms` table to sync Github SBOM data ([#16796](#16796)) ([992434f](992434f))
* Bump `go-github` to v59 in Github plugin ([#16797](#16797)) ([5b89ef6](5b89ef6))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/plugin/source/github automerge Automatically merge once required checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants