Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ENT - update hasSBOM ingestion for large SBOMs and increase batch size #1748

Merged
merged 1 commit into from Mar 4, 2024

Conversation

pxp928
Copy link
Collaborator

@pxp928 pxp928 commented Mar 4, 2024

Description of the PR

Update hasSBOM ingestion to batch the includes into separate batched queries to avoid pq: got 70000 parameters but PostgreSQL only supports 65535 parameters

Updated Batch size to 5000 for improved performance

see: ent/ent#2539

PR Checklist

  • All commits have a Developer Certificate of Origin (DCO) -- they are generated using -s flag to git commit.
  • All new changes are covered by tests
  • If GraphQL schema is changed, make generate has been run
  • If OpenAPI spec is changed, make generate has been run
  • If collectsub protobuf has been changed, make proto has been run
  • All CI checks are passing (tests and formatting)
  • All dependent PRs have already been merged

…bulk ingestion

Signed-off-by: pxp928 <parth.psu@gmail.com>
@pxp928 pxp928 requested review from mrizzi, jeffmendoza, mihaimaruseac and lumjjb and removed request for mihaimaruseac March 4, 2024 23:28
@kodiakhq kodiakhq bot merged commit 277c791 into guacsec:main Mar 4, 2024
8 checks passed
@pxp928 pxp928 deleted the ent-fix-postgres-max-parm branch March 5, 2024 00:03
arorasoham9 pushed a commit to arorasoham9/guac that referenced this pull request Apr 1, 2024
…bulk ingestion (guacsec#1748)

Signed-off-by: pxp928 <parth.psu@gmail.com>
arorasoham9 pushed a commit to arorasoham9/guac that referenced this pull request Apr 4, 2024
…bulk ingestion (guacsec#1748)

Signed-off-by: pxp928 <parth.psu@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants