Skip to content

Adding clustering support for python storage write api #35526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

TanuSharma2511
Copy link
Contributor

@TanuSharma2511 TanuSharma2511 commented Jul 4, 2025

It resolves a part of #35329

1.Dynamic clustering is not supported yet. It makes sense to support dynamic clustering after we support dynamic schema for Python Bigquery Storage Write API.
2.Clustering Support for Bigquery Managed IO.


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • Update CHANGES.md with noteworthy changes.
  • If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels
Python tests
Java tests
Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

Copy link

codecov bot commented Jul 9, 2025

Codecov Report

Attention: Patch coverage is 25.00000% with 3 lines in your changes missing coverage. Please review.

Project coverage is 56.54%. Comparing base (57b151a) to head (5bee162).
Report is 291 commits behind head on master.

Files with missing lines Patch % Lines
...gquery/providers/PortableBigQueryDestinations.java 25.00% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #35526      +/-   ##
============================================
+ Coverage     56.48%   56.54%   +0.05%     
- Complexity     3299     3370      +71     
============================================
  Files          1185     1191       +6     
  Lines        181776   182157     +381     
  Branches       3409     3456      +47     
============================================
+ Hits         102678   102998     +320     
- Misses        75829    75847      +18     
- Partials       3269     3312      +43     
Flag Coverage Δ
java 70.77% <25.00%> (+0.17%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions bot removed the website label Jul 9, 2025
@TanuSharma2511 TanuSharma2511 marked this pull request as ready for review July 9, 2025 15:08
Copy link
Contributor

github-actions bot commented Jul 9, 2025

Assigning reviewers:

R: @shunping for label python.
R: @chamikaramj for label java.

Note: If you would like to opt out of this review, comment assign to next reviewer.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

@TanuSharma2511
Copy link
Contributor Author

Run Python_Dataframes PreCommit 3.11

@TanuSharma2511
Copy link
Contributor Author

Run Python_Dataframes PreCommit 3.12

@TanuSharma2511
Copy link
Contributor Author

@liferoad

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant