Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using spark default parallelism as default #668

Conversation

davidrabinowitz
Copy link
Member

No description provided.

@davidrabinowitz
Copy link
Member Author

/gcbrun

@codecov
Copy link

codecov bot commented Jun 9, 2022

Codecov Report

Merging #668 (377abbb) into master (6b96f81) will decrease coverage by 0.10%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #668      +/-   ##
==========================================
- Coverage   19.26%   19.16%   -0.11%     
==========================================
  Files         135      135              
  Lines        5206     5187      -19     
  Branches      598      587      -11     
==========================================
- Hits         1003      994       -9     
+ Misses       4140     4130      -10     
  Partials       63       63              
Flag Coverage Δ
integrationtest-2.12 20.17% <0.00%> (-0.01%) ⬇️
integrationtest-2.4 17.98% <0.00%> (-0.01%) ⬇️
integrationtest-3.1 17.61% <0.00%> (-0.01%) ⬇️
nightly ?
unittest 19.16% <0.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
.../bigquery/connector/common/ReadSessionCreator.java 0.00% <0.00%> (ø)
...com/google/cloud/spark/bigquery/BigQueryUtil.scala 46.15% <0.00%> (-3.85%) ⬇️
...ud/spark/bigquery/pushdowns/BigQueryStrategy.scala 50.84% <0.00%> (-2.38%) ⬇️
...bigquery/pushdowns/SparkBigQueryPushdownUtil.scala 68.96% <0.00%> (-1.04%) ⬇️
...ud/spark/bigquery/pushdowns/BigQuerySQLQuery.scala 89.23% <0.00%> (-0.33%) ⬇️
...loud/spark/bigquery/pushdowns/SortLimitQuery.scala 95.23% <0.00%> (-0.22%) ⬇️
.../bigquery/pushdowns/SparkExpressionConverter.scala 91.19% <0.00%> (-0.05%) ⬇️
...le/cloud/spark/bigquery/BigQueryStreamWriter.scala 0.00% <0.00%> (ø)
...le/cloud/spark/bigquery/examples/Shakespeare.scala 0.00% <0.00%> (ø)
...gle/cloud/spark/bigquery/BigQueryWriteHelper.scala 2.53% <0.00%> (+0.23%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b96f81...377abbb. Read the comment docs.

@davidrabinowitz davidrabinowitz merged commit a639aab into GoogleCloudDataproc:master Jun 9, 2022
@davidrabinowitz davidrabinowitz deleted the default-parallelism-from-spark branch June 9, 2022 22:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants