Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break parquet CI into its own workflow, standardize step names #2190

Merged
merged 7 commits into from
Jul 28, 2022

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Jul 27, 2022

Which issue does this PR close?

re #2149

Rationale for this change

Having the CI checks broken up by crate is a prerequisite for more specific triggering them

What changes are included in this PR?

  1. Break parquet CI into its own workflow
  2. Standardize testing names
  3. Run clippy on each target (remove overall workspace run)
  4. Run tests for each target (remove overall workspace run)

Are there any user-facing changes?

@@ -77,14 +77,23 @@ jobs:
uses: ./.github/actions/setup-builder
with:
rust-version: stable
- name: Test compilation with different features
- name: Check compilation
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By breaking each command into its own step it becomes easier to see which check failed on CI. The actual checks run are the same

run: |
cargo test -p arrow-flight --all-features

clippy:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed the workspace clippy check and instead ran clippy on each individual crate

@github-actions github-actions bot added the parquet Changes to the parquet crate label Jul 27, 2022
# under the License.

---
# tests for parquet crate
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests were extracted from rust.yml

@@ -94,34 +81,6 @@ jobs:
export RUSTFLAGS="-C debuginfo=0"
cargo test

clippy:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Each of the individual crate workflows now run clippy so there is no reason to run it in the workspace as well

@@ -182,6 +182,10 @@ impl RleEncoder {
self.bit_writer.bytes_written()
}

pub fn is_empty(&self) -> bool {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out clippy complains about this in parquet crate, so I fixed it

.github/workflows/parquet.yml Show resolved Hide resolved
@alamb alamb changed the title Break parquet CI into its own workflow, standardize names, Break parquet CI into its own workflow, standardize step names Jul 27, 2022
@codecov-commenter
Copy link

codecov-commenter commented Jul 27, 2022

Codecov Report

Merging #2190 (2b12b90) into master (42135a9) will decrease coverage by 0.00%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #2190      +/-   ##
==========================================
- Coverage   82.61%   82.60%   -0.01%     
==========================================
  Files         239      239              
  Lines       62255    62257       +2     
==========================================
- Hits        51430    51428       -2     
- Misses      10825    10829       +4     
Impacted Files Coverage Δ
parquet/src/encodings/rle.rs 92.42% <0.00%> (-0.37%) ⬇️
parquet_derive/src/parquet_field.rs 65.98% <0.00%> (-0.23%) ⬇️
parquet/src/encodings/encoding.rs 93.43% <0.00%> (-0.20%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us.

@alamb alamb marked this pull request as ready for review July 27, 2022 14:20
@alamb alamb added the development-process Related to development process of arrow-rs label Jul 27, 2022
@alamb alamb requested a review from tustvold July 27, 2022 17:01
@tustvold tustvold merged commit 9d8f700 into apache:master Jul 28, 2022
@ursabot
Copy link

ursabot commented Jul 28, 2022

Benchmark runs are scheduled for baseline = 7199b1b and contender = 9d8f700. 9d8f700 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-rs-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@alamb alamb deleted the alamb/break_out_parquet_ci branch July 28, 2022 13:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development-process Related to development process of arrow-rs parquet Changes to the parquet crate
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants