ARROW-10929: [Rust] Change CI to use Stable Rust #8930

alamb · 2020-12-15T21:19:02Z

This is a cherry-pick of https://github.com/jorgecarleitao/arrow/commit/ca66d6d945e265dd2c83464bd80ff1dd7d231f7c by @jorgecarleitao

It runs all tests except the simd using stable -- The SIMD feature still require nightly rust, but the default features do not (after #8698)

Update: It also silences a few clippy lints which start complaining on stable -- I'll comment inline

github-actions · 2020-12-15T21:29:35Z

https://issues.apache.org/jira/browse/ARROW-10929

alamb · 2020-12-15T21:34:32Z

🤔 the windows CI build is failing when using stable:


failures:

---- encodings::encoding::tests::test_bool stdout ----
thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet\src\util\bit_util.rs:73:18

alamb · 2020-12-16T12:30:55Z

Hmm, I ran everything under valgrind locally and it did not detect any errors and the test has passed on windows a subsequent run. However, now I am getting a clippy error. I will fix that.

andygrove · 2020-12-16T17:15:23Z

thinking the windows CI build is failing when using stable:


failures:

---- encodings::encoding::tests::test_bool stdout ----
thread 'encodings::encoding::tests::test_bool' panicked at 'Invalid byte when reading bool', parquet\src\util\bit_util.rs:73:18

I just saw this locally too and it passed on a second run. I filed https://issues.apache.org/jira/browse/ARROW-10943

alamb · 2020-12-16T22:43:28Z

rust/datafusion/src/lib.rs

@@ -17,7 +17,6 @@
 #![warn(missing_docs)]
 // Clippy lints, some should be disabled incrementally
 #![allow(
-    clippy::field_reassign_with_default,


this does not exist in stable clippy

alamb · 2020-12-16T22:44:39Z

rust/datafusion/src/logical_plan/plan.rs

@@ -561,6 +561,7 @@ impl From<&PlanType> for String {

 /// Represents some sort of execution plan, in String form
 #[derive(Debug, Clone, PartialEq)]
+#[allow(clippy::rc_buffer)]


Clippy is complaining about https://rust-lang.github.io/rust-clippy/master/index.html#rc_buffer - but I think in this case using the Arc is really about cheaply copying the String rather than it being mutable. The same happens for the other places I disabled this lint in this PR

I am not enough of a Rust expert to know if there is some better way to pass around a cheaply cloneable Vec or String that would appease Clippy.

Did it give an alternative in the suggestion?

Looks like Arc<str> and Arc<[T]> should be the alternatives

I agree @Dandandan . I changed then a couple of times, but since the focus of this PR was the CI in stable, which includes build, tests, etc. I considered clippy to be a minor hassle that we can deal with later.

@Dandandan -- It did give alternatives but I did not figure out how to apply it.

The actually clippy lint didn't make a whole lot of sense to me as it seems to be focused on the mutability of the Arc/Rc's elements, which doesn't seem to apply here. From my reading of this code and the other callsites, the use case of using Arc<String> is not mutability of the contained String but rather cheaply copying it around.

Using Arc<str> doesn't make sense as the actual String is allocated at runtime (it is the formatted version of an explain plan)

There is some discussion about why it can make sense to use Arc<str> over Arc<String>:
https://users.rust-lang.org/t/arc-string-vs-arc-str/30571

of course, difference is minimal, only saving a bit of indirection/some bytes.

You can use into to convert a String or &str into a Arc<str> or Rc<str>
https://doc.rust-lang.org/std/sync/struct.Arc.html#impl-From%3CString%3E

Example:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=eb6db938ae03b001edca59aba1512d33

codecov-io · 2020-12-16T23:03:25Z

Codecov Report

Merging #8930 (5788152) into master (d65ba4e) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##           master    #8930   +/-   ##
=======================================
  Coverage   83.25%   83.25%           
=======================================
  Files         196      196           
  Lines       48116    48116           
=======================================
  Hits        40059    40059           
  Misses       8057     8057

Impacted Files	Coverage Δ
rust/datafusion/src/logical_plan/expr.rs	`76.76% <ø> (ø)`
rust/datafusion/src/logical_plan/plan.rs	`87.93% <ø> (ø)`
rust/parquet/src/util/cursor.rs	`83.18% <ø> (ø)`
rust/parquet/src/util/memory.rs	`89.57% <ø> (ø)`
rust/arrow/src/array/array_struct.rs	`88.68% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d65ba4e...5788152. Read the comment docs.

jorgecarleitao

LGTM. Thanks for taking the time to address this. Good stuff!

@jorgecarleitao

This is a cherry-pick of https://github.com/jorgecarleitao/arrow/commit/ca66d6d945e265dd2c83464bd80ff1dd7d231f7c by @jorgecarleitao It runs all tests except the simd using `stable` -- The SIMD feature still require nightly rust, but the default features do not (after apache#8698) Update: It also silences a few clippy lints which start complaining on stable -- I'll comment inline Closes apache#8930 from alamb/ARROW-10929-stable-ci Lead-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Co-authored-by: Jorge C. Leitao <jorgecarleitao@gmail.com> Signed-off-by: Jorge C. Leitao <jorgecarleitao@gmail.com>

alamb changed the title ~~ARROW-10929 [Rust] Change CI to use Stable Rust~~ ARROW-10929: [Rust] Change CI to use Stable Rust Dec 15, 2020

alamb force-pushed the ARROW-10929-stable-ci branch from 0493360 to f3718ff Compare December 16, 2020 11:47

jorgecarleitao and others added 3 commits December 16, 2020 17:19

Migrated CI to stable rust.

66f7d8b

Fix clippy lint

ebe8924

Silence over zealous clippy lint

5788152

alamb force-pushed the ARROW-10929-stable-ci branch from 4df2551 to 5788152 Compare December 16, 2020 22:42

alamb commented Dec 16, 2020

View reviewed changes

alamb requested review from sunchao, jorgecarleitao and nevi-me December 17, 2020 00:32

nevi-me approved these changes Dec 21, 2020

View reviewed changes

jorgecarleitao approved these changes Dec 21, 2020

View reviewed changes

jorgecarleitao closed this in 970fc21 Dec 21, 2020

alamb deleted the ARROW-10929-stable-ci branch December 21, 2020 12:22

This was referenced Dec 21, 2020

[Rust] Migrate CI tests to stable rust #26857

Closed

[Rust] rustfmt cleanup #26952

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

ARROW-10929: [Rust] Change CI to use Stable Rust #8930

ARROW-10929: [Rust] Change CI to use Stable Rust #8930

Uh oh!

alamb commented Dec 15, 2020 •

edited

Loading

Uh oh!

github-actions bot commented Dec 15, 2020

Uh oh!

alamb commented Dec 15, 2020

Uh oh!

alamb commented Dec 16, 2020

Uh oh!

andygrove commented Dec 16, 2020

Uh oh!

alamb Dec 16, 2020

Uh oh!

alamb Dec 16, 2020

Uh oh!

Dandandan Dec 21, 2020

Uh oh!

jorgecarleitao Dec 21, 2020

Uh oh!

alamb Dec 21, 2020

Uh oh!

Dandandan Dec 21, 2020

Uh oh!

codecov-io commented Dec 16, 2020

Uh oh!

jorgecarleitao left a comment

Uh oh!

Uh oh!

ARROW-10929: [Rust] Change CI to use Stable Rust #8930

ARROW-10929: [Rust] Change CI to use Stable Rust #8930

Uh oh!

Conversation

alamb commented Dec 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Dec 15, 2020

Uh oh!

alamb commented Dec 15, 2020

Uh oh!

alamb commented Dec 16, 2020

Uh oh!

andygrove commented Dec 16, 2020

Uh oh!

alamb Dec 16, 2020

Choose a reason for hiding this comment

Uh oh!

alamb Dec 16, 2020

Choose a reason for hiding this comment

Uh oh!

Dandandan Dec 21, 2020

Choose a reason for hiding this comment

Uh oh!

jorgecarleitao Dec 21, 2020

Choose a reason for hiding this comment

Uh oh!

alamb Dec 21, 2020

Choose a reason for hiding this comment

Uh oh!

Dandandan Dec 21, 2020

Choose a reason for hiding this comment

Uh oh!

codecov-io commented Dec 16, 2020

Codecov Report

Uh oh!

jorgecarleitao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

alamb commented Dec 15, 2020 •

edited

Loading