Skip to content

Upgrade Arrow to 37#810

Merged
eddyxu merged 6 commits intomainfrom
changhiskhan/upgrade-arrow
Apr 27, 2023
Merged

Upgrade Arrow to 37#810
eddyxu merged 6 commits intomainfrom
changhiskhan/upgrade-arrow

Conversation

@changhiskhan
Copy link
Copy Markdown
Contributor

Can't go to 38 yet even though it's out. Datafusion is on 37 latest.

We need 37 for Will's fix for handling schema metadata.

Copy link
Copy Markdown
Member

@eddyxu eddyxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems fine to me, other than a few concept changes , i assume that is what is done in Arrow

Comment thread rust/src/arrow/linalg.rs
for idx in chosen.iter() {
let s = self.data.slice(idx * dim, dim);
builder.append_slice(as_primitive_array::<Float32Type>(s.as_ref()).values());
builder.append_slice(s.values());
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Slice is strong typed now?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, must be part of the effort towards arrow-rs arrow2 merging?

Comment thread rust/src/arrow/linalg.rs
vec![27.46873242, 22.64318501, 8.55838823, 5.9857232, 2.01489966].as_slice(),
epsilon = 0.0001,
);
u.data()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why can we just use slice comparision here?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

complains about ScalarBuffe not having AbsDiffEq

@eddyxu eddyxu merged commit 01ad7ac into main Apr 27, 2023
@eddyxu eddyxu deleted the changhiskhan/upgrade-arrow branch April 27, 2023 21:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants