Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MINOR] Update arrow-rs dependencies #33

Merged
merged 3 commits into from
Apr 23, 2021
Merged

Conversation

alamb
Copy link
Contributor

@alamb alamb commented Apr 22, 2021

Bring in the latest arrow-rs branch

I will eventually automate this process, but for now I will do it manually.

Copy link
Member

@jorgecarleitao jorgecarleitao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing it manually is also fine; this is easy this time because there weren't any backward incomp. changes.

@Dandandan
Copy link
Contributor

Dandandan commented Apr 22, 2021

@alamb I think you also need to update the other crates within the repo to match the version

@alamb
Copy link
Contributor Author

alamb commented Apr 22, 2021

@alamb I think you also need to update the other crates

Good call @Dandandan will do

@andygrove
Copy link
Member

Please take a look at the related #23

@andygrove
Copy link
Member

@alamb Would it make sense for DataFusion to re-export Arrow so that dependent crates automatically get the same Arrow version?

@returnString
Copy link
Contributor

@alamb Would it make sense for DataFusion to re-export Arrow so that dependent crates automatically get the same Arrow version?

This would make things much easier for consumers, +1 from me :)

On the automation front, has anyone else used Dependabot for this kind of thing before? I've started using it recently for Python, Rust, and the actual actions in my GHA workflows, and it's really nice, e.g. open dependency PRs for my convergence crates as defined with some trivial metadata.

@alamb
Copy link
Contributor Author

alamb commented Apr 23, 2021

@alamb Would it make sense for DataFusion to re-export Arrow so that dependent crates automatically get the same Arrow version?
This would make things much easier for consumers, +1 from me :)

Good idea @andygrove and @returnString -- I filed #36 to do so

On the automation front, has anyone else used Dependabot for this kind of thing before?

I have not, but it sounds like a good idea to me. Filed #37

@alamb alamb merged commit 9d720f4 into apache:master Apr 23, 2021
@houqp houqp added the development-process Related to development process of DataFusion label Jul 29, 2021
@alamb alamb deleted the alamb/bump-deps branch October 6, 2022 18:29
andygrove added a commit that referenced this pull request Jan 12, 2023
* Initial commit

* initial commit

* failing test

* table scan projection

* closer

* test passes, with some hacks

* use DataFrame (#2)

* update README

* update dependency

* code cleanup (#3)

* Add support for Filter operator and BinaryOp expressions (#4)

* GitHub action (#5)

* Split code into producer and consumer modules (#6)

* Support more functions and scalar types (#7)

* Use substrait 0.1 and datafusion 8.0 (#8)

* use substrait 0.1

* use datafusion 8.0

* update datafusion to 10.0 and substrait to 0.2 (#11)

* Add basic join support (#12)

* Added fetch support (#23)

Added fetch to consumer

Added limit to producer

Added unit tests for limit

Added roundtrip_fill_none() for testing when None input can be converted to 0

Update src/consumer.rs

Co-authored-by: Andy Grove <andygrove73@gmail.com>

Co-authored-by: Andy Grove <andygrove73@gmail.com>

* Upgrade to DataFusion 13.0.0 (#25)

* Add sort consumer and producer (#24)

Add consumer

Add producer and test

Modified error string

* Add serializer/deserializer (#26)

* Add plan and function extension support (#27)

* Add plan and function extension support

* Removed unwraps

* Implement GROUP BY (#28)

* Add consumer, producer and tests for aggregate relation

Change function extension registration from absolute to relative anchor
(reference)

Remove operator to/from reference

* Fixed function registration bug

* Add test

* Addressed PR comments

* Changed field reference from mask to direct reference (#29)

* Changed field reference from masked reference to direct reference

* Handle unsupported case (struct with child)

* Handle SubqueryAlias (#30)

Fixed aggregate function register bug

* Add support for SELECT DISTINCT (#31)

Add test case

* Implement BETWEEN (#32)

* Add case (#33)

* Implement CASE WHEN

* Add more case to test

* Addressed comments

* feat: support explicit catalog/schema names in ReadRel (#34)

* feat: support explicit catalog/schema names in ReadRel

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* fix: use re-exported expr crate

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>

* move files to subfolder

* RAT

* remove rust.yaml

* revert .gitignore changes

* tomlfmt

* tomlfmt

Signed-off-by: Ruihang Xia <waynestxia@gmail.com>
Co-authored-by: Daniël Heres <danielheres@gmail.com>
Co-authored-by: JanKaul <jankaul@mailbox.org>
Co-authored-by: nseekhao <37189615+nseekhao@users.noreply.github.com>
Co-authored-by: Ruihang Xia <waynestxia@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development-process Related to development process of DataFusion
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants