GH-34775: [R] arrow_table: as.data.frame() sometimes returns a tbl and sometimes a data.frame #35173
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
Component: R
awaiting committer review
… return data.frames
…mes and update tests to use it
thisisnic
force-pushed
the
GH-34775_new_as_df
branch
from
8a3d84b
to
652952c
Compare
paleolimbot
approved these changes
May 3, 2023
There was a problem hiding this comment.
This looks good! It seems like a cleaner solution than what we currently have. I like the idea of dropping metadata on the way in where possible because I seem to remember that we can skip some calls from C++ into R if there is no metadata to restore which speeds things up a bit.
github-actions
bot
added
awaiting merge
|
Benchmark runs are scheduled for baseline = 2ee0345 and contender = 205ceb9. 205ceb9 is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
|
['Python', 'R'] benchmarks have high level of regressions. |
liujiacheng777
pushed a commit
to LoongArch-Python/arrow
that referenced
this pull request
May 11, 2023
…tbl and sometimes a data.frame (apache#35173) Features of this PR: * Ensures that calling `as.data.frame()` on Arrow objects returns base R `data.frame` objects. * Drops the `class` attribute metadata of input objects of `data.frame` class (i.e. that don't have inherit from any additional classes other than `data.frame`). This results in us sacrificing roundtrip class fidelity for `data.frame` objects (i.e. if we input a base R data.frame, convert it to an Arrow Table, and then convert it back to R, we get a tibble). However, we now have consistency in the type of returned objects, retain roundtrip fidelity for other (non-class) metadata, and guarantee that `as.data.frame()` returns a base R data.frame. Users who wish to input and return a `data.frame` object can call `as.data.frame()` on the returned object. * Implements `dplyr::collect()` for StructArrays so that these objects can still be returned as tibbles if needed. * Renames `expect_data_frame()` to `expect_equal_data_frame()` for clarity, and updates it to convert both the object and expected object to data.frames. * Closes: apache#34775 Authored-by: Nic Crane <thisisnic@gmail.com> Signed-off-by: Nic Crane <thisisnic@gmail.com>
ArgusLi
pushed a commit
to Bit-Quill/arrow
that referenced
this pull request
May 15, 2023
…tbl and sometimes a data.frame (apache#35173) Features of this PR: * Ensures that calling `as.data.frame()` on Arrow objects returns base R `data.frame` objects. * Drops the `class` attribute metadata of input objects of `data.frame` class (i.e. that don't have inherit from any additional classes other than `data.frame`). This results in us sacrificing roundtrip class fidelity for `data.frame` objects (i.e. if we input a base R data.frame, convert it to an Arrow Table, and then convert it back to R, we get a tibble). However, we now have consistency in the type of returned objects, retain roundtrip fidelity for other (non-class) metadata, and guarantee that `as.data.frame()` returns a base R data.frame. Users who wish to input and return a `data.frame` object can call `as.data.frame()` on the returned object. * Implements `dplyr::collect()` for StructArrays so that these objects can still be returned as tibbles if needed. * Renames `expect_data_frame()` to `expect_equal_data_frame()` for clarity, and updates it to convert both the object and expected object to data.frames. * Closes: apache#34775 Authored-by: Nic Crane <thisisnic@gmail.com> Signed-off-by: Nic Crane <thisisnic@gmail.com>
rtpsw
pushed a commit
to rtpsw/arrow
that referenced
this pull request
May 16, 2023
…tbl and sometimes a data.frame (apache#35173) Features of this PR: * Ensures that calling `as.data.frame()` on Arrow objects returns base R `data.frame` objects. * Drops the `class` attribute metadata of input objects of `data.frame` class (i.e. that don't have inherit from any additional classes other than `data.frame`). This results in us sacrificing roundtrip class fidelity for `data.frame` objects (i.e. if we input a base R data.frame, convert it to an Arrow Table, and then convert it back to R, we get a tibble). However, we now have consistency in the type of returned objects, retain roundtrip fidelity for other (non-class) metadata, and guarantee that `as.data.frame()` returns a base R data.frame. Users who wish to input and return a `data.frame` object can call `as.data.frame()` on the returned object. * Implements `dplyr::collect()` for StructArrays so that these objects can still be returned as tibbles if needed. * Renames `expect_data_frame()` to `expect_equal_data_frame()` for clarity, and updates it to convert both the object and expected object to data.frames. * Closes: apache#34775 Authored-by: Nic Crane <thisisnic@gmail.com> Signed-off-by: Nic Crane <thisisnic@gmail.com>
This was referenced Aug 3, 2023
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Features of this PR:
Ensures that calling
as.data.frame()on Arrow objects returns base Rdata.frameobjects.Drops the
classattribute metadata of input objects ofdata.frameclass (i.e. that don't have inherit from any additional classes other thandata.frame). This results in us sacrificing roundtrip class fidelity fordata.frameobjects (i.e. if we input a base R data.frame, convert it to an Arrow Table, and then convert it back to R, we get a tibble). However, we now have consistency in the type of returned objects, retain roundtrip fidelity for other (non-class) metadata, and guarantee thatas.data.frame()returns a base R data.frame. Users who wish to input and return adata.frameobject can callas.data.frame()on the returned object.Implements
dplyr::collect()for StructArrays so that these objects can still be returned as tibbles if needed.Renames
expect_data_frame()toexpect_equal_data_frame()for clarity, and updates it to convert both the object and expected object to data.frames.Closes: [R] arrow_table: as.data.frame() sometimes returns a tbl and sometimes a data.frame #34775