Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-14237: [R] [CI] Disable altrep in R <= 3.5 #11404

Closed
wants to merge 10 commits into from

Conversation

thisisnic
Copy link
Member

Added a new helper function to skip tests on particular R versions

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@thisisnic
Copy link
Member Author

@github-actions crossbow submit test-r-versions

@github-actions
Copy link

Revision: 935a966

Submitted crossbow builds: ursacomputing/crossbow @ actions-937

Task Status
test-r-versions Github Actions

@jonkeane
Copy link
Member

Hmm, so it looks like the 3.5 is failing for partial matching issues separate from the altrep stuff.

for 3.4 and below, I think I see what's happening now which is that altrep.cpp is only (fully) built if R has altrep, but then we reference a function from that in https://github.com/apache/arrow/blob/master/r/src/array_to_vector.cpp#L69, so on 3.4 altrep.hpp has nothing in it (since most/all of the file was ignored), and therefore altrep::MakeAltrepVector is not defined

@jonkeane
Copy link
Member

And that partial match error might actually be related to https://bugs.r-project.org/show_bug.cgi?id=17449

@thisisnic
Copy link
Member Author

Assuming that we tuned on partial matching in a recently merged PR, I guess we could turn it off for R < 3.5.0, but not sure what would make sense to do about the altrep problem.

@jonkeane
Copy link
Member

@github-actions crossbow submit test-r-versions

@github-actions
Copy link

Revision: 2f244a3

Submitted crossbow builds: ursacomputing/crossbow @ actions-938

Task Status
test-r-versions Github Actions

@jonkeane
Copy link
Member

@github-actions crossbow submit test-r-versions

@github-actions
Copy link

Revision: 1d4e7bf

Submitted crossbow builds: ursacomputing/crossbow @ actions-942

Task Status
test-r-versions Github Actions

@jonkeane
Copy link
Member

@github-actions crossbow submit test-r-versions

@github-actions
Copy link

Revision: 714a35d

Submitted crossbow builds: ursacomputing/crossbow @ actions-945

Task Status
test-r-versions Github Actions

@jonkeane
Copy link
Member

@github-actions crossbow submit tests-r-versions

@jonkeane
Copy link
Member

@github-actions crossbow submit test-r-versions

@github-actions
Copy link

Revision: 9517709

Submitted crossbow builds: ursacomputing/crossbow @ actions-946

Task Status
test-r-versions Github Actions

@jonkeane
Copy link
Member

@github-actions crossbow submit test-r-versions

@github-actions
Copy link

Revision: 9bc470e

Submitted crossbow builds: ursacomputing/crossbow @ actions-947

Task Status
test-r-versions Github Actions

// try altrep first
// try altrep first
#if defined(HAS_ALTREP)
void Init_Altrep_classes(DllInfo * dll);
SEXP alt = altrep::MakeAltrepVector(chunked_array_);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MakeAltrepVector() is defined no matter what so that only conditional compiling would only happen in altrep.cpp and so it always returns R_NilValue when altrep is not defined, which signals that an altrep vector could not be made and so to do the normal thing

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, these are the errors we were seeing on compilation without adding the conditional here, is there some other way to get past these without a conditional here?

array_to_vector.cpp: In member function ‘SEXPREC* arrow::r::Converter::ScheduleConvertTasks(arrow::r::RTasks&, std::shared_ptr<arrow::r::Converter>)’:
array_to_vector.cpp:69:16: error: ‘altrep’ has not been declared
     SEXP alt = altrep::MakeAltrepVector(chunked_array_);
                ^~~~~~
array_to_vector.cpp: In member function ‘SEXPREC* arrow::r::Converter::MaybeAltrep()’:
array_to_vector.cpp:124:31: error: ‘altrep’ has not been declared
   SEXP MaybeAltrep() { return altrep::MakeAltrepVector(chunked_array_); }
                               ^~~~~~

https://github.com/ursacomputing/crossbow/runs/3885687364?check_suite_focus=true#step:7:1423

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I probably have misspend something. I'll have a look.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 thank you!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the bit in arrow_types.h should be:

namespace altrep {

#if defined(HAS_ALTREP)
void Init_Altrep_classes(DllInfo* dll);
#endif

SEXP MakeAltrepVector(const std::shared_ptr<ChunkedArray>& chunked_array);

}  // namespace altrep

i.e. the SEXP MakeAltrepVector be out of the #if defined(HAS_ALTREP)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#if defined(HAS_ALTREP)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and then there should be no need for using #if defined(HAS_ALTREP) in other places

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these! I've managed to eliminate all (or almost all ... there was one more I want to dig in a bit more before I delete it) and it worked: https://github.com/ursacomputing/crossbow/runs/3897917054?check_suite_focus=true

Thanks again!

r/tests/testthat/test-Array.R Show resolved Hide resolved
r/tests/testthat/test-chunked-array.R Show resolved Hide resolved
@jonkeane
Copy link
Member

@github-actions crossbow submit test-r-versions

@github-actions
Copy link

Revision: 03506da

Submitted crossbow builds: ursacomputing/crossbow @ actions-958

Task Status
test-r-versions Github Actions

@jonkeane
Copy link
Member

@github-actions crossbow submit test-r-versions

@jonkeane
Copy link
Member

@github-actions autotune

@github-actions
Copy link

Revision: 4eaa1cb

Submitted crossbow builds: ursacomputing/crossbow @ actions-959

Task Status
test-r-versions Github Actions

r/DESCRIPTION Outdated Show resolved Hide resolved
@jonkeane jonkeane closed this in cb1f15b Oct 14, 2021
@ursabot
Copy link

ursabot commented Oct 14, 2021

Benchmark runs are scheduled for baseline = 7eba115 and contender = cb1f15b. cb1f15b is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.31% ⬆️0.0%] ursa-thinkcentre-m75q
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants