Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-14074: [C++][Compute] C++ consumer of compute IR #11384

Closed
wants to merge 27 commits into from

Conversation

bkietz
Copy link
Member

@bkietz bkietz commented Oct 11, 2021

Provide a consumer for compute IR.

The consumer produces Declaration objects from buffers of Compute IR. This allows us to specify mappings from IR to ExecNode graphs without needing those specific nodes in the registry or requiring the registered factories to support all features declared in IR.

This is tested by using the flatbuffer compiler to convert JSON into the corresponding binary representation.

@github-actions
Copy link

@apache apache deleted a comment from github-actions bot Oct 12, 2021
@bkietz bkietz force-pushed the 14074-Sketch-a-C-consumer-of-co branch 2 times, most recently from d04021b to 1d4c37e Compare October 28, 2021 15:59
@bkietz bkietz marked this pull request as ready for review October 29, 2021 13:52
@bkietz bkietz force-pushed the 14074-Sketch-a-C-consumer-of-co branch from 4ac1d95 to 9c9d62e Compare October 29, 2021 14:10
@bkietz bkietz requested a review from cpcloud October 29, 2021 14:49
cpp/cmake_modules/BuildUtils.cmake Outdated Show resolved Hide resolved
cpp/src/arrow/compute/api_vector.h Show resolved Hide resolved
cpp/src/arrow/compute/exec/CMakeLists.txt Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/exec_plan.h Outdated Show resolved Hide resolved
cpp/src/arrow/compute/exec/ir_consumer.h Show resolved Hide resolved
cpp/src/arrow/ipc/metadata_internal.h Show resolved Hide resolved
@@ -302,7 +302,7 @@ struct TemporalScalar : internal::PrimitiveScalar<T> {
using internal::PrimitiveScalar<T>::PrimitiveScalar;
using ValueType = typename TemporalScalar<T>::ValueType;

explicit TemporalScalar(ValueType value, std::shared_ptr<DataType> type)
TemporalScalar(ValueType value, std::shared_ptr<DataType> type)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably just my seriously rusty C++, but is this to allow copy-list initialization somewhere else?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the constructor could accept a single argument, explicit would allow implicit conversion to TemporalScalar. Since the constructor requires two arguments the keyword is fully redundant (which clang-tidy nagged me about when I opened the file, prompting the edit)

cpp/src/arrow/type.h Outdated Show resolved Hide resolved
@@ -578,6 +607,19 @@ Result<std::vector<PlatformFilename>> ListDir(const PlatformFilename& dir_path)

#endif

Status SetWorkingDir(const PlatformFilename& dir_path) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Setting the working directory is a fraught in the presence of multiple threads.

You can almost always accomplish what you need to do after running chdir in a thread-safe way without chdir.

So, with that in mind: why is this function necessary?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can be removed, will do

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FTR, I used this because flatc always outputs the binary representation to the current working directory (it will not accept the -o option) and it seemed neater to ensure we were always writing that to the temporary directory we allocated

cpp/src/arrow/util/io_util.h Outdated Show resolved Hide resolved
@bkietz
Copy link
Member Author

bkietz commented Nov 5, 2021

@cpcloud I think I've addressed all your comments, is this good to merge?

@bkietz bkietz closed this in 528625e Nov 5, 2021
@ursabot
Copy link

ursabot commented Nov 5, 2021

Benchmark runs are scheduled for baseline = dce1415 and contender = 528625e. 528625e is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed ⬇️1.03% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.36% ⬆️0.04%] ursa-thinkcentre-m75q
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True

@bkietz bkietz deleted the 14074-Sketch-a-C-consumer-of-co branch November 30, 2021 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants