Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-10392: [C++][Gandiva] Avoid string copy while evaluating IN expression #8530

Closed
wants to merge 3 commits into from

Conversation

pprudhvi
Copy link
Contributor

No description provided.

@pprudhvi
Copy link
Contributor Author

@github-actions crossbow submit -g gandiva

@github-actions
Copy link

@github-actions
Copy link

Revision: 84515e8

Submitted crossbow builds: ursa-labs/crossbow @ actions-680

Task Status
gandiva-jar-osx TravisCI
gandiva-jar-xenial TravisCI

@pprudhvi
Copy link
Contributor Author

pprudhvi commented Oct 27, 2020

still couldn't avoid string copy - the copy is now happening inside HashTable

#0  0x00007f8b9e690159 in __lll_unlock_wake_private () from /lib64/libc.so.6
#1  0x00007f8b9e60df78 in _L_unlock_14902 () from /lib64/libc.so.6
#2  0x00007f8b9e60afe5 in malloc () from /lib64/libc.so.6
#3  0x00007f8b9d3d20cd in operator new(unsigned long) () from /lib64/libstdc++.so.6
#4  0x00007f8b9d430c79 in std::string::_Rep::_S_create(unsigned long, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#5  0x00007f8b9d432531 in char* std::string::_S_construct<char const*>(char const*, char const*, std::allocator<char> const&, std::forward_iterator_tag) () from /lib64/libstdc++.so.6
#6  0x00007f8b9d4325ed in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) () from /lib64/libstdc++.so.6
#7  0x00007f8b595c7c4a in std::_Hashtable<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, nonstd::sv_lite::basic_string_view<char, std::char_traits<char> >, std::allocator<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Identity, std::equal_to<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::hash<nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > >, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, true, true> >::count(nonstd::sv_lite::basic_string_view<char, std::char_traits<char> > const&) const () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87
#8  0x00007f8b595c7258 in gdv_fn_in_expr_lookup_utf8 () from /data/yarn/data/usercache/dremio/appcache/application_1603758047742_0001/container_1603758047742_0001_01_000003/tmp/libgandiva_jni.so87f06318-3d22-4bda-be14-ef2458081d87

@bkietz , any reason why the copy is happening even when we are looking for a string_view in hashtable<string_view>?

@pprudhvi
Copy link
Contributor Author

Turns out string_view hash function internally creates a string
https://github.com/martinmoene/string-view-lite/blob/master/include/nonstd/string_view.hpp#L1583

@pprudhvi
Copy link
Contributor Author

@github-actions crossbow submit -g gandiva

@github-actions
Copy link

Revision: eec6dfd

Submitted crossbow builds: ursa-labs/crossbow @ actions-682

Task Status
gandiva-jar-osx TravisCI
gandiva-jar-xenial TravisCI

@bkietz bkietz self-requested a review October 27, 2020 13:38
Copy link
Member

@bkietz bkietz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me, any idea what is causing the linker errors?

@pprudhvi
Copy link
Contributor Author

MinGW link problem is caused by msys2/MINGW-packages#7170 .
It'll be fixed in upstream.

#8521 (comment)

@pravindra pravindra closed this in 42cb805 Oct 28, 2020
pprudhvi added a commit to pprudhvi/arrow that referenced this pull request Oct 28, 2020
…ression

Closes apache#8530 from pprudhvi/stringalloc

Authored-by: Prudhvi Porandla <prudhvi.porandla@icloud.com>
Signed-off-by: Pindikura Ravindra <ravindra@dremio.com>
@pprudhvi pprudhvi deleted the stringalloc branch October 28, 2020 16:39
pprudhvi added a commit to pprudhvi/arrow that referenced this pull request Oct 28, 2020
…ression

Closes apache#8530 from pprudhvi/stringalloc

Authored-by: Prudhvi Porandla <prudhvi.porandla@icloud.com>
Signed-off-by: Pindikura Ravindra <ravindra@dremio.com>
pprudhvi added a commit to dremio/arrow that referenced this pull request Oct 28, 2020
…ression (#19)

Closes apache#8530 from pprudhvi/stringalloc

Authored-by: Prudhvi Porandla <prudhvi.porandla@icloud.com>
Signed-off-by: Pindikura Ravindra <ravindra@dremio.com>
GeorgeAp pushed a commit to sirensolutions/arrow that referenced this pull request Jun 7, 2021
…ression

Closes apache#8530 from pprudhvi/stringalloc

Authored-by: Prudhvi Porandla <prudhvi.porandla@icloud.com>
Signed-off-by: Pindikura Ravindra <ravindra@dremio.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants