New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
adding prefetching_iterator and related tests used for prefetching containers within lambda functions #2179
Conversation
The files you added miss a newline at the end. This needs to get fixed. |
for(std::size_t i=0; i<vector_size; ++i) range[i] = i + begin; | ||
it_begin = range.begin(); | ||
it_end = range.begin() + vector_size - 1; | ||
chunk_size = p_factor * 64ul / sizeof(T); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
where does the 64 come from?
sithhell : Honestly we are thinking about having prefetching as a policy parameter, so it can be used by other algorithms too. Also, current prefetching could be apply easily on transform algorithm too. chunk_size used for prefetching here, is determined based on the cache line size. On x86 cache lines are 64 bytes. As far as I know, there is not any command to find out cache line sizes on different architectures. So, I used 64 as a default for determining chunk_size here. Instead of using _mm_prefetch, reading the next element in the container can be used for prefetching next chunk_size, but it is more expensive than using _mm_prefetch. Do you have any suggestion? According to the range index, first I used boost::irange, but I found it to have a high overhead which prevents scaling. So I changed it to vector, and it scales much better. I will try using array_view instead of initializer_list. |
Can anyone please tell me why my pull request fails? |
@zkhatami88 inspect is unhappy with the code: https://circle-artifacts.com/gh/STEllAR-GROUP/hpx/3961/artifacts/0/tmp/circle-artifacts.sEApRbB/hpx_inspect_report.html |
Am 25.05.2016 3:27 nachm. schrieb "Zahra Khatami" <notifications@github.com
It would be really nice to have it in its own headers then.
Turn it into a configurable (through cmake) define).
As said. This works only for x86. It will not work on other platforms.
Having a vector sounds wasteful. That's all I'm saying.
|
@@ -71,11 +72,13 @@ namespace hpx { namespace parallel { HPX_INLINE_NAMESPACE(v1) | |||
[f, proj](std::size_t /*part_index*/, | |||
InIter part_begin, std::size_t part_size) mutable | |||
{ | |||
typedef typename util::loop_n_iterator_mapping< | |||
InIter>::type iterator_type; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This could be combined with the loop_n<>
template.
@zkhatami88 I will have a look |
|
||
namespace hpx { namespace parallel { namespace util | ||
{ | ||
/////////////////////////////////////////////////////////////////////////// | ||
namespace detail | ||
{ | ||
/////////////////////////////////////////////////////////////////////// | ||
// Helper class to repeatedly call a function starting from a given | ||
// Helper class to repeatedly call a function starting from a givena |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
there is a typo here.
@zkhatami88 FWIW, I'm working on fixing things here, feel free to chime in. |
Making things work...
@zkhatami88 I'd be happy to merge this once the conflicts with our master branch are resolved. |
@hkaiser sure, that's a good idea. Also, the test is failed, I am going to fix it too. |
My comments haven't been addressed.
|
@sithhell Thank you so much for your comments. I fixed typo in loop.hpp and in prefetching.hpp, sequencer is implemented to have iteration over pefetching containers in each time step instead of having recursion. |
@@ -21,7 +21,7 @@ | |||
#include <hpx/parallel/algorithms/detail/is_negative.hpp> | |||
#include <hpx/parallel/util/detail/algorithm_result.hpp> | |||
#include <hpx/parallel/util/foreach_partitioner.hpp> | |||
#include <hpx/parallel/util/loop.hpp> | |||
#include <hpx/parallel/util/prefetching.hpp> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This change is not correct. We should not pull in the prefetch.hpp
header into the for_each.hpp
. If a user wants to use prefetching he/she should explicitly include an appropriate header file. Also, for_each depends on loop.hpp
, so why did you remove it?
@hkaiser loop.hpp is included in the header in prefetching.hpp, so by having prefetching.hpp in the headers in for_each.hpp, loop.hpp will be included as well. |
@zkhatami88 for_each depends on If including |
@hkaiser oh ok. I am going to fix it and I will push another commit. |
@@ -223,22 +226,15 @@ namespace hpx { namespace parallel { HPX_INLINE_NAMESPACE(v1) | |||
sequential(ExPolicy, InIter first, InIter last, F && f, | |||
Proj && proj) | |||
{ | |||
typedef typename util::detail::loop<InIter>::type it_type; | |||
|
|||
return util::loop(first, last, | |||
[&f, &proj](Iter curr) | |||
{ | |||
f(hpx::util::invoke(proj, *curr)); | |||
}); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hkaiser :
Test fails because of this line, that:
/hpx/hpx/parallel/algorithms/for_each.hpp:231:24: error: no viable conversion from returned value of type hpx::local_raw_vector_iterator<int, _gnu_cxx::_normal_iterator<int *, std::vector<int, std::allocator<int> > > >
to function return type hpx::local_vector_iterator<int>
In all lambdas in this file, hpx::util::invoke(f, hpx::util::invoke(proj, *curr));
is used except this line, shouldn't we change f(hpx::util::invoke(proj, *curr));
to hpx::util::invoke(f, hpx::util::invoke(proj, *curr));
as well?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll have a look
This is superseded by #2228 |
prefetching_iterator is created within loop.hpp to prefetch all containers within lambda functions.
for_each.hpp is changed as well to work with loop_n within loop.hpp to call prefetching_iterator.
foreach_prefetching.cpp, foreach_prefetching_executors.cpp and foreach_tests_prefetching.hpp are related tests created for testing the prefetching_iterator performance.