Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xindexview function for advanced indexing #88

Merged
merged 4 commits into from
Feb 4, 2017

Conversation

wolfv
Copy link
Member

@wolfv wolfv commented Jan 28, 2017

I hope this doesn't come across as spam :) but I have attempted an implementation for an "index view". ie. one can provide a list of indices and those will be selected into a linear view.
Essentially, it allows for numpys advanced indexing like so:

	xt::xarray<double> rn = xt::random::rand<double>({3, 3}, 0, 1);

	auto v = xt::make_xindexview(rn, {{1,1}, {2,2}});
	auto v2 = xt::make_boolview(rn, (rn > 0.2));
	std::cout << v << std::endl;
	std::cout << v2 << std::endl;
	v += 2.0;

	v2 += 15;
	std::cout << rn << std::endl;

With corresponding output:

{{0.135477, 0.835009, 0.968868},
 {0.221034, 0.308167, 0.547221},
 {0.188382, 0.992881, 0.996461}}

{0.308167, 0.996461}

{0.835009, 0.968868, 0.221034, 0.308167, 0.547221, 0.992881, 0.996461}

{{0.135477, 15.835, 15.9689},
 {15.221, 17.3082, 15.5472},
 {0.188382, 15.9929, 17.9965}}

Let me know what you think.

@JohanMabille
Copy link
Member

Who will stop you ? :)

That's definitely a must have. I guess this implementation is not finished (since some operators are commented and there is no test) ?

@wolfv
Copy link
Member Author

wolfv commented Jan 30, 2017

So this was functionality wise already working.

However, I am wondering: Does it make sense to implement xstepper in a way that it takes a reference to the underlying m_indices and uses the begin() end() of the underlying vector of indices?
If you think yes, then I'd try to improve the xstepper in that regard.

I'll also add some tests!

@JohanMabille
Copy link
Member

JohanMabille commented Jan 31, 2017

If that improves the performance of the stepper, that makes sense. Else you should avoid to use the private implementation of the indexview and rely on its interface.

@wolfv
Copy link
Member Author

wolfv commented Jan 31, 2017

Ok, I have implemented the xstepper with the public interface of the underlying view.

Let me know what you think!

Currently can't really figure out why the clang 3.6 build is failing.

I think one improvement could be to use a heap allocated shape if a compile time known list of indices is given.

@JohanMabille
Copy link
Member

JohanMabille commented Jan 31, 2017

That looks like a dangling reference bug. We experienced it before on other tests, a workaround is to avoid nested calls to functions that return xview and/or xfunction. #85 should fix this kind of problems.

I'll have a look at your code soon !

@wolfv wolfv force-pushed the xindexview branch 3 times, most recently from 4748127 to 26aadd0 Compare February 2, 2017 12:49
@wolfv
Copy link
Member Author

wolfv commented Feb 2, 2017

I'm giving up ... is there an easy way to install old clang in a docker image?

@JohanMabille
Copy link
Member

I don't know how to do that :/ Maybe you could try running the tests with valgrind ? Even if the tests pass, valgrind could detect some memory issues that lead to the crash with clang.

@wolfv
Copy link
Member Author

wolfv commented Feb 2, 2017

Ok, I installed clang3.6 in a docker image and found the bug ...

when return x[i]++ clang3.6 apparently returns the element of the array (x[i]) instead of x ... :)

@JohanMabille
Copy link
Member

Well done !

*
* @tparam T the function type
* @tparam R the return type of the function
* @tparam S the shape type of the generator
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be replaced by the documentation corresponding to this class. The same for the remaining of the documentation

* @tparam S the shape type of the generator
*/
template <class T, class S>
class xindexview : public xview_semantic<xindexview<T, S>>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only about style here, but I think T should be replaced by E. T generally implies a basic type parameter, while E implies more an expression, which is the kind of types you expect here.

reference operator[](const xindex& index);

template <class It>
reference element(const It& first, const It& last);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Iterators should be passed by value, not by const ref.

const_reference operator[](const xindex& index) const;

template <class It>
const_reference element(const It& first, const It& last) const;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same remark as for the non const reference

using temporary_type = typename xcontainer_inner_types<self_type>::temporary_type;
using base_index_type = get_index_type<shape_type>;

xindexview(T& f, const indices_type&& indices) noexcept;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's a typo here, you can't combine const and rvalue reference. Depending on how you want to instantiate an xindexvie you might consider rvalue reference and const lvalue reference overloads, or better universal references.

//@{
/**
* Constructs an xindexview applying the specified function over the
* given shape.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same remark as for the class declaration, you should rewrite the documentation so it matches what this code does.

template <class T>
auto inline make_xindexview(T& arr, const std::vector<xindex>&& indices)
{
return xt::xindexview<T, std::vector<std::size_t>>(arr, std::move(indices));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same remark as for the xindex_view constructor, you can't mix const and rvalue reference. You can't use universal references if you want to allow initializer lists, so you'll need overloads for rvalue reference and const lvalue reference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm... I have fixed this issue. Can you check if I got it right? I'm not a super-expert on these fancy references :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The constructor of xindex_view is good.

The make_xindexview function may be problematic, generally you should avoid to overload on universal references since they are a "catch all" arguments. However this seems to work and I don't want to wait more for this feature (Travis delays are killing me).

}

template <class T, class O>
auto inline make_xboolview(T& arr, const O& bool_arr)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether something like filter_view or make_xfilter would be a better name.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed it to make_xfilter for now.

template <bool is_const, class T, class S>
inline void xindexview_stepper<is_const, T, S>::step_back(size_type dim, size_type n)
{
m_index[dim] -= 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about it but shouldn't we have m_index[dim] -= n ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed.

@JohanMabille
Copy link
Member

JohanMabille commented Feb 3, 2017

I add it to 0.4 milestone since you're very close to finish this.
Should fix #105.

@JohanMabille
Copy link
Member

Let's merge !

@JohanMabille JohanMabille merged commit 90743c9 into xtensor-stack:master Feb 4, 2017
@JohanMabille JohanMabille mentioned this pull request Feb 4, 2017
@wolfv wolfv deleted the xindexview branch February 6, 2017 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants