Issue 6798 - Integrate overloadings for multidimensional indexing and slicing #443
Conversation
|
This patch is failing on all 64 bit platforms: http://d.puremagic.com/test-results/pull.ghtml?runid=12403 |
|
Revamped changes. |
|
(background: I'm doing a review of pull requests < 1000 as they tend to be controversial) This is nice, self-consistent, and tastefully designed. The question is scope and intended usage. This is a relatively hefty language addition, with a very narrow clientele. I'm not sure whether those clients actually need this stuff, and if they do, whether the current form is satisfactory to them. (For example one issue is that the slice limits are computed outside the index expression, so presumably they'd make certain usages difficult.) One other issue discussed in the enhancement request is that there might be a need for strides etc. So I'm torn about this. One one hand it's a work of beauty. On the other it seems of low impact and unclear user base. Thoughts? |
There is no two argument version of fstp (That form is in some of the old Intel manuals, but it's redundant).
|
I vote for merging this. Or at the very least, some subset of this that supports multidimensional opDollar with slicing. Some background: I'm working on a multidimensional array implementation that uses the variadic form of opIndex to dereference array elements, and it works wonderfully (opDollar works as well). However, slicing syntax currently is not supported, so I've implemented a workaround using opSlice with two array arguments, so one could write A[[0,0,0]..[2,2,2]] for example, but the problem is that opDollar doesn't work correctly ($ always gets mapped to opDollar!0, so A[[0,0,0]..[$,$,$]] does not do the right thing when the array has different lengths along each dimension. I've tried various alternative ways to provide convenient slicing syntax, but none of them can support opDollar correctly. It would be very nice if slicing can also support variadic arguments, since currently it's a hole in the language -- opIndex supports multidimensional arrays but opSlice doesn't. |
|
I'd like to include with this some code that is commented out for the moment that issues a warning and corrective action when opSlice, opSliceUnary, opSliceAssign, and opSliceOpAssign as those will now be obsolete. |
|
Ping on this. Now that time has passed we should be a tad wiser so maybe we have new insights in the matter. Thoughts? |
|
Slices and strides make code more expressive and can save hours of debugging especially if one deals with matrices. The design suggested is really good. It doesn't break existing code. It reduces the number of methods one has to implement (e.g. I just can not understand why it is still not merged. |
|
Sorry I'm failing to rebase this change to git master. Currently this change will break |
|
This needs a rebase (perhaps a rebase -i). There are dozens of unrelated commits in here. Otherwise, I think this is awesome. This finishes the job started with opDollar, and it's something we've needed to do eventually. What happens if you do: |
|
And, now I have a small performance worry. To represent slice, it would need to pack the start and end of the slice. struct S {
auto opSlice(int x, int y) { return tuple(x, y); }
void opIndex(int, Tuple!(int, int) slice) {}
}
S s;
s[1, 2..3]; // 2..3 will always need creating tuple objectIs this acceptable? |
|
OK. I finished rebasing. |
|
I vote also for merging this! multidimensional slicing is very interesting for people doing numerical computations! |
|
Finished rebasing. |
|
Is my understanding that this is a large breakage of existing uses of opSlice? |
| } | ||
| else | ||
| { | ||
| ae->error("multi-dimentional slicing requires template opSlice"); |
|
For the record Walter and I are on board with this language addition as long as there's no breakage of existing code. Thanks! |
|
The pull description states:
Looking at the tests, it does seem to break alias-this'd tuples. Kenji confirmed it in a comment:
|
|
@9rnsr can you work some magic to keep alias this working? |
Change to recursive call for later fallback mechanism.
All of N-dimensional array operations are now translated to ArrayExp. But currently they are immediately translated to SliceExp, so have no effect.
If it fails, fall back to opSlice for backward compatibility.
If it fails, fall back to opSliceAssign for backward compatibility.
If it fails, fall back to opSliceUnary for backward compatibility.
If it fails, fall back to opSliceOpAssign for backward compatibility.
|
Auto-merge toggled on |
Issue 6798 - Integrate overloadings for multidimensional indexing and slicing
|
Thanks! |
| @@ -1069,6 +1069,18 @@ class ArrayLengthExp : public UnaExp | |||
| void accept(Visitor *v) { v->visit(this); } | |||
| }; | |||
|
|
|||
| struct IntervalExp : Expression | |||
There was a problem hiding this comment.
This should have been a class.
|
So, it appears I have mis-understood what this PR actually does, and instead have the simple question of why? Why is this useful? Why would you want a slice operation to potentially call an This isn't the first addition to the D language that's been merged that really shouldn't have been, not without a more public review process. Perhaps requiring a formal review, such as what is required for the addition of Phobos modules, when adding or removing features or behaviors from the D language itself? |
|
Huh? This pull has been referenced multiple times over the course of (at least) the past year in the forum. Is that not public enough? Or are we supposed to copy-n-paste the comments / code from the pull to the forum in order to make it more public? I'm confused. |
|
Regarding your question about opIndex, the idea here is that opSlice!n(a,b) will return some kind of type that represents a slice from a to b in the n'th position, and opIndex will be overloaded to take arguments of this type and implement the actual slicing this way. The idea is that you should be able to call opSlice to get a type representing an index range, and then you can use that range object in multiple subsequent calls to opIndex. This is more flexible than limiting opSlice to do the actual slicing, then you have to keep track of the endpoints of the range manually each time. |
|
The thing is that the way it's been being done is subjective to the actual reader of the forum topics, by using the same process that phobos modules go through, it produces a non-subjective result, in the form of votes. |
|
But that still doesn't address the fact that your using |
|
I don't know why you think that opIndex is only intended to retrieve a single element. In my own multidimensional array implementation, I've done |
|
Your use of the ranges there is born out of the lack of a proper way to do it, |
|
I find the use of opIndex for both purposes more uniform, because it allows generic implementations that handle all cases in a single interface. Subdimensional slicing, in my implementation, actually returns a different type depending on how many arguments are slices and how many are indices (the types are segmented by dimension; the 0-dimensional case aliases to the element type); the case where all are indices is only one of the many possible combinations. Arbitrarily segmenting In any case, you're arguing with the wrong person; you should be asking Kenji the rationale for this design, since he's the one who implemented it! :) |
|
Great work folks! I do have a concern, though: it looks like you're bringing Python 2.0-style slice handling to D, even though it turned out there were limitations to that model that could have been avoided with a different design. It worries me that Julia, a language specifically designed ~15 years later to make up for the shortcomings in existing scientific platforms, chose an alternate approach. More discussion of that here: It looks like I timed this pretty poorly, but I actually just wrote up a DIP that addresses the problem of "advanced" indexing in a more general way, and enables more MATLAB/Julia-style numerical features, while reducing language complexity and maintaining backwards compatibility: Again, I know the timing's horrible, after all the work that's gone into this, but I think it's important to get numerical syntax right, and I think a bit of discussion about the pros and cons of each approach would do the language some good. Discussion receptacle: |
|
@quickfur In the proposal, the member function |
|
@MasonMcGill Sometimes |
|
@9rnsr Understood. My request was for a discussion about why this approach is better than something like http://wiki.dlang.org/DIP58 , and whether this is meant to be the "last stop" for the evolution of numerical syntax in D, or part of a larger transition. |
|
I believe that it is related to the code meaning of indexing, by introducing this under the name of |
|
@Orvid I agree it was related. But, by supporting multi-dimensional slicing, |
IMHO opSlice would not be the correct term. The only sense in which it is a "slice" is that with an n-dimensional array, it doesn't reduce the dimensionality, whereas an "index" normally does. But we don't even enforce those semantics. For example, you could create something with an infinite number of dimensions, and you could make slicing reduce the dimensionality as well. Really it is a strided slice, which is a totally different beast, neither an index nor a slice, it doesn't have much in common with normal slices. Pragmatically, the opIndex syntax generalizes, whereas the opSlice syntax does not. |
|
While I do see your point that |
|
Is there a corresponding documentation PR for this? |
|
@Orvid No, in this PR |
|
documentation? |
|
I'll see if I can work up a PR for the docs today. |
|
Docs PR: dlang/dlang.org#625 |
| static assert(a[0] == 1); | ||
| //static assert(is(A[1] == int)); | ||
| //static assert(is(a[1] == int)); | ||
| static assert(A[2] == "foo"); | ||
| //static assert(A[2] == "foo"); |
There was a problem hiding this comment.
What was the reasoning behind this? (only noticed it as I went to see how this handled both opSlice and opIndex being assigned)
There was a problem hiding this comment.
I don't remember the precise reason, but it had been interfere with the new operator overloading rule. And the test case was added without deep thinking when issue 8735 was fixed (actually it was added by me).
So I was disable it again because it would not introduce serious use code breaking.
|
This pull request introduced a regression: |
http://d.puremagic.com/issues/show_bug.cgi?id=6798
This patch is additional enhancement of opDollar (issue 3474 and #442).
Enable the mixing operator overloadings of indexing and slicing
a[$-1, 2..$]is translated toa.opIndex(a.opDollar!0 - 1, a.opSlice!1(2, a.opDollar!1))The slicing
lwr..uprinside bracket is converted toa.opSlice!(dimension)(lwr, upr).This enhancement never break existing codes.