{{ message }}

# openjdk / panama-foreign

Closed
wants to merge 5 commits into from
Closed

wants to merge 5 commits into from

## Conversation

### ChrisHegarty commented May 20, 2020 • edited by openjdk bot

Hi,

As part of feedback on the Foreign Memory API (when experimenting with its usage internally in the JDK), a small number of potential usability enhancements could be made to the API. This is the fourth such, and last on my current todo list.

This change proposes to add a new method:
MemorySegment::mismatch

The mismatch semantic is very useful for building equality and comparison logic on top of segments. I found that I needed such when modeling and comparing native socket address in the JDK implementation. It is possible to write your own, but requires a non-trivial amount of not-trivial code - to do it right! I also think that we can provide a more efficient implementation building on top of the JDK's internal mismatch support.

I still need to do some perf testing and add a micro-benchmake ( Maurizio suggested possibly amending TestBulkOps ). There is also the question about possibly improving the JDK's internal implementation to work on long sizes (which could be done separately). For now, I just want to share the idea, along with the proposed specification and initial implementation.

### Progress

• Change must not contain extraneous whitespace
• Change must be properly reviewed

### Reviewers

$git fetch https://git.openjdk.java.net/panama-foreign pull/180/head:pull/180 $ git checkout pull/180

 Initial mismatch implementation 
 ba7c832 

### bridgekeeper bot commented May 20, 2020

 👋 Welcome back chegar! A progress list of the required criteria for merging this PR into foreign-memaccess will be added to the body of your pull request.
bot added the label May 20, 2020

### Webrevs

reviewed

 Looks good - I've added some comments. Test is very comprehensive - thanks!
Show resolved Hide resolved
Show resolved Hide resolved
 } for (; off < minSize; off++) { if (UNSAFE.getByte(this.base(), this.min() + off) != UNSAFE.getByte(that.base(), that.min() + off)) {

#### mcimadamore May 20, 2020 Collaborator

This code could be simplified/rewritten to use MemoryAddress' and VH, instead of unsafe access with object/offset addressing. E.g. you could maintain a MemoryAddresslocal variable in the loop instead of theoffsetand keep increasing that address on each iteration ofArraySupport::vectorizedMismatch. Then, when you get out of the loop, the address already points at the base of the region to compare, and a simple for loop with an indexed VH should do the job.

 @Test(dataProvider = "slices") public void testSameValues(MemorySegment ss1, MemorySegment ss2) { out.format("testSameValues s1:%s, s2:%s\n", ss1, ss2); MemorySegment s1 = initializeSegment(ss1); MemorySegment s2 = initializeSegment(ss2); if (s1.byteSize() == s2.byteSize()) { assertEquals(s1.mismatch(s2), -1); // identical assertEquals(s2.mismatch(s1), -1); } else if (s1.byteSize() > s2.byteSize()) { assertEquals(s1.mismatch(s2), s2.byteSize()); // proper prefix assertEquals(s2.mismatch(s1), s2.byteSize()); } else { assert s1.byteSize() < s2.byteSize(); assertEquals(s1.mismatch(s2), s1.byteSize()); // proper prefix assertEquals(s2.mismatch(s1), s1.byteSize()); } } @Test(dataProvider = "slices") public void testDifferentValues(MemorySegment s1, MemorySegment s2) { out.format("testDifferentValues s1:%s, s2:%s\n", s1, s2); s1 = initializeSegment(s1); s2 = initializeSegment(s2); for (long i = s2.byteSize() -1 ; i >= 0; i--) { long expectedMismatchOffset = i; BYTE_HANDLE.set(s2.baseAddress().addOffset(i), (byte) 0xFF); if (s1.byteSize() == s2.byteSize()) { assertEquals(s1.mismatch(s2), expectedMismatchOffset); assertEquals(s2.mismatch(s1), expectedMismatchOffset); } else if (s1.byteSize() > s2.byteSize()) { assertEquals(s1.mismatch(s2), expectedMismatchOffset); assertEquals(s2.mismatch(s1), expectedMismatchOffset); } else { assert s1.byteSize() < s2.byteSize(); var off = Math.min(s1.byteSize(), expectedMismatchOffset); assertEquals(s1.mismatch(s2), off); // proper prefix assertEquals(s2.mismatch(s1), off); } } }
Comment on lines +57 to +99

#### mcimadamore May 20, 2020 Collaborator

How important is it that these tests operate on slices? Looking at the test code, it could have worked equally well if the input parameters were just two sizes, and then you did an explicit allocation (or maybe also receive a segment factory from the provider, so that you can test different segment kinds).

#### ChrisHegarty May 20, 2020 Author Member

Originally I had a version of the test that did compare specific segments, but it didn't scale well to different sizes and kinds ( we need to test both above and below the 8 byte threshold ). I removed a number of slice sizes, which greatly reduces the combinations. This may be enough, or I can certainly revisit the test's structure.

approved these changes

### PaulSandoz left a comment

 Very nice. Improving the JDK implementation is good. In fact i think you could so that now. In ArraysSupport, with strip mining:  public static int vectorizedMismatchLarge( Object a, long aOffset, Object b, long bOffset, long length, int log2ArrayIndexScale)  Then you can specialize mismatch of memory segments for length threshold and type (the threshold only really makes sense for the first check, once you go over Integer.MAX_VALUE the cost of another round with a small length part is really low overall).

### openjdk bot commented May 20, 2020 • edited

 @ChrisHegarty This change now passes all automated pre-integration checks, type /integrate in a new comment to proceed. After integration, the commit message will be: Add MemorySegment::mismatch Reviewed-by: mcimadamore, psandoz  If you would like to add a summary, use the /summary command. To credit additional contributors, use the /contributor command. To add additional solved issues, use the /issue command. Since the source branch of this PR was last updated there have been 143 commits pushed to the foreign-memaccess branch: f024ca7: Automatic merge of master into foreign-memaccess 046af8c: Automatic merge of jdk:master into master de37507: 8245619: Remove unused methods in UnixNativeDispatcher 113c48f: 8215401: Add isEmpty default method to CharSequence 7d330d3: 8245335: [TESTBUG] DeterministicDump.java fails with release JVM f72b574: 8245459: Add support for complex filter value var handle adaptation ea38873: 8239480: Support for CLDR version 37 b5b6ae3: 8245241: Incorrect locale provider preference is not logged e3be308: 8245260: Missing file header for test/hotspot/jtreg/containers/docker/TEST.properties 270674c: Merge ... and 133 more: https://git.openjdk.java.net/panama-foreign/compare/0dc2fed332fdfdbecbd4008c650d2f0ba77e1e6a...foreign-memaccess As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid automatic rebasing, please merge foreign-memaccess into your branch, and then specify the current head hash when integrating, like this: /integrate f024ca7de9d28bbaa27e57e8d4f547db5ea06002. As you do not have Committer status in this project, an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@mcimadamore, @PaulSandoz) but any other Committer may sponsor as well. ➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).
bot added the label May 20, 2020

### mlbridge bot commented May 20, 2020

 Mailing list message from Maurizio Cimadamore on panama-dev: On 20/05/2020 16:36, Paul Sandoz wrote: On Wed, 20 May 2020 14:06:48 GMT, Chris Hegarty wrote: Hi, As part of feedback on the Foreign Memory API (when experimenting with its usage internally in the JDK), a small number of potential usability enhancements could be made to the API. This is the fourth such, and last on my current todo list. This change proposes to add a new method: MemorySegment::mismatch The mismatch semantic is very useful for building equality and comparison logic on top of segments. I found that I needed such when modeling and comparing native socket address in the JDK implementation. It is possible to write your own, but requires a non-trivial amount of not-trivial code - to do it right! I also think that we can provide a more efficient implementation building on top of the JDK's internal mismatch support. I still need to do some perf testing and add a micro-benchmake ( Maurizio suggested possibly amending TestBulkOps ). There is also the question about possibly improving the JDK's internal implementation to work on long sizes (which could be done separately). For now, I just want to share the idea, along with the proposed specification and initial implementation. Comments welcome. Very nice. Improving the JDK implementation is good. In fact i think you could so that now. In ArraysSupport, with strip mining:  public static int vectorizedMismatchLarge$$Object a\, long aOffset\, Object b\, long bOffset\, long length\, int log2ArrayIndexScale$$  Then you can specialize mismatch of memory segments for length threshold and type (the threshold only really makes sense for the first check, once you go over Integer.MAX_VALUE the cost of another round with a small length part is really low overall). To be clear - are you suggesting to add vectorizedMismatchLarge inside ArraySupport (but not add intrinsic support for that, for now) - and then to have the memory segment implementation to call either that or the standard version based on whether the segment is small or not? Maurizio

### mlbridge bot commented May 20, 2020

 Mailing list message from Paul Sandoz on panama-dev: Yes, add the ?large? strip mining implementation to ArraysSupport. I think it unlikely this method ever needs to be made intrinsic. For large loop bounds it's important that we don?t starve the system (allow for safe points), so it?s easier to do that from Java than in the intrinsic stub or in C2. Then the memory segment code can use the those primitives based on thresholds. I suspect it can get away with just calling the large version, after a check for a small threshold value and a scalar loop. Paul. On May 20, 2020, at 8:53 AM, Maurizio Cimadamore wrote: On 20/05/2020 16:36, Paul Sandoz wrote: On Wed, 20 May 2020 14:06:48 GMT, Chris Hegarty wrote: Hi, As part of feedback on the Foreign Memory API (when experimenting with its usage internally in the JDK), a small number of potential usability enhancements could be made to the API. This is the fourth such, and last on my current todo list. This change proposes to add a new method: MemorySegment::mismatch The mismatch semantic is very useful for building equality and comparison logic on top of segments. I found that I needed such when modeling and comparing native socket address in the JDK implementation. It is possible to write your own, but requires a non-trivial amount of not-trivial code - to do it right! I also think that we can provide a more efficient implementation building on top of the JDK's internal mismatch support. I still need to do some perf testing and add a micro-benchmake ( Maurizio suggested possibly amending TestBulkOps ). There is also the question about possibly improving the JDK's internal implementation to work on long sizes (which could be done separately). For now, I just want to share the idea, along with the proposed specification and initial implementation. Comments welcome. Very nice. Improving the JDK implementation is good. In fact i think you could so that now. In ArraysSupport, with strip mining: public static int vectorizedMismatchLarge$$Object a\, long aOffset\, Object b\, long bOffset\, long length\, int log2ArrayIndexScale$$  Then you can specialize mismatch of memory segments for length threshold and type (the threshold only really makes sense for the first check, once you go over Integer.MAX_VALUE the cost of another round with a small length part is really low overall). To be clear - are you suggesting to add vectorizedMismatchLarge inside ArraySupport (but not add intrinsic support for that, for now) - and then to have the memory segment implementation to call either that or the standard version based on whether the segment is small or not? Maurizio
 Move implementation into vectorizedMismatchLarge, and address other r… 
 c5414aa 
…eview comments.
reviewed
Show resolved Hide resolved
 Merge remote-tracking branch 'origin/foreign-memaccess' into mismatch 
 add1fe0 
 Integrate Paul's review comment 
 caf136f 
approved these changes
approved these changes

 Looks good
 return i; } } return thisSize != thatSize ? length : -1;

#### mcimadamore May 20, 2020 Collaborator

I guess I'm a bit confused - shouldn't this method return (as per javadoc), -1 if there's no mismatch? In this code path we found no mismatch, and yet, if sizes are different we return length, suggesting there's a mismatch. I now realize that mismatch is taken quite literally - e.g. no mismatch really means the two things are identical in contents and size --- which is, I realize, what Arrays::mismatch also does.

IMHO the javadoc of the various mismatch routines could use more clarity around what a mismatch is. But maybe that's something for another day.

### mlbridge bot commented May 20, 2020

 Mailing list message from Paul Sandoz on panama-dev: If the two segments are are different lengths and one segment is a proper prefix of the other (all elements are equal) there is a mismatch in the length. It?s described more formally in the JavaDoc of the Arrays.mismatch methods. e.g *

If the two arrays share a common prefix then the returned index is the * length of the common prefix and it follows that there is a mismatch * between the two elements at that index within the respective arrays. * If one array is a proper prefix of the other then the returned index is * the length of the smaller array and it follows that the index is only * valid for the larger array. * Otherwise, there is no mismatch. Paul.

### ChrisHegarty commented May 21, 2020

 The proposed MemorySegment::mismatch spec wording is aligned with similar in the buffer classes and elsewhere. If it needs clarification then we should probably do so consistently. As @mcimadamore said, "that's something for another day" ;-)

### ChrisHegarty commented May 21, 2020

 /integrate
bot added the label May 21, 2020

### openjdk bot commented May 21, 2020

 Add null test scenario 
 d525e57 
bot removed the label May 21, 2020
approved these changes

 Looks good!

### openjdk bot commented May 21, 2020

 @mcimadamore The PR has been updated since the change author (@ChrisHegarty) issued the integrate` command - the author must perform this command again.

### ChrisHegarty commented May 22, 2020

 /integrate

### openjdk bot commented May 22, 2020

bot added the label May 22, 2020

### mcimadamore commented May 25, 2020

bot closed this May 25, 2020
bot added and removed labels May 25, 2020