Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

problem using SharedHashMapBuilder ... #36

Closed
Cotton-Ben opened this issue Jun 26, 2014 · 6 comments
Closed

problem using SharedHashMapBuilder ... #36

Cotton-Ben opened this issue Jun 26, 2014 · 6 comments

Comments

@Cotton-Ben
Copy link

Hi,

I fetched/merged into my clone'd workspaces all updates from each of my upstream views of the various OpenHFT parent repo modules.

I then re-ran my Test at https://github.com/Cotton-Ben/HugeCollections/blob/master/collections/src/test/java/net/openhft/collections/fromdocs/com/jpmorgan/pingpong_latency/PingPongPlayerLeft.java

via

java  org.junit.runner.JUnitCore net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft

Since this update, when the following line #18 in my Test is executed:

SharedHashMap<String, BondVOInterface> shmLeft = new SharedHashMapBuilder()
                .generatedValueType(true)
                .entrySize(320)
                .create(
                        new File("/dev/shm/BondPortfolioSHM"),
                        String.class,
                        BondVOInterface.class
                );

This test now fails with the following exception:

Time: 0.107
There was 1 failure:
1) bondExample(net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft)
java.lang.IllegalArgumentException: invalid size: 343048307875968
    at net.openhft.lang.io.MappedStore.<init>(MappedStore.java:54)
    at net.openhft.lang.io.MappedStore.<init>(MappedStore.java:49)
    at net.openhft.collections.AbstractVanillaSharedHashMap.createMappedStoreAndSegments(VanillaSharedHashMap.java:148)
    at net.openhft.collections.VanillaSharedHashMap.<init>(VanillaSharedHashMap.java:45)
    at net.openhft.collections.SharedHashMapBuilder.create(SharedHashMapBuilder.java:264)
    at net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft.bondExample(PingPongPlayerLeft.java:18)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runners.Suite.runChild(Suite.java:127)
    at org.junit.runners.Suite.runChild(Suite.java:26)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
    at org.junit.runner.JUnitCore.runMain(JUnitCore.java:96)
    at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:47)
    at org.junit.runner.JUnitCore.main(JUnitCore.java:40)

FAILURES!!!
Tests run: 1,  Failures: 1

Should I expect the way that line #18 above is coded in my Test to work as before without exception? Any recommendation for remedy?
@peter-lawrey
Copy link
Owner

It is possible there have been changes in the format. Can you delete the
old files first? We need to get to a point where old formats are supported.

On 26 June 2014 20:53, Ben Cotton notifications@github.com wrote:

Hi,

I fetched/merged into my clone'd workspaces all updates from each of my
upstream views of the various OpenHFT parent repo modules.

I then re-ran my Test at
https://github.com/Cotton-Ben/HugeCollections/blob/master/collections/src/test/java/net/openhft/collections/fromdocs/com/jpmorgan/pingpong_latency/PingPongPlayerLeft.java

via

java org.junit.runner.JUnitCore net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft

Since this update, when the following line #18
#18 in my Test is
executed:

SharedHashMap<String, BondVOInterface> shmLeft = new SharedHashMapBuilder()
.generatedValueType(true)
.entrySize(320)
.create(
new File("/dev/shm/BondPortfolioSHM"),
String.class,
BondVOInterface.class
);

This test now fails with the following exception:

Time: 0.107
There was 1 failure:

  1. bondExample(net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft)
    java.lang.IllegalArgumentException: invalid size: 343048307875968
    at net.openhft.lang.io.MappedStore.(MappedStore.java:54)
    at net.openhft.lang.io.MappedStore.(MappedStore.java:49)
    at net.openhft.collections.AbstractVanillaSharedHashMap.createMappedStoreAndSegments(VanillaSharedHashMap.java:148)
    at net.openhft.collections.VanillaSharedHashMap.(VanillaSharedHashMap.java:45)
    at net.openhft.collections.SharedHashMapBuilder.create(SharedHashMapBuilder.java:264)
    at net.openhft.collections.fromdocs.com.jpmorgan.pingpong_latency.PingPongPlayerLeft.bondExample(PingPongPlayerLeft.java:18)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runners.Suite.runChild(Suite.java:127)
    at org.junit.runners.Suite.runChild(Suite.java:26)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
    at org.junit.runner.JUnitCore.runMain(JUnitCore.java:96)
    at org.junit.runner.JUnitCore.runMainAndExit(JUnitCore.java:47)
    at org.junit.runner.JUnitCore.main(JUnitCore.java:40)

FAILURES!!!
Tests run: 1, Failures: 1

Should I expect the way that line #18
#18 above is coded in my
Test to work as before without exception? Any recommendation for remedy?

—
Reply to this email directly or view it on GitHub
#36.

@Cotton-Ben
Copy link
Author

Not really sure what you mean by "Can you delete the
old files first?"

@Cotton-Ben
Copy link
Author

Ah! I got it

rm -fr /dev/shm/*

did the trick!! ;-)

Thanks Peter ...

@peter-lawrey
Copy link
Owner

Like I said, you should be able to read old file with newer versions of SHM.
We are in the process of documenting and solidifying the format so this
issue doesn't arise.

I am also working on ways to make data changes more robust with
self-describing data formats. I have some plans to support both zero-copy
in place data structures which also support data format changes.

On 26 June 2014 21:13, Ben Cotton notifications@github.com wrote:

Closed #36 #36.

—
Reply to this email directly or view it on GitHub
#36 (comment).

@Cotton-Ben
Copy link
Author

As always, great work!

I am also working on ways to make data changes more robust with
self-describing data formats

Oh man ... you've got our minds percolating now (also, as always).

@peter-lawrey
Copy link
Owner

You can have a format which allows random access, but also has self
describing data embedded (which doesn't change of course) so it can still
be read if the format changes or you don't have access to such information.

On 26 June 2014 21:24, Ben Cotton notifications@github.com wrote:

As always, great work!

I am also working on ways to make data changes more robust with
self-describing data formats

Oh man ... you've got our minds percolating now (also, as always).

—
Reply to this email directly or view it on GitHub
#36 (comment)
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@peter-lawrey @Cotton-Ben