Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Perm Gen space error on large in-memory topicmaps #237

Closed
GoogleCodeExporter opened this issue Mar 16, 2015 · 4 comments
Closed

Perm Gen space error on large in-memory topicmaps #237

GoogleCodeExporter opened this issue Mar 16, 2015 · 4 comments

Comments

@GoogleCodeExporter
Copy link

GoogleCodeExporter commented Mar 16, 2015

Reproducing the problem:

Set your JAVA_OPTS to "-Xms256m -Xmx2048m" to allow plenty of Heap Space 
(remove any 
other setting to be able to reproduce the error). Then run the following code:

InMemoryTopicMapStore store = new InMemoryTopicMapStore();
TopicMapIF tm = store.getTopicMap();
TopicMapBuilderIF builder = tm.getBuilder();

int i = 0;
while (true) {
    TopicIF t = builder.makeTopic();
    String oid = t.getObjectId();
    i++;
    if (i % 10000 == 0) {
        System.out.println("topics: " + i + ", last oid: " + oid);
    }
}

This should create around 1.000.000 topics (no problem, plenty of heap 
available), but crash on a 
OutOfMemory Error (PermGen space), with a trace like:

Caused by: java.lang.OutOfMemoryError: PermGen space
        at java.lang.String.intern(Native Method)
        at 
net.ontopia.topicmaps.impl.basic.SubjectIdentityCache.registerObject(SubjectIden
tityCache.java:1
39)
        at 
net.ontopia.topicmaps.impl.utils.AbstractSubjectIdentityCache$TopicAddedHandler.
processEvent(
AbstractSubjectIdentityCache.java:176)
        at 
net.ontopia.topicmaps.impl.utils.ObjectTreeManager$EventHandler.treeAddEvent(Obj
ectTreeMana
ger.java:126)
        at 
net.ontopia.topicmaps.impl.utils.ObjectTreeManager$EH01.processEvent(ObjectTreeM
anager.java
:154)
        at 
net.ontopia.topicmaps.impl.utils.ObjectTreeManager.processEvent(ObjectTreeManage
r.java:105)
        at net.ontopia.topicmaps.impl.basic.TopicMap.processEvent(TopicMap.java:343)
        at net.ontopia.topicmaps.impl.basic.TopicMap.fireEvent(TopicMap.java:289)
        at net.ontopia.topicmaps.impl.basic.TopicMap.addTopic(TopicMap.java:131)
        at net.ontopia.topicmaps.impl.basic.TopicMapBuilder.createTopic(TopicMapBuilder.java:31)
        at net.ontopia.topicmaps.impl.basic.TopicMapBuilder.makeTopic(TopicMapBuilder.java:36)
        at nl.tweedekamer.ipk.harvest.vip.Crasher.main(Crasher.java:17)


Cause:
The PermGen space is used by JVM's to store class meta data (definitions, 
constants etc) AND 
Strings that were created using String.intern() (see 
http://java.sun.com/j2se/1.5.0/docs/api/java/lang/String.html#intern() ). 
Within Ontopia, Object 
ID's (oid) are created using String.intern(), probably to enhance speed when 
looking up id's 
trough the SubjectIdentityCache. Eventually, the String.intern() generated 
strings will take up all 
available PermGen space, and crash. Most JVM's have a cmline option to increase 
the PermGen 
space, but this masks the cause of the problem instead of fixing it. 
Note: Maven and Netbeans don't seem to use the PermGen cmdline option.

Problem:
As stated, the SubjectIdentityCache is used to cache the oid's of objects in a 
topicmap. It 
maintains a mapping of oid to objects. However, this uses a HashMap to store 
the oid-object 
mapping, which in turn uses object.hash() == object2.hash() comparison. 
Therefore, the speed 
increase intended by the use of String.intern() is not active. 

Solution:
Either NOT use String.intern() for oid's, and thus remove the limit of topics 
one can have in a 
topicmap; or use a Mapping that will utilize the speed increase gained from 
String.intern().


Original issue reported on code.google.com by qsieb...@gmail.com on 4 May 2010 at 9:43

@GoogleCodeExporter
Copy link
Author

GoogleCodeExporter commented Mar 16, 2015

Fixed in revision 1025 by removing String.intern(). It was needed in an earlier 
version of the class, but isn't any more.

(BTW: Thank you for an excellent issue report. Didn't leave much work for us. :)

Original comment by lar...@gmail.com on 11 May 2010 at 7:41

  • Added labels: Component-Engine

@GoogleCodeExporter
Copy link
Author

GoogleCodeExporter commented Mar 16, 2015

So far, this seems to fix my issue. I have not been able to get a PermGen space 
error 
yet.

Original comment by qsieb...@gmail.com on 17 May 2010 at 8:30

@GoogleCodeExporter
Copy link
Author

GoogleCodeExporter commented Mar 16, 2015

Even with full use of memory (4Gb) that i have available, I only get expected 
Heap errors. I think this issue is 
fixed.

Original comment by qsieb...@gmail.com on 17 May 2010 at 11:46

@GoogleCodeExporter
Copy link
Author

GoogleCodeExporter commented Mar 16, 2015

Excellent. Now closing.

Original comment by lar...@gmail.com on 25 May 2010 at 8:29

  • Changed state: Fixed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant