Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
GitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Revise saveConcept method to use save() instead of saveAndFlush() #575
Revise saveConcept method to use myConceptDao.save() instead of myConceptDao.saveAndFlush(), to avoid excessive CPU utilization when persisting a large CodeSystem (e.g. one with 399837 concepts).
Commentary: It seems the Hibernate flush operation contains code which performs an O(N^2) operation on the cache, and saveAndFlush causes that to be performed N times. Not good when N is 399837.
I think this change is reasonably safe, because saveConcept is only used within a transaction context. However, this may also benefit from a review by someone more familiar with HAPI's inner workings.
…ptDao.saveAndFlush (to avoid overloading CPU with flush computations when persisting large codesystems, e.g. one with 399837 concepts).
Does your great big transaction bundle save correctly with this change?
I think the original intent of the constant flushing was to avoid having the JPA session keeping all of the entities in memory as it saves. Presumably that was an unneccesary "optimization" if this is working with your proposed change. :)
Yes, my great big transaction bundle saves correctly with this change. Here's an abbreviated log file excerpt ...