Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

GORM : batch importing large datasets and a performance benchmarking app


We have a new client that will be importing large payment data files into out application and it appears that this topic (Zach) has some common mind share right now. Over the years I have seem many recommendations for hibernate performance tuning with large sets of data and most recently for GORM as well. What I haven't seen are any sample straight forward benchmark apps to fiddle with. So I jammed the beginnings of one together and here it is.

Zach posted a link to a project he was having problems with using GPars to do a bulk data import with GPars and I ran with that as a good base.


The GPars is the Groovy way to do parallel processing so you can take advantage of all those cores for longer running areas of your code. Its theoretically perfect for splitting up batch process to take advantage of validation and processing in your cores will keeping the statments firing into your database.

Key conclusions

The 4 key factors I have discovered so far to really speed things up are..

  1. Use GPars so you are not firing on the proverbial 1 cylinder.
  2. follow the concepts in the hibernate suggestions here in for chaps 13.1 and 13.2 and set your jdbc.batch_size then go to Ted's article here
  3. use small transaction batches and keep them the same size as the jdbc.batch_size. DO NOT (auto)commit on every insert
  4. Don't use GORM data binding if you can avoid it.
  • DON"T do this -> new SomeGormClass(yourPropSettings) or = [name:'jim',color:'red'] or use bindData()
  • DO explicitly set the values on the fields or your gorm object -> = 'jim' ...etc

My Bench Mark Results and details

LoaderService has the different benchmarks to run and its called form Bootstrap using when I do run-war

  • 115k CSV records on a macbook pro 2.8 dual core. 1024mb ram was given to the VM and these were run using prod run-war
  • I'm using MySql as the DB and its installed on my mac too so GPars can't really get all the cores
  • all of these have jdbc.batch_size = 50 and use the principals from #2 above and flush/clear every 50 rows
  • The test where the batch insert happen in a single transaction can't be tested with GPars since a single transaction can't span multiple threads
  • Databinding is NOT used on the first 3 tests,except for the last test. You can see that using Databinding pretty much doubles the time for both the normal way and using GPars
  • the winner seems to be gpars and batched (smaller chunks) transactions
Normal with GPars
Single transaction 91 s Not applicable
Commit Tran (auto) each record 298 s 98 s
Batched Transactions 92 s 49 s
Commit every 50 records
Batch Tran every 50 190 s 94 s
Using Databinding
Spring's SimpleJdbc 37 s 27 s


  • My hunch is that GPars difference will be even more significant on a quad with HT. Could use some help here

More background and reading

Here are a 2 links you should read that will give you some background information on processing large bulk data batches. read up through 13.2 and read this entire post

Thomas Lin setup a test for some processing for GPars

and the gpars docs


grails gpars batch load benchmark






No releases published


No packages published