grails-sharding

Grails Sharding Plugin

Dependency : Grails 2.0.0+

Documentation

Description The Sharding plugin allows application programmers to define multiple database shards to horizontally scale their user's data across multiple database schema and servers. The programmer can easily change the database the set of Domain Classes are targeting with a simple service call and objects used to segment the data are assigned a shard based on all shards usage and capacity. This is not available to grails programs out of the box as while there is more than one DataSource and SessionFactory there is no way to switch the default DataSource based on runtime behavior.

Dependancies

The Sharding plugin follows the convention over configuration philosiphy of grails but in some cases needs the programmer to provide some configuration. This configuration is in two places. First you have to define all of the data sources your application will use in the normal grails-app/conf/DataSource.groovy. In here you should have at least four data sources:

 -- Default Datasource
 -- Index Datasource - should be the same database as defined for the
    default.
 -- Shard Data Source
 -- Shard Data Source
 -- Shard ....
 -- etc

In addition to the normal DSL for multiple data sources there is one additional configuration value you must add, the shard attribute. This defines whether this datasource is a shard that should be included as we assign shards to objects.

Note: You MUST have a DataSource named datasource_index where in the index object will live

Here is an example DataSource.groovy

dataSource {
	pooled = true
	driverClassName = "org.h2.Driver"
	username = "sa"
	password = ""
}
hibernate {
	cache.use_second_level_cache=true
	cache.use_query_cache=false
	cache.provider_class='net.sf.ehcache.hibernate.EhCacheProvider'
}

// environment specific settings
environments {
	development {
		dataSource {
			shard = false
			dbCreate = "update" // one of 'create', 'create-drop','update'
			url = "jdbc:mysql://localhost:3306/shardINDEX"
			username = "root"
		}
		dataSource_index {
			shard = false
			dbCreate = "update" // one of 'create', 'create-drop','update'
			url = "jdbc:mysql://localhost:3306/shardINDEX"
			username = "root"
		}
		dataSource_shard1001 {
			shard = true
			dbCreate = "update" // one of 'create', 'create-drop','update'
			url = "jdbc:mysql://localhost:3306/shard1001"
			username = "root"
		}
		dataSource_shard1002 {
			shard = true
			dbCreate = "update" // one of 'create', 'create-drop','update'
			url = "jdbc:mysql://localhost:3306/shard1002"
			username = "root"
		}
	}
	test {
		...
	}
	production {
		...
	}
}

In addition to the DataSource.groovy file you must also create a domain class that will be used to associate data with a shard. Most often this class is a user class but could be any thing you would like. Once you have created that domain class you need to do the following:

       -- Create a string field that will hold the shard assignment (typically
          called shard)
       -- Set the datasources attribute for the class to be 'index'
       -- Add the com.jeffrick.grails.plugin.sharding.annotation.Shard
          annotation to that class
       -- Create a fieldName and indexDataSourceName attributes of the
          annotation

Here is an example class (UserIndex.groovy):

import com.jeffrick.grails.plugin.sharding.annotation.Shard

@Shard(fieldName = "shard", indexDataSourceName = "dataSource_index")
class UserIndex {

	String userName

	String shard

	static mapping = {
		datasources(['index'])
	}
}

Domain Classes that should be sharded Finally any Domain Class that you want to be sharded you must set to use the ALL datasource to do this add the following to your domain class: static mapping = { datasource 'ALL' }

Shard Domain Class The plugin creates a Domain Class called Shard which contains a record for every shard in the system. Included in this Domain Class is the capacity of the shard and the number of objects assigned to the shard. This class is queried every time a new object (typically a user) is created within the system to assign a shard based on the usage to capacity ratio. This allows for shards to be located on different types of hardware that should take either a more or less objects.

Usage To install the plugin in your application add a dependency in the "plugins" section of your BuildConfig.groovy, e.g.

compile ':sharding:0.7'

but be sure to replace "0.7" with the latest version.

Authors

Jeff Rick jeffrick@gmail.com Robert Woollam

Please report any issues to the Grails User mailing list and/or write up an issue in JIRA at http://jira.codehaus.org/browse/GRAILSPLUGINS under the Grails-Sharding component.