HOWTO
Clone this wiki locally
This page shows examples for working with Fastnate. The full source code can be found in the fastnate-examples module
For easier questions you should check the FAQ. If you have any other question, don't hesitate to ask (for example here).
How to import data without writing any line of code
Fastnate will try to import every file which is somewhere in the entities
folder of your data folder. Your data folder is a setting which you may configure using the property fastnate.data.folder
and which defaults to data
. This means, it will look for a folder in your current directory or in your class path which is called data
.
See this data folder as an example: fastnate-examples/src/main/data/entities
Out of the box fastnate-data is only able to import XML files from the entities folder. If you include the fastnate-csv module, you can even read from CSV files. Both types are read in a generic fashion by mapping the names of the JPA properties in the entities to the columns resp. elements of the files.
Here is an extract of the XML file:
<Persons>
<Person firstName="Jude" lastName="Smith" organisation="Alphabet Inc." active="true" entryDate="2013-01-31">
<supervisor>
<Person firstName="Mark" lastName="Unknown" organisation="Alphabet Inc." active="yes" />
</supervisor>
...
</Person>
...
</Persons>
(You can find the full XML file and the example for the CSV file in the mentioned example folder)
How to create a customized import of entities
If you want to create all or some of the entities with code, you can do this with a DataProvider
. See this example:
public class PersonData extends AbstractDataProvider {
@Resource
private OrganisationData organisations;
@Getter
private final List<Person> entities = new ArrayList<>();
/** Creates the example persons. */
@Override
public void buildEntities() throws IOException {
final Person nate = new Person("Nate", "Smith");
nate.setActive(true);
nate.setOrganisation(this.organisations.getByName().get("Fastnate"));
final Person john = new Person("John", "Doe");
john.setSupervisor(nate);
this.entities.add(john);
}
}
The referenced OrganisationData class can be found below in the How to import data from custom CSV files section.
We can reference any data that was created by the injected DataProviders
, because buildEntities()
of the different DataProvider
s is called in the order of their creation. The buildEntities()
method of any injected DataProvider
is always called, before my buildEntities()
method.
In the above example we don't need to add nate
to the entities, as john
references nate
and all references are persisted as well.
We can even reference any entity which was imported by the generic file importers by its unique key:
...
@Resource
private EntityRegistration entityRegistration;
public void buildEntities() throws IOException {
...
this.entityRegistration.invokeOnEntity(Organisation.class, "My Organisation",
john.getPreviousOrganisations()::add);
How to import data from custom CSV files
Data from CSV files which are not an exact mapping of the properties may be read by creating a custom data provider and using the CsvDataImporter
from the fastnate-csv
module:
public class OrganisationData extends AbstractDataProvider {
@Resource
private DataFolder dataFolder;
@Resource
private GeneratorContext context;
@Getter
private List<Organisation> entities;
@Override
public void buildEntities() throws IOException {
// Lookup the definition of the entity class.
final EntityClass<Organisation> entityClass = this.context.getDescription(Organisation.class);
// Create the importer
final CsvDataImporter<Organisation> importer = new CsvDataImporter<>(entityClass);
// Map the "id" and "name" column with the default mappings
importer.addDefaultColumnMapping("id");
importer.addDefaultColumnMapping("name");
// Map the "web" column to the "url" property
importer.addColumnMapping("web", Organisation::setUrl);
// Add a currency converter for the profit column
importer.addColumnMapping("profit", float.class,
new FormatConverter<>(NumberFormat.getCurrencyInstance(Locale.US)), Organisation::setProfit);
// Ignore the comment column
importer.addIgnoredColumn("comment");
this.entities = importer.importFile(this.dataFolder.findFile("organisations.csv"));
}
}
(this is just an extract, the complete example can be found here)
How to use Fastnate after going to production mode
As long as you are able to rebuild your database from scratch when something changes in your model or in you imported data, you are fine using the schema generation and import features of Hibernate and Fastnate. This could even be the case in production mode, when your database is "read only" and contains only data which was collected somewhere else. But as soon as a user is able to modify the imported data, you can't rebuild the database anymore, because this would destroy any user data. So you need a strategy for these changes.
One option is to create change scripts and apply them during deployment or startup time, for example with Liquibase or Flyway. To generate these script one can store the previous schema.sql
file (generated by Hibernate) and data.sql
file (generated by Fastnate) and compare them before each release or after every change with the now generated files.
You can even create a Liquibase changeset file with all the necessary insert statements by using the LiquibaseStatementsWriter, supported configuration properties:
-
fastnate.data.statements.writer
- Should be set toLiquibaseStatementsWriter
-
fastnate.liquibase.file
- The name and path to the generated file, defaults tochangelog.xml
in the current directory -
fastnate.liquibase.version
- The version of Liquibase to reference in the header, defaults to2.0
You can compare the generated changeset with the previous changeset to identify the necessary inserts / updates for your "real" changeset.
To compare the schema changes, you could use the diff tools provided by Liquibase. There is even a Hibernate specific support.
If you use Flyway or if you want to compare the plain schema.sql, here is an example for how to generate that with hibernate-tools in Maven:
<profile>
<id>generate-schema</id>
<build>
<defaultGoal>antrun:run</defaultGoal>
<plugins>
<plugin>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.7</version>
<configuration>
<target name="hbm2ddl">
<taskdef name="hibernatetool" classname="org.hibernate.tool.ant.HibernateToolTask"
classpathref="maven.compile.classpath" />
<delete file="${basedir}/src/main/sql/schema.sql" />
<hibernatetool destdir="${basedir}/src/main/sql">
<classpath refid="maven.compile.classpath" />
<jpaconfiguration persistenceunit="database"
propertyFile="${basedir}/src/main/sql/schema-generation.properties" />
<hbm2ddl outputfilename="schema.sql" export="false" drop="true" format="false" />
</hibernatetool>
</target>
</configuration>
</plugin>
</plugins>
</build>
<dependencies>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-tools</artifactId>
<version>5.0.2.Final</version>
</dependency>
</dependencies>
</profile>
As we don't want to run the Hibernate-Tools against the online database configured in the persistence.xml, we are using a properties file src/main/sql/schema-generation.properties
which configures the offline database and the apropriate dialect:
hibernate.connection.provider_class=org.hibernate.engine.jdbc.connections.internal.UserSuppliedConnectionProviderImpl
hibernate.dialect=...the hibernate database dialect, e.g. org.hibernate.dialect.H2Dialect...
# If you turned on any second level cache in the persistence.xml, you will need to turn it of:
hibernate.cache.use_second_level_cache=false
hibernate.cache.use_query_cache=false
To generate the schema you only need to call mvn -Pgenerate-schema
now.
What is Fastnate? Read more at fastnate.org.