Skip to content

Commit

Permalink
Merge pull request #40 from corcorb/DA-2015-cleanup-code
Browse files Browse the repository at this point in the history
cleanup code
  • Loading branch information
karkum committed Sep 22, 2015
2 parents 4f428db + 6bd8b68 commit c0c43fd
Show file tree
Hide file tree
Showing 22 changed files with 294 additions and 258 deletions.
33 changes: 20 additions & 13 deletions README.md
Expand Up @@ -36,7 +36,7 @@ jPile allows the client to configure whether entities are updated when inserting

# How do I run the tests?

jPile needs a local MySQL running and Apache Maven. Create a new database schema called 'jpile' using `CREATE DATABASE jpile CHARACTER SET utf8 COLLATE utf8_general_ci`. The test classes use `root` with no password to login. The username and password is located in `AbstractIntTestForJPile` class.
jPile needs a local MySQL running and Apache Maven. Create a new database schema called 'jpile' using `CREATE DATABASE jpile CHARACTER SET utf8 COLLATE utf8_general_ci`. The test classes use `root` with no password to login. The username and password is located in `AbstractIntTestForJPile` class.

All test cases will automatically create and drop the required tables for integration tests. After creating the local database, you should be able to run `mvn clean install` to run all the tests and install locally.

Expand All @@ -59,17 +59,12 @@ jPile is very easy to use. If you are using Maven, then add the following depend
The most common use case is to create a new instance of `HierarchicalInfileObjectLoader`. You have to provide a valid database `Connection`. `HierarchicalInfileObjectLoader` doesn't rely on a database pool because it needs to disable foreign key constraints. Using multiple connections would fail because each new connection would have foreign key constraints enabled by default. Below shows how to do this.

```java
Connection connection = ...;
HierarchicalInfileObjectLoader hierarchicalInfileObjectLoader = new HierarchicalInfileObjectLoader();

try {
hierarchicalInfileObjectLoader.setConnection(connection);

hierarchicalInfileObjectLoader.persist(myEntity);
// Add more using persist()
} finally {
hierarchicalInfileObjectLoader.close();
connection.close(); // Don't forget to close the connection
try (Connection connection = ...;
HierarchicalInfileObjectLoader hierarchicalInfileObjectLoader = new HierarchicalInfileObjectLoader()
) {
hierarchicalInfileObjectLoader.setConnection(connection);
hierarchicalInfileObjectLoader.persist(myEntity);
// Add more using persist()
}
```

Expand All @@ -84,9 +79,21 @@ By running the performance test: ```mvn clean install -Dperformance```
25,000 fake objects were created. Each object has a Customer, Contact (One-to-one) and 4 Products (One-to-many) which have a Supplier (Many-to-one). All these objects were saved using simple MySQL prepared statements, Hibernate, and jPile. The results were as follows:

* Prepared Statements - 60s
* Hibernate - 40s
* Hibernate - 40s
* jPile - 6s

## Performance Graph

![Performance Graph](http://i.imgur.com/2yiT2.jpg)

# FindBugs

jPile uses the FindBugs tool to perform various static analysis checks.
FindBugs is run automatically when `mvn verify` is run.
You can run `mvn site` and look at the generated `findbugs.html` to list any bugs found.

FindBugs can also be configured in your IDE of choice.
Maven is configured to use the following settings:

- Analysis Effort: Maximal
- Minimum Confidence to Report: Medium
51 changes: 43 additions & 8 deletions pom.xml
Expand Up @@ -28,8 +28,9 @@
</licenses>

<properties>
<spring-version>3.1.2.RELEASE</spring-version>
<spring-version>3.2.6.RELEASE</spring-version>
<slf4j-version>1.7.2</slf4j-version>
<findbugs-maven-version>3.0.1</findbugs-maven-version>
</properties>


Expand Down Expand Up @@ -63,13 +64,6 @@
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<systemPropertyVariables>
<!-- compile with -Dperformance=true to run IntPerformanceHierarchicalInfileObjectLoaderTest -->
<propertyName>performance</propertyName>
<buildDirectory>${performance}</buildDirectory>
</systemPropertyVariables>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
Expand All @@ -90,6 +84,23 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>${findbugs-maven-version}</version>
<configuration>
<effort>Max</effort>
<threshold>Medium</threshold>
<xmlOutput>true</xmlOutput>
</configuration>
<executions>
<execution>
<goals>
<goal>check</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

Expand Down Expand Up @@ -134,6 +145,14 @@
<scope>provided</scope>
</dependency>

<!-- Runtime Spring Context -->
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>${spring-version}</version>
<scope>runtime</scope>
</dependency>

<!-- Drivers supported -->
<dependency>
<groupId>mysql</groupId>
Expand Down Expand Up @@ -167,6 +186,12 @@
<version>${spring-version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>3.6.8.Final</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-entitymanager</artifactId>
Expand Down Expand Up @@ -198,4 +223,14 @@
<scope>test</scope>
</dependency>
</dependencies>

<reporting>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>findbugs-maven-plugin</artifactId>
<version>${findbugs-maven-version}</version>
</plugin>
</plugins>
</reporting>
</project>
5 changes: 5 additions & 0 deletions release_notes.txt
@@ -1,3 +1,8 @@
New in 1.7.12
Many minor code improvements
Minor fixes for performance test
Now uses Spring 3.2.6

New in 1.7.11
Added greater precision when persisting float values

Expand Down
@@ -1,5 +1,6 @@
package com.opower.persistence.jpile.infile;

import com.google.common.collect.ImmutableList;
import com.opower.persistence.jpile.infile.driver.C3P0JdbcDriverSupport;
import com.opower.persistence.jpile.infile.driver.HikariJdbcDriverSupport;
import com.opower.persistence.jpile.infile.driver.MysqlJdbcDriverSupport;
Expand All @@ -12,8 +13,6 @@
import java.util.ArrayList;
import java.util.List;

import static com.google.common.collect.ImmutableList.of;

/**
* Generic Spring callback for executing the 'LOAD DATA INFILE' pattern of streaming data in
* batch to MySQL. This will not work on databases other than MySQL. It requires simply
Expand All @@ -37,7 +36,7 @@
public class InfileStatementCallback implements JdbcUtil.StatementCallback<List<Exception>> {

private static final List<JdbcDriverSupport> SUPPORTED_DRIVERS =
of(new HikariJdbcDriverSupport(), new C3P0JdbcDriverSupport(), new MysqlJdbcDriverSupport());
ImmutableList.of(new HikariJdbcDriverSupport(), new C3P0JdbcDriverSupport(), new MysqlJdbcDriverSupport());

// SQL statement
private String loadInfileSql;
Expand All @@ -60,7 +59,7 @@ public List<Exception> doInStatement(Statement statement) throws SQLException {
for (JdbcDriverSupport support : SUPPORTED_DRIVERS) {
if (support.accept(statement)) {
support.doWithStatement(statement, this.inputStream);
statement.execute(loadInfileSql);
statement.execute(this.loadInfileSql);
return extractWarnings(statement.getWarnings());
}
}
Expand All @@ -74,7 +73,7 @@ public List<Exception> doInStatement(Statement statement) throws SQLException {
* @return list of warnings
*/
private List<Exception> extractWarnings(SQLWarning warning) {
List<Exception> warnings = new ArrayList<Exception>(1000);
List<Exception> warnings = new ArrayList<>(1000);
while (warning != null) {
warnings.add(warning);
warning = warning.getNextWarning();
Expand All @@ -96,10 +95,10 @@ public interface JdbcDriverSupport {
boolean accept(Statement statement);

/**
* Sets the inputstream correctly on the statement
* Sets the input stream correctly on the statement
*
* @param statement the statement
* @param inputStream the inputstream to set
* @param inputStream the input stream to set
* @throws java.sql.SQLException if a sql exception occurs
*/
void doWithStatement(Statement statement, InputStream inputStream) throws SQLException;
Expand Down
Expand Up @@ -5,7 +5,6 @@
import com.opower.persistence.jpile.infile.InfileStatementCallback;

import java.io.InputStream;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.sql.SQLException;
import java.sql.Statement;
Expand Down Expand Up @@ -42,17 +41,11 @@ public boolean accept(Statement statement) {
@Override
public void doWithStatement(Statement statement, InputStream inputStream) throws SQLException {
try {
Method m = com.mysql.jdbc.Statement.class.getMethod(INFILE_MUTATOR_METHOD, new Class[]{InputStream.class});
Method m = com.mysql.jdbc.Statement.class.getMethod(INFILE_MUTATOR_METHOD, InputStream.class);
C3P0ProxyStatement proxyStatement = (C3P0ProxyStatement) statement;
proxyStatement.rawStatementOperation(m, C3P0ProxyStatement.RAW_STATEMENT, new Object[]{inputStream});
}
catch (NoSuchMethodException e) {
throw Throwables.propagate(e);
}
catch (IllegalAccessException e) {
throw Throwables.propagate(e);
}
catch (InvocationTargetException e) {
catch (ReflectiveOperationException e) {
throw Throwables.propagate(e);
}
}
Expand Down

0 comments on commit c0c43fd

Please sign in to comment.