JPA read side documentation

docs/build.sbt

@@ -27,7 +27,8 @@ lazy val docs = project
       "com.typesafe.play" %% "play-logback" % PlayVersion % Test,
       "org.apache.logging.log4j" % "log4j-api" % "2.7" % "test",
       "com.softwaremill.macwire" %% "macros" % "2.2.5" % "provided",
-      "org.projectlombok" % "lombok" % "1.16.10"
+      "org.projectlombok" % "lombok" % "1.16.10",
+      "org.hibernate" % "hibernate-core" % "5.2.5.Final"
     ),
     javacOptions ++= Seq("-encoding", "UTF-8", "-source", "1.8", "-target", "1.8", "-parameters", "-Xlint:unchecked", "-Xlint:deprecation"),
     testOptions in Test += Tests.Argument("-oDF"),
@@ -59,10 +60,24 @@ lazy val docs = project
     markdownStageIncludeWebJars := false,
     markdownSourceUrl := Some(url(s"https://github.com/lagom/lagom/edit/$branch/docs/manual/"))
 
-  ).dependsOn(serviceIntegrationTestsJavadsl, persistenceJdbcJavadsl,
-      serviceIntegrationTestsScaladsl, persistenceCassandraScaladsl, persistenceJdbcScaladsl, testkitJavadsl,
-      testkitScaladsl, brokerScaladsl, playJson, kafkaBroker, pubsubScaladsl, immutables % "test->compile", 
-      theme % "run-markdown", devmodeScaladsl)
+  )
+  .dependsOn(
+    serviceIntegrationTestsJavadsl,
+    persistenceJdbcJavadsl,
+    persistenceJpaJavadsl,
+    serviceIntegrationTestsScaladsl,
+    persistenceCassandraScaladsl,
+    persistenceJdbcScaladsl,
+    testkitJavadsl,
+    testkitScaladsl,
+    brokerScaladsl,
+    playJson,
+    kafkaBroker,
+    pubsubScaladsl,
+    immutables % "test->compile",
+    theme % "run-markdown",
+    devmodeScaladsl
+  )
 
 lazy val parentDir = Path.fileProperty("user.dir").getParentFile
 
@@ -71,6 +86,7 @@ lazy val serviceIntegrationTestsJavadsl = ProjectRef(parentDir, "integration-tes
 lazy val serviceIntegrationTestsScaladsl = ProjectRef(parentDir, "integration-tests-scaladsl")
 lazy val persistenceJdbcJavadsl = ProjectRef(parentDir, "persistence-jdbc-javadsl")
 lazy val persistenceJdbcScaladsl = ProjectRef(parentDir, "persistence-jdbc-scaladsl")
+lazy val persistenceJpaJavadsl = ProjectRef(parentDir, "persistence-jpa-javadsl")
 lazy val persistenceCassandraScaladsl = ProjectRef(parentDir, "persistence-cassandra-scaladsl")
 lazy val testkitJavadsl = ProjectRef(parentDir, "testkit-javadsl")
 lazy val testkitScaladsl = ProjectRef(parentDir, "testkit-scaladsl")

docs/manual/java/guide/cluster/ReadSideJDBC.md

@@ -0,0 +1,85 @@
+# JDBC Read-Side support
+
+This page is specifically about Lagom's support for relational database read-sides using JDBC.  Before reading this, you should familiarize yourself with Lagom's general [[read-side support|ReadSide]] and [[relational database read-side support overview|ReadSideRDBMS]].
+
+## Query the Read-Side Database
+
+Let us first look at how a service implementation can retrieve data from a relational database using JDBC.
+
+@[imports](code/docs/home/persistence/JdbcReadSideQuery.java)
+@[service-impl](code/docs/home/persistence/JdbcReadSideQuery.java)
+
+Note that the [`JdbcSession`](api/index.html?com/lightbend/lagom/javadsl/persistence/jdbc/JdbcSession.html) is injected in the constructor. `JdbcSession` allows access to a connection from the connection pool, using the `withConnection` method, and will manage transactions using the `withTransaction` method.  Importantly, `JdbcSession` also manages execution of the blocking JDBC calls in a thread pool designed to handle it, which is why the `withConnection` and `withTransaction` methods return `CompletionStage`.
+
+## Update the Read-Side
+
+We need to transform the events generated by the [[Persistent Entities|PersistentEntity]] into database tables that can be queried as illustrated in the previous section. For that we will implement a [`ReadSideProcessor`](api/index.html?com/lightbend/lagom/javadsl/persistence/ReadSideProcessor.html) with assistance from the [`JdbcReadSide`](api/index.html?com/lightbend/lagom/javadsl/persistence/jdbc/JdbcReadSide.html) support component. It will consume events produced by persistent entities and update one or more database tables that are optimized for queries.
+
+This is how a `ReadSideProcessor` class looks like before filling in the implementation details:
+
+@[imports](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+@[initial](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+
+You can see that we have injected the JDBC read-side support, this will be needed later.
+
+You should already have implemented tagging for your events as described in the [[Read-Side documentation|ReadSide]], so first we'll implement the `aggregateTags` method in our read-side processor stub, like so:
+
+@[tag](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+
+### Building the read-side handler
+
+The other method on the `ReadSideProcessor` is `buildHandler`.  This is responsible for creating the [ReadSideHandler](api/index.html?com/lightbend/lagom/javadsl/persistence/ReadSideProcessor.ReadSideHandler.html) that will handle events.  It also gives the opportunity to run two callbacks, one is a global prepare callback, the other is a regular prepare callback.
+
+[`JdbcReadSide`](api/index.html?com/lightbend/lagom/javadsl/persistence/jdbc/JdbcReadSide.html) has a `builder` method for creating a builder for these handlers, this builder will create a handler that will automatically manage transactions and handle read-side offsets for you.  It can be created like so:
+
+@[create-builder](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+
+The argument passed to this method is an identifier for the read-side processor that Lagom should use when it persists the offset. Lagom will store the offsets in a table that it will automatically create itself if it doesn't exist. If you would prefer that Lagom didn't automatically create this table for you, you can turn off this feature by setting `lagom.persistence.jdbc.create-tables.auto=false` in `application.conf`. The DDL for the schema for this table is as follows:
+
+```sql
+CREATE TABLE read_side_offsets (
+  read_side_id VARCHAR(255), tag VARCHAR(255),
+  sequence_offset bigint, time_uuid_offset char(36),
+  PRIMARY KEY (read_side_id, tag)
+)
+```
+
+### Global prepare
+
+The global prepare callback runs at least once across the whole cluster.  It is intended for doing things like creating tables and preparing any data that needs to be available before read side processing starts.  Read side processors may be sharded across many nodes, and so tasks like creating tables should usually only be done from one node.
+
+The global prepare callback is run from an Akka cluster singleton.  It may be run multiple times - every time a new node becomes the new singleton, the callback will be run.  Consequently, the task must be idempotent.  If it fails, it will be run again using an exponential backoff, and the read side processing of the whole cluster will not start until it has run successfully.
+
+Of course, setting a global prepare callback is completely optional, you may prefer to manage database tables manually, but it is very convenient for development and test environments to use this callback to create them for you.
+
+Below is an example method that we've implemented to create tables:
+
+@[create-table](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+
+It can then be registered as the global prepare callback in the `buildHandler` method:
+
+@[register-global-prepare](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+
+### Prepare
+
+In addition to the global prepare callback, there is also a prepare callback that can be specified by calling [`builder.setPrepare`](api/index.html?com/lightbend/lagom/javadsl/persistence/jdbc/JdbcReadSide.ReadSideHandlerBuilder.html#setPrepare-com.lightbend.lagom.javadsl.persistence.jdbc.JdbcReadSide.ConnectionBiConsumer-). This will be executed once per shard, when the read side processor starts up.
+
+If you read the [[Cassandra read-side support|ReadSideCassandra]] guide, you may have seen this used to prepare database statements for later use. JDBC `PreparedStatement` instances, however, are not guaranteed to be thread-safe, so the prepare callback should not be used for this purpose with relational databases.
+
+Again this callback is optional, and in our example we have no need for a prepare callback, so none is specified.
+
+### Event handlers
+
+The event handlers take an event and a connection, and update the read-side accordingly.
+
+Here's an example callback for handling the `PostAdded` event:
+
+@[post-added](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+
+This can then be registered with the builder using `setEventHandler`:
+
+@[set-event-handler](code/docs/home/persistence/JdbcBlogEventProcessor.java)
+
+Once you have finished registering all your event handlers, you can invoke the `build` method and return the built handler:
+
+@[build](code/docs/home/persistence/JdbcBlogEventProcessor.java)

docs/manual/java/guide/cluster/ReadSideJPA.md

@@ -0,0 +1,138 @@
+# JPA Read-Side support
+
+This page is specifically about Lagom's support for relational database read-sides using JPA.  Before reading this, you should familiarize yourself with Lagom's general [[read-side support|ReadSide]] and [[relational database read-side support overview|ReadSideRDBMS]].
+
+## Project dependencies
+
+To use JPA support, add the following in your project's build:
+
+In Maven:
+
+```xml
+<dependency>
+    <groupId>com.lightbend.lagom</groupId>
+    <artifactId>lagom-javadsl-persistence-jpa_2.11</artifactId>
+    <version>${lagom.version}</version>
+</dependency>
+```
+
+In sbt:
+
+@[jpa-dependency](code/build-cluster.sbt)
+
+You will also need to add dependencies on your JPA provider (such as [Hibernate ORM](http://hibernate.org/orm/) or [EclipseLink](http://www.eclipse.org/eclipselink/)) and database driver.
+
+
+## Configuration
+
+JPA support builds on top of Lagom's support for [[storing persistent entities in a relational database|PersistentEntityRDBMS]]. See that guide for instructions on configuring Lagom to use the correct JDBC driver and database URL.
+
+Next, we need to configure JPA to communicate with our database, and optionally configure Lagom to initialize a JPA persistence unit.
+
+JPA is configured using a file called [`persistence.xml`](http://docs.oracle.com/javaee/7/tutorial/persistence-intro003.htm#JEETT01162). Create a file at `src/main/resources/META-INF/persistence.xml` in your service implementation project using this template as a guide:
+
+@[persistence-unit](code/docs/home/persistence/persistence.xml)
+
+By default, Lagom expects the persistence unit to be named "default", as it is in this example, but this can be changed in your `application.conf`.
+
+Initializing the persistence unit requires communicating with the configured database. Lagom will automatically retry initialization if it fails up to a maximum number of retries before failing permanently and exiting. The maximum number of retries, initial retry interval, and optional back-off factor are all configurable in `application.conf`.
+
+The full set of configuration options that Lagom provides for initializing JPA is here:
+
+@[persistence](../../../../../persistence-jpa/javadsl/src/main/resources/reference.conf)
+
+## Write a JPA entity class
+
+JPA [entities](http://docs.oracle.com/javaee/7/tutorial/persistence-intro001.htm#BNBQA) represent tables in the read-side database. Here is an example of a JPA entity representing a summary of a blog post, which could be used to query for an index of all blog entries:
+
+@[entity](code/docs/home/persistence/BlogSummaryJpaEntity.java)
+
+Note that JPA entities are required to follow the typical JavaBeans style of mutable objects with getters and setters, instead of Lagom's usage of [[immutable objects|Immutable]]. JPA entities are not thread safe, and it is important to ensure that they're only used within the scope of a Lagom-managed transaction. We'll see how to accomplish this later.
+
+## Query the Read-Side Database
+
+Let us next look at how a service implementation can retrieve data from a relational database using JPA.
+
+@[imports](code/docs/home/persistence/JpaReadSideQuery.java)
+@[service-impl](code/docs/home/persistence/JpaReadSideQuery.java)
+
+Note that the [`JpaSession`](api/index.html?com/lightbend/lagom/javadsl/persistence/jpa/JpaSession.html) is injected in the constructor. `JpaSession` allows access to the JPA `EntityManager`, and will manage transactions using the `withTransaction` method.  Importantly, `JpaSession` also manages execution of the blocking JPA calls in a thread pool designed to handle it, which is why the `withTransaction` method returns `CompletionStage`.
+
+As noted above, it's important to prevent mutable JPA entity instances from escaping the thread used to execute the blocking JPA calls. To achieve this, in the query itself, we use a JPQL [constructor expression](http://docs.oracle.com/javaee/7/tutorial/persistence-querylanguage005.htm#JEETT00746) to return immutable `PostSummary` instances from the query instead of mutable `BlogSummaryJpaEntity` instances. JPA requires constructor expressions to use the fully-qualified name of the class to construct. You could also convert to immutable data in other ways, such as by returning JPA entities from your query and then converting them explicitly, but use of constructor expressions is a convenient way to do this that avoids extra code and object allocation.
+
+## Update the Read-Side
+
+We need to transform the events generated by the [[Persistent Entities|PersistentEntity]] into database tables that can be queried as illustrated in the previous section. For that we will implement a [`ReadSideProcessor`](api/index.html?com/lightbend/lagom/javadsl/persistence/ReadSideProcessor.html) with assistance from the [`JpaReadSide`](api/index.html?com/lightbend/lagom/javadsl/persistence/jpa/JpaReadSide.html) support component. It will consume events produced by persistent entities and update one or more database tables that are optimized for queries.
+
+This is how a `ReadSideProcessor` class looks like before filling in the implementation details:
+
+@[imports](code/docs/home/persistence/JpaBlogEventProcessor.java)
+@[initial](code/docs/home/persistence/JpaBlogEventProcessor.java)
+
+You can see that we have injected the JPA read-side support, this will be needed later.
+
+You should already have implemented tagging for your events as described in the [[Read-Side documentation|ReadSide]], so first we'll implement the `aggregateTags` method in our read-side processor stub, like so:
+
+@[tag](code/docs/home/persistence/JpaBlogEventProcessor.java)
+
+### Building the read-side handler
+
+The other method on the `ReadSideProcessor` is `buildHandler`.  This is responsible for creating the [ReadSideHandler](api/index.html?com/lightbend/lagom/javadsl/persistence/ReadSideProcessor.ReadSideHandler.html) that will handle events.  It also gives the opportunity to run two callbacks, one is a global prepare callback, the other is a regular prepare callback.
+
+[`JpaReadSide`](api/index.html?com/lightbend/lagom/javadsl/persistence/jpa/JpaReadSide.html) has a `builder` method for creating a builder for these handlers, this builder will create a handler that will automatically manage transactions and handle read-side offsets for you.  It can be created like so:
+
+@[create-builder](code/docs/home/persistence/JpaBlogEventProcessor.java)
+
+The argument passed to this method is an identifier for the read-side processor that Lagom should use when it persists the offset. Lagom will store the offsets in a table that it will automatically create itself if it doesn't exist. If you would prefer that Lagom didn't automatically create this table for you, you can turn off this feature by setting `lagom.persistence.jdbc.create-tables.auto=false` in `application.conf`. The DDL for the schema for this table is as follows:
+
+```sql
+CREATE TABLE read_side_offsets (
+  read_side_id VARCHAR(255), tag VARCHAR(255),
+  sequence_offset bigint, time_uuid_offset char(36),
+  PRIMARY KEY (read_side_id, tag)
+)
+```
+
+### Global prepare
+
+The global prepare callback runs at least once across the whole cluster.  It is intended for doing things like creating tables and preparing any data that needs to be available before read side processing starts.  Read side processors may be sharded across many nodes, and so tasks like creating tables should usually only be done from one node.
+
+The global prepare callback is run from an Akka cluster singleton.  It may be run multiple times - every time a new node becomes the new singleton, the callback will be run.  Consequently, the task must be idempotent.  If it fails, it will be run again using an exponential backoff, and the read side processing of the whole cluster will not start until it has run successfully.
+
+Of course, setting a global prepare callback is completely optional, you may prefer to manage database tables manually, but it is very convenient for development and test environments to use this callback to create them for you.
+
+Below is an example method that we've implemented to create the schema:
+
+@[create-schema](code/docs/home/persistence/JpaBlogEventProcessor.java)
+
+In this case, we're using the JPA [`generateSchema`](https://docs.oracle.com/javaee/7/api/javax/persistence/Persistence.html#generateSchema-java.lang.String-java.util.Map-) method along with a Hibernate-specific property that can add missing tables and columns to existing schemas, as well as create the schema from scratch, but won't remove any existing data.
+
+It can then be registered as the global prepare callback in the `buildHandler` method:
+
+@[register-global-prepare](code/docs/home/persistence/JpaBlogEventProcessor.java)
+
+### Prepare
+
+In addition to the global prepare callback, there is also a prepare callback that can be specified by calling [`builder.setPrepare`](api/index.html?com/lightbend/lagom/javadsl/persistence/jpa/JpaReadSide.ReadSideHandlerBuilder.html#setPrepare-java.util.function.BiConsumer-). This will be executed once per shard, when the read side processor starts up.
+
+If you read the [[Cassandra read-side support|ReadSideCassandra]] guide, you may have seen this used to prepare database statements for later use. JPA `Query` and `CriteriaQuery` instances, however, are not guaranteed to be thread-safe, so the prepare callback should not be used for this purpose with relational databases.
+
+Again this callback is optional, and in our example we have no need for a prepare callback, so none is specified.
+
+### Event handlers
+
+The event handlers take an event and a JPA `EntityManger`, and update the read-side accordingly.
+
+Here's an example callback for handling the `PostAdded` event:
+
+@[post-added](code/docs/home/persistence/JpaBlogEventProcessor.java)
+
+This can then be registered with the builder using `setEventHandler`:
+
+@[set-event-handler](code/docs/home/persistence/JpaBlogEventProcessor.java)
+
+Event handlers, as well as callbacks, are automatically wrapped in a transaction that commits automatically when the handler succeeds or rolls back when it throws an exception. It's safe to use JPA entities in your event handlers, but as noted above, it's important to ensure that they do not escape into other threads. You can assign them to local variables, as in this example, or pass them as arguments to synchronous methods that don't retain a reference to the entities in some other scope. Avoid assigning JPA entities to instance or static fields, providing them to code that executes in another thread, or passing them to methods that might do so themselves.
+
+Once you have finished registering all your event handlers, you can invoke the `build` method and return the built handler:
+
+@[build](code/docs/home/persistence/JpaBlogEventProcessor.java)