Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HSEARCH-2594 JSR-352 (Batch) integration #1566

Closed
wants to merge 123 commits into from

Conversation

@yrodiere
Copy link
Member

commented Oct 18, 2017

https://hibernate.atlassian.net//browse/HSEARCH-2594

This adds JSR-352 integration through two publicly exposed modules:

  • hibernate-search-jsr352-core, which contains all runtime-independent code (and happens to be enough for the JBatch reference implementation)
  • hibernate-search-jsr352-jberet, which adds jberet-specific glue, mainly related to CDI (which is not mandatory for JSR-352 runtimes)

There are a few remaining sub-tasks, but I think I'll move them to independent tickets once this PR has been merged, because they are not critical: https://hibernate.atlassian.net/issues/?jql=parent%20%3D%20HSEARCH-2594%20and%20resolution%20is%20empty%20ORDER%20BY%20status%20DESC

Note that there are quite a lot of commits, but I already tried my best to squash commits together and remove merge commits (I removed all merge commits), while still preserving authorship information and avoiding huge commits that nobody will undestand.

I added performance tests as part of this PR. Results are available here: http://ci.hibernate.org/view/Search/job/hibernate-search-performance-orm/. See in particular how the FileSystemSessionMassIndexer and FileSystemJsr352MassIndexer scenarios resulted in similar execution times.

To make your review as painless as possible, I would recommend to proceed like this:

  • Read the documentation (manual-index.asciidoc, marker jsr352-integration). If it's not clear how to use it after that, we probably have to work on it.
  • Have a look at integration tests:
    • org.hibernate.search.jsr352.massindexing.BatchIndexingJobIT#simple
    • org.hibernate.search.jsr352.massindexing.BatchIndexingJobIT#criteria
    • org.hibernate.search.jsr352.massindexing.BatchIndexingJobIT#hql
    • org.hibernate.search.jsr352.massindexing.BatchIndexingJobIT#partitioned
    • org.hibernate.search.jsr352.massindexing.RestartChunkIT#failureDuringFirstCheckpointBetweenTwoWrites_fullScope
  • Read the architecture description in jsr352/README.md
  • Dive into the full diff... Reviewing commit by commit will probably be way too much work.
@gunnarmorling

This comment has been minimized.

Copy link
Member

commented Oct 18, 2017

@mincong-h

This comment has been minimized.

Copy link
Contributor

commented Oct 18, 2017

Thank you @yrodiere for all your contributions and helps of this contribution 💪 💪 👏 👏

@yrodiere

This comment has been minimized.

Copy link
Member Author

commented Oct 18, 2017

@mincong-h Thanks to you for your work! I know that JSR-352 can be full of surprises, not all of which are pleasant ones ;)
But let's wait for the review before we celebrate :)

@Sanne

This comment has been minimized.

Copy link
Member

commented Oct 20, 2017

Awesome job everyone! I plan to dedicate a couple of days next week to playing with this.

@Sanne Sanne added Needs rebase and removed Ready for review labels Oct 25, 2017

@yrodiere yrodiere force-pushed the yrodiere:HSEARCH-2594 branch from 6f57b7d to 4a6483b Oct 25, 2017

mincong-h and others added 20 commits Apr 15, 2016
HSEARCH-2594 Initial work on JSR-352 integration
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/
It is the result of squashing commits
135a190 to f9d1716
(see git log below).

commit f9d1716
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jul 5 02:14:30 2016 +0200

    #40 add IT for checkpoint: RestartChunkIT

commit 5e5e177
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 4 16:38:27 2016 +0200

    #81 correct spelling error

commit d442d06
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 4 09:27:40 2016 +0200

    #74 change unit test MassIndexerTest

commit 236e1d8
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 4 09:21:42 2016 +0200

    #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)`

commit 58f8f32
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 4 09:07:54 2016 +0200

    #81 clean up gitignore, parent pom, mysql module in WF

commit 9d569b1
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 3 23:47:24 2016 +0200

    #81 clean up IT in Java SE
    - properties are not used locally

commit e9e1574
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 3 23:43:44 2016 +0200

    #81 clean up IT in Java EE

    In pom.xml
    - Properties are not used. Properties should be provided by parent module or core module.
    - JBeret SE dependencies are useless.
    - We don't use absolute path here
    - There's not webapp to package. Arquillian create war archive with its own dependencies.
    - Exclude surefire because there's no unit test here.

    In persistence.xml
    - It should be placed in the META-INF folder as the standard.

    In MassIndexerIT
    - Change persistence.xml's location

commit 5027a33
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 3 23:08:19 2016 +0200

    #81 clean up core/pom.xml
    - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict.
    - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more.
    - Same principal for JSTL.
    - Test resources were set for integration test. But now, ITs have been moved to independent modules.
    - Plugin for dependency path is not used since there isn't any absolute path in the project
    - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more.
    - Unit test (surefire) does not need any additional requirement for instant
    - No IT in core module, so no failsafe, no WF unpack, no resources copy

commit 32f6b3b
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 3 20:47:57 2016 +0200

    #63 add value assertions in Java SE integration test

commit e6a5699
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 3 16:32:24 2016 +0200

    #63 deny JobOperator construction inside the MassIndexerImpl
    pass it through the setter method.

commit 25adb82
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 3 12:32:38 2016 +0200

    #63 fix JobOperator instantiation issue
    In the previous version, the job operator cannot be instantiated and show the below error :

    java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated
            at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136)
            at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589)
            at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29)
            at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118)
            at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
            at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
            at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
            at java.lang.Class.newInstance(Class.java:442)
            at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
            at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
            at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
            at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57)
            at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41)

    And I found out that if there's only one @test annotation in the integration test, the problem get fixed.
    I don't know why and it is very weird.

commit 5a99766
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 2 19:14:15 2016 +0200

    #80 LuceneWork execution should be asynchronous

commit 75bcf3e
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 2 19:11:35 2016 +0200

    #73 change ref assignment for EntityManager inside the job

commit bd0fe3a
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 1 19:14:12 2016 +0200

    #63 replace entity `Address`, `Stock` by `Company`

commit 36f8ce1
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 1 06:35:57 2016 +0200

    #76 update README

commit 5ac099c
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 1 06:31:33 2016 +0200

    #69 leave mysql and use only h2

commit cd6289f
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 29 23:42:18 2016 +0200

    #69 code enhancement

commit 18e2965
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 29 23:36:49 2016 +0200

    #69 change addEntityCount to Synchronized mode

commit ad3b5b9
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 29 23:16:44 2016 +0200

    #67 extend timeout value
    the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality

commit accf7e1
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 29 22:20:24 2016 +0200

    #69 code enhancement

commit 2151742
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 28 07:51:54 2016 +0200

    #63 try to use my own MassIndexer (failed)
    revert to Gunnar's one in MassIndexerIT.java
    However, there're still some more dependencies added

commit 1fc14a7
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 28 00:01:34 2016 +0200

    #63 replace official indexer by my own indexer (Java SE)
    but this is not the finished version

commit 889e782
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 27 21:29:30 2016 +0200

    #68 use embedded H2 database for Java SE testing

commit 7e49180
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 27 12:56:35 2016 +0200

    #63 separate core module and integration tests modules
    and remove useless resources files, e.g. sql & csv

commit 51e2636
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 27 11:45:11 2016 +0200

    #63 create core module

commit ed557c0
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 27 10:42:29 2016 +0200

    #63 configure pom file for Java SE IT module

commit 6ddbb7c
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 26 23:54:12 2016 +0200

    #63 start module hibernate-search-integrationtest-javase

commit 931ca09
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 16:02:37 2016 +0200

    #69 add batch status check

commit 3a89e11
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 11:52:48 2016 +0200

    #71 move persistence.xml to src/test/resources

commit 64727ba
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 10:51:02 2016 +0200

    #69 write count test

commit ab9df58
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 10:50:16 2016 +0200

    #70 IdProducerBatchlet produces less IDs than expected

commit 917a011
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 10:49:17 2016 +0200

    #43 add lucene files to .gitignore

commit 9d8214c
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 10:06:38 2016 +0200

    #69 read count test

commit 7b61a5d
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 10:05:36 2016 +0200

    #69 add BatchTestHelper, copied from javaee7-samples

commit 4a3941f
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 25 10:04:22 2016 +0200

    #49 change logger level

commit 47aec79
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jun 23 23:40:32 2016 +0200

    #60 some mixed enhancement

commit be25a49
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jun 23 23:37:42 2016 +0200

    #65 fix entity type usage
    use IndexingContext instead of batch properties

commit 2f40c3c
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jun 23 23:34:56 2016 +0200

    #49 change log level

commit 1ff59ad
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jun 23 23:30:19 2016 +0200

    #46 Rename package into org.hibernate.search.jsr352

commit 254d44f
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jun 23 23:28:45 2016 +0200

    #66 use remote mysql server

commit 1d4be37
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 22 00:12:10 2016 +0200

    #60 add JBoss Deployment Structure File
    for class loading

commit 29e5a24
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 22 00:11:23 2016 +0200

    #60 add shrink-wrap resolver dependency
    for war archive creation

commit 655e1ff
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 22 00:09:19 2016 +0200

    #60 change WF to debug mode

commit 73decd1
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 22 00:06:36 2016 +0200

    #60 fix PU naming error

commit 8a0438e
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 22 00:05:23 2016 +0200

    #60 move src/main/webapp to test/main/webapp

commit e9c858c
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 22 00:04:18 2016 +0200

    #60 remove wrong placed persistence.xml

commit cd9aca8
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 21 19:01:26 2016 +0200

    #64 disable IT log redirection

commit 5426048
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 20 23:17:32 2016 +0200

    #59 remove ds.xml and include mysql jar file

commit f8d292c
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 20 22:15:47 2016 +0200

    #59 add mysql resources to src/wildflyConfig

commit 00d45bc
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 20 11:17:46 2016 +0200

    #58 add datasource for MySQL

commit 64a1b5c
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 20 09:46:27 2016 +0200

    #57 clarify unit test and integration test
    - rename DeploymentTest.java to DeploymentIT.java
    - change pom.xml configuration

commit 9a95b24
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 19 20:33:03 2016 +0200

    #49 use Logger instead of System.out.println

commit 0449388
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 19 19:38:25 2016 +0200

    #28 change README content
    The previous demos have been deleted. The current README should only focus on 3 points :
    1. What's new using JSR352 implementation ?
    2. How to run it ?
    3. Why we need it ?

commit 7c6662c
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 19 17:09:57 2016 +0200

    #50 delete MANIFEST.MF

commit 95690a9
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 19 15:42:14 2016 +0200

    #54 add unit test for mass indexer

commit 522f178
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 19 15:40:46 2016 +0200

    #48 separate interface and implementation

commit 38c4f54
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 19 09:09:33 2016 +0200

    #52 add deprecated class manually during war creation

commit 31610a7
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 19 08:37:59 2016 +0200

    #52 delete commented static configuration

commit 4884f46
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 18 21:47:54 2016 +0200

    #52 💩 only god knows why it works 💦 💦 💦
    - pom.xml
      - add arquillian protocol support (not sure if necessary)
      - change WF from embedded to managed
      - add tag <testResources> to enable arquillian xml file lookup
      - add bit strings to find the absolute path to dependencies (not sure if necessary)
      - add detailed configuration to unit test plugin surefire (copied from HSEARCH)
      - add failsafe (not sure if necessary)
      - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory)
      - add resources plugin to transfer the defined WF config from src to test directory
    - DeploymentTest
      - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found
    - arquillian.xml
      - disable the "javaVmArguments", otherwise the server cannot start
    - wildflyCondif/*
      - config for WF container

commit 675c319
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jun 16 23:21:50 2016 +0200

    #52 add Arquillian feature (failed)
    - add JUnit as a maven dependency
    - add arquillian components as maven dependencies (not finished, don't know exactly what to add)
    - add wildfly-arquillian-container-embedded, but not sure how it works
    - add class DeploymentTest

commit 68082d7
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 15 21:31:09 2016 +0200

    #48 split of public API and internal implementation packages

commit d339328
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 15 20:48:51 2016 +0200

    #37 move test classes from src/main/java to src/test/java
    - entities
    - sessions
    - servlets (should be deleted soon)
    - META-INF (except batch XML file mass-index.xml)

commit a67ff0d
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 15 20:07:30 2016 +0200

    #45 Fetch WildFly for testing via maven-dependency-plugin
    There're other minor modifications for fixing warnings during the build.

commit 47f2895
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 14 23:20:33 2016 +0200

    #14 add dependency and modify artifact info

commit 522f1ac
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 14 22:45:03 2016 +0200

    #41 Move content of folder jsr352 to the root folder

commit 5c46ff1
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 14 22:13:57 2016 +0200

    #41 remove jse-chunk and zoo-jpa

commit 7fd6cd2
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 14 02:27:58 2016 +0200

    #35 update project name in pom.xml

commit ac276b9
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 14 02:26:28 2016 +0200

    #42 change Deployment Assembly settings

commit a7a3e93
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 14 02:24:05 2016 +0200

    #41 add SQL script for importing CSV files

commit d6024c6
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 21:49:46 2016 +0200

    #41 add SQL script for importing yahoo-finance-rht.csv

commit c8f6634
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 21:44:51 2016 +0200

    #41 change yahoo finance stock CSV file for redhat to lowercase

commit 7ed4223
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 21:36:58 2016 +0200

    #41 provide MySQL dump script for creating tables (without data)

commit 917b64a
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 21:13:09 2016 +0200

    #36 Move "./WebContents" to "./src/main/webapp"

commit 81adb41
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 21:10:18 2016 +0200

    #14 update pom.xml
    - add dependencies
    - rename artifactId (issue #35)
    - change folder location from ./WebContent to ./src/main/webapp (issue #36)

commit b258cf4
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 19:45:57 2016 +0200

    #35 rename "us-address" to "jsr352"

commit 6f80618
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 17:46:02 2016 +0200

    #28 change diagram for mass-indexer (overview)
    Prepare for the first tag. This image should have smaller width and all components up to date.

commit fa1e12c
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 17:21:27 2016 +0200

    #41 update javadoc and enhance code

commit 1fb8ae6
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 16:26:46 2016 +0200

    #41 update javadoc for chunk-step "produceLuceneDoc"
    Reader / Processor / Writer

commit a8f7aaa
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 15:47:00 2016 +0200

    #41 clean code for IdProducerBatchlet

commit 9820a46
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 14:32:45 2016 +0200

    #34 use IndexingContext for entity row count

commit 430ea9e
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 01:12:27 2016 +0200

    #33 change job to use dynamic configuration of partition plan

commit e733a7f
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 13 01:06:09 2016 +0200

    #22 use generic type in BatchItemProcessor

commit 0b28f46
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 11 21:55:24 2016 +0200

    #32 add EntityPartitionMapper

commit 518b035
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 11 20:45:06 2016 +0200

    #31 get class type Class<?> from String

commit b734016
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 11 16:08:47 2016 +0200

    #30 add table `stock`, provided by yahoo finance

commit 61b5406
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 11 15:13:42 2016 +0200

    #24 #25 add <decision> tags in job "mass-index" work flow

commit 6967351
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 11 15:08:57 2016 +0200

    #28 change diagram for mass indexer (overview)
    The image provided in the commit is generated by the JBoss tool in Eclipse,
    so it helps to standardize the progress display

commit c65f41d
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 8 20:07:03 2016 +0200

    #24 change decision id in job xml

commit d127e1e
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 7 23:42:49 2016 +0200

    #28 add diagram for mass indexer (chunk)

commit 2f314a1
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 7 23:28:31 2016 +0200

    #28 resize image

commit e43b4ce
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jun 7 22:59:02 2016 +0200

    #28 add diagram for mass indexer (overview)

commit 36440be
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 6 21:46:24 2016 +0200

    #24 add decider and purger batchlet before the doc production chunk
    Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet.

commit 55f0137
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 6 19:52:29 2016 +0200

    #26 delete IdReaderBatchlet
    This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion.

commit 7d2b5b5
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 6 19:47:28 2016 +0200

    #26 delete previous demo's chunk classes
    AddressReader + AddressProcessor + AddressWriter + Job XML

commit 4474125
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 6 17:26:53 2016 +0200

    #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue

commit e4c9f04
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 6 12:27:42 2016 +0200

    #22 Use generic type in BatchItemProcessor
    Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map>

commit 2a67c66
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 6 10:55:21 2016 +0200

    #4 performance test for indexing 1M entities
    10:51:58,187 INFO  [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%).
    10:51:58,187 INFO  [stdout] (Batch Thread - 1) #analyzeStatus(...) called.
    10:51:58,187 INFO  [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called.
    10:51:58,188 INFO  [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called.
    10:51:58,466 INFO  [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms.

commit 9ccde4f
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jun 6 10:45:32 2016 +0200

    #21 modify variable naming and javadoc to clarify the analyzer

commit 911990f
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 5 21:36:27 2016 +0200

    #21 show mass index progress using PartitionAnalyzer

commit 7072f6f
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 5 18:59:08 2016 +0200

    #19 decrease item-count to avoid OutOfMemory exception

commit 392a40b
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 5 18:58:06 2016 +0200

    #17 pass IndexShardingStrategy through IndexingContext

commit de93ad6
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 5 18:56:33 2016 +0200

    #21 Collect and analyse items processed from different partitions
    Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet.

commit 6d2ae3f
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jun 5 17:59:34 2016 +0200

    #20 disable the cache in a session

commit 9b8faa8
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jun 4 19:07:22 2016 +0200

    #13 add AddLuceneWork execution in job "mass-index"
    The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress.
    The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer.

commit 81891b9
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jun 2 20:54:21 2016 +0200

    #13 change BatchItemWriter to do lucene work.
    The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions.

commit 6d70db1
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jun 1 23:33:05 2016 +0200

    #15 fix unaware how to convert value

commit c6e8fe1
Author: mincong <mincong.h@gmail.com>
Date:   Mon May 30 23:48:06 2016 +0200

    #13 add javadoc

commit 4e7fcdc
Author: mincong <mincong.h@gmail.com>
Date:   Mon May 30 21:50:44 2016 +0200

    #13 add documentation for reader

commit 0ec55aa
Author: mincong <mincong.h@gmail.com>
Date:   Mon May 30 21:50:12 2016 +0200

    #13 change processor's role and add documentation

commit e636368
Author: mincong <mincong.h@gmail.com>
Date:   Mon May 30 13:43:45 2016 +0200

    #13 mass indexer using JSR 352 - id production

commit 382fe8c
Author: mincong <mincong.h@gmail.com>
Date:   Sun May 29 20:38:25 2016 +0200

    #7 Parallel processing for IdReaderBatchlet

commit 941c3b0
Author: mincong <mincong.h@gmail.com>
Date:   Sat May 28 22:51:08 2016 +0200

    #10 Use customized context to pass properties between steps

commit c361d22
Author: mincong <mincong.h@gmail.com>
Date:   Sat May 28 21:25:37 2016 +0200

    #8 change data structure from ArrayList to Array (4.0% faster)

commit 5739a13
Author: mincong <mincong.h@gmail.com>
Date:   Sat May 28 12:00:02 2016 +0200

    #9 Set batchlet properties via job xml
    Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release.

commit 14a638a
Author: mincong <mincong.h@gmail.com>
Date:   Fri May 27 18:29:37 2016 +0200

    load ids v2
    Now the class IdProducerBatchlet can load all entities' ids and print to terminal.

commit 25c2014
Author: mincong <mincong.h@gmail.com>
Date:   Fri May 27 00:38:56 2016 +0200

    add id producer batchlet (not finished)
    This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer

commit eadadae
Author: mincong <mincong.h@gmail.com>
Date:   Sun May 22 22:36:51 2016 +0200

    execute LuceneWork in AddressProcessor

commit 53881cd
Author: mincong <mincong.h@gmail.com>
Date:   Sun May 22 20:15:21 2016 +0200

    #1 move operations related to session from processor to reader

commit aaf0441
Author: mincong <mincong.h@gmail.com>
Date:   Sat May 21 16:28:22 2016 +0200

    change batch job from a print demo to lucene demo

commit b72158b
Author: mincong <mincong.h@gmail.com>
Date:   Mon May 16 00:40:24 2016 +0200

    add demo batch job `print-addresses-job`

commit 539c244
Author: mincong <mincong.h@gmail.com>
Date:   Sat May 14 15:21:23 2016 +0200

    building demo jse-chunk

commit a5db451
Author: mincong <mincong.h@gmail.com>
Date:   Thu May 5 21:50:33 2016 +0200

    add different types of query
    - keyword
    - fuzzy
    - wildcard

commit dcde70e
Author: mincong <mincong.h@gmail.com>
Date:   Thu May 5 17:48:13 2016 +0200

    add basic search feature (keyword query)

commit 4a103d1
Author: mincong <mincong.h@gmail.com>
Date:   Sun May 1 15:45:48 2016 +0200

    optimise mass indexer settings
    https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer

commit 2b2ac86
Author: mincong <mincong.h@gmail.com>
Date:   Sun May 1 14:21:29 2016 +0200

    change hibernate search method `index()` into async mode

commit 322c58a
Author: mincong <mincong.h@gmail.com>
Date:   Sun May 1 09:50:00 2016 +0200

    add Hibernate Search feature into `us-address`

commit 05b0ee1
Author: mincong <mincong.h@gmail.com>
Date:   Sat Apr 30 11:08:25 2016 +0200

    add JPA example `us-address`
    The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later.

commit 135a190
Author: mincong <mincong.h@gmail.com>
Date:   Sat Apr 16 17:03:37 2016 +0200

    add example zoo-jpa
HSEARCH-2594 Adjustments on JSR-352 integration
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/
It is the result of squashing commits
f8c7156 to a1cd1a6
(see git log below).

commit a1cd1a6
Author: Gunnar Morling <gunnar.morling@googlemail.com>
Date:   Tue Jul 5 17:35:27 2016 +0200

    #92 Exposing entities to index via batch job context

commit 257f8e6
Author: Gunnar Morling <gunnar.morling@googlemail.com>
Date:   Tue Jul 5 12:35:15 2016 +0200

    #93 Avoiding usage of Class#forName()

commit 2f0c4a0
Author: Gunnar Morling <gunnar.morling@googlemail.com>
Date:   Tue Jul 5 12:33:15 2016 +0200

    #91 Creating EntityIndexingStepData to share data within the entity indexing batch step;

    * It's set up before the item reader runs via a listener
    * Avoids concurrency issues of sharing the index sharding strategy via the IndexingContext

commit de0b0ca
Author: Gunnar Morling <gunnar.morling@googlemail.com>
Date:   Tue Jul 5 10:28:57 2016 +0200

    #88 Cleaning up Maven dependencies

commit f8c7156
Author: Gunnar Morling <gunnar.morling@googlemail.com>
Date:   Tue Jul 5 09:59:49 2016 +0200

    #78 Flushing out pending index works after batch job has run
HSEARCH-2594 Initial work on JSR-352 - second part
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/
It is the result of squashing commits
691bced to 4ee5027
(see git log below).

Also, a binary file (image) was removed.

commit 4ee5027
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Wed Aug 17 08:11:36 2016 +0200

    #132 add module "modules" v1

commit 0da6342
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Mon Aug 15 23:37:03 2016 +0200

    #122 update doc
    - delete index-optimization doc
    - rename massindexer352 to user-guide
    - add developer-guide #130

commit 694f855
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Mon Aug 15 14:33:54 2016 +0200

    #137 move the BM script to annotations

commit 8b446c0
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Mon Aug 15 13:59:13 2016 +0200

    #135 Use more specific names for chunk classes

commit 7c5e4f1
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sun Aug 14 19:38:41 2016 +0200

    #130 prove the indexed results

commit ed7d35d
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sun Aug 14 16:11:13 2016 +0200

    #130 performance test

commit 7eeefb0
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sun Aug 14 11:11:45 2016 +0200

    #113 add transaction timeout to avoid em warning
    could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED

commit d380e1e
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 13 16:41:36 2016 +0200

    #133 use purgeAll from FullTextEntityManager

commit 106071e
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 13 14:20:23 2016 +0200

    #128 enhance value assertion and util in test

commit a709c10
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 13 12:55:43 2016 +0200

    #96 enable test RestartChunkIT
    - #131 batch status test
    - #123 use byteman to stop the job

commit 6848144
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Fri Aug 12 22:16:24 2016 +0200

    #131 enable batch status test

commit 91b52c3
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 11 23:23:55 2016 +0200

    #133 clarification for emf and jobOperator under JavaSE

commit 3495eaa
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 11 19:52:29 2016 +0200

    #133 merge mass indexer interface and impl

commit e76bd5e
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 11 19:34:18 2016 +0200

    #133 rename analyzer and collector

commit 72a6211
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 11 19:06:38 2016 +0200

    #133 update comments in PartitionAnalyzer

commit 6f55657
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 11 18:41:37 2016 +0200

    #133 delete unused comparators

commit f57d08e
Author: mincong <mincong.h@gmail.com>
Date:   Thu Aug 11 00:49:30 2016 +0200

    #122 add persistence configuration

commit 76fdeac
Author: mincong <mincong.h@gmail.com>
Date:   Tue Aug 9 08:55:19 2016 +0200

    #127 prove the existence of root entities

commit 7ba5450
Author: mincong <mincong.h@gmail.com>
Date:   Tue Aug 9 07:48:23 2016 +0200

    #127 use mock & avoid add getters / setters for tests

commit 0f42691
Author: mincong <mincong.h@gmail.com>
Date:   Mon Aug 8 21:11:34 2016 +0200

    #125 attribute naming and position enhancement
    - use full upper case for static final attributes
    - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last.

commit f091d19
Author: mincong <mincong.h@gmail.com>
Date:   Mon Aug 8 20:24:24 2016 +0200

    #125 extends abstract class to decrease code complexity

commit 88e89f4
Author: mincong <mincong.h@gmail.com>
Date:   Sun Aug 7 15:51:09 2016 +0200

    #119 unwrap session from emf directly

commit 251ff72
Author: mincong <mincong.h@gmail.com>
Date:   Sun Aug 7 15:43:16 2016 +0200

    #125 separate obtention of ID name in an util class

commit 5777c32
Author: mincong <mincong.h@gmail.com>
Date:   Sun Aug 7 15:25:22 2016 +0200

    #125 update javadoc and clean up code

commit 967db61
Author: mincong <mincong.h@gmail.com>
Date:   Sun Aug 7 14:54:49 2016 +0200

    #126 fix boundary setting error

commit 750b7f6
Author: mincong <mincong.h@gmail.com>
Date:   Sun Aug 7 10:52:18 2016 +0200

    #96 enable mass indexer under Java SE

commit 6a85592
Author: mincong <mincong.h@gmail.com>
Date:   Sat Aug 6 18:50:25 2016 +0200

    #125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch

commit 35ff8f6
Author: mincong <mincong.h@gmail.com>
Date:   Sat Aug 6 18:10:11 2016 +0200

    #21 rebuild the indexing monitor

commit fc70ac8
Author: mincong <mincong.h@gmail.com>
Date:   Sat Aug 6 10:42:41 2016 +0200

    #122 enhance documentation
    - introduction
    - example usage
    - different configurations

commit 2bf1a50
Author: mincong <mincong.h@gmail.com>
Date:   Fri Aug 5 22:59:01 2016 +0200

    #114 delete utility class referenced from other project

commit 896794d
Author: mincong <mincong.h@gmail.com>
Date:   Fri Aug 5 22:09:23 2016 +0200

    #55 add unit test for ItemReader

commit 7e36592
Author: mincong <mincong.h@gmail.com>
Date:   Fri Aug 5 20:58:36 2016 +0200

    #125 rename attribute partitionCapacity -> rowsPerPartition

commit 4e0cb5a
Author: mincong <mincong.h@gmail.com>
Date:   Fri Aug 5 20:30:07 2016 +0200

    #74 accept single class type as entry for root entity(s) selection

commit 35c2c1f
Author: mincong <mincong.h@gmail.com>
Date:   Fri Aug 5 20:20:43 2016 +0200

    #125 fix typo

commit 66316bb
Author: mincong <mincong.h@gmail.com>
Date:   Fri Aug 5 20:07:52 2016 +0200

    #125 add javadoc & license for PartitionMapperTest

commit dea58bd
Author: mincong <mincong.h@gmail.com>
Date:   Thu Aug 4 23:45:05 2016 +0200

    #121 mock for PartitionMapper

commit e29fa33
Author: mincong <mincong.h@gmail.com>
Date:   Thu Aug 4 23:43:53 2016 +0200

    #125 add toString to test entity types

commit db2d444
Author: mincong <mincong.h@gmail.com>
Date:   Tue Aug 2 23:50:35 2016 +0200

    #121 mock for PartitionMapper
    not finished

commit 2c88280
Author: mincong <mincong.h@gmail.com>
Date:   Mon Aug 1 16:47:42 2016 +0200

    #122 add documentation

commit 296f621
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 31 12:38:09 2016 +0200

    #109 prove the entire dataset is indexed
    using all context query

commit 949c2c5
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 31 10:00:54 2016 +0200

    #122 create document module

commit 61619e4
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 31 00:49:50 2016 +0200

    #116 revert changes : use partitionCapacity again

commit 1f304b0
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 30 16:30:42 2016 +0200

    #124 enhance code

commit de7a96d
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 30 12:25:11 2016 +0200

    #120 #124 enhance the boundary logic
    Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements :
    - move embedded class PartitionMapper._Unit to PartitionUnit
    - embed PartitionBoundary class to PartitionUnit
    - rename partitionIndex to partitionID

commit 0be59ab
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 30 09:22:44 2016 +0200

    #120 enhance the partition boundary concept
    - cancel the "unique partition" concept, one entity type should have at least 2 partitions
    - change the closed interval to Right-Closed interval

commit c64f4ec
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 30 00:05:35 2016 +0200

    #108 several enhancements for partitioning
    - add an additional partition at the end of each entity type to handle new inserted entities
    - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge

commit bb69ee0
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 29 00:19:46 2016 +0200

    #120 change R-closed interval to full closed interval

commit ad7b077
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 28 23:56:29 2016 +0200

    #120 replace firstID and lastID by divider
    Different dividers provide a partition boundary mechanism.

commit a7ab1e3
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 28 00:23:21 2016 +0200

    #109 prove the entire dataset is indexed in RestartIT

commit 701555e
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 27 23:50:55 2016 +0200

    #123 adapt MassIndexerIT to RestartIT
    so that the restartability can be proved inside this class.

commit 818caa3
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 27 23:49:33 2016 +0200

    #123 fixe typo error for Byteman script in maven pom.xml

commit 5324146
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 27 23:48:09 2016 +0200

    #124 delete unused method in Helper class

commit ffb8edd
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 27 23:47:21 2016 +0200

    #89 fix typo error for "family"

commit 1aefe2a
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 27 19:40:56 2016 +0200

    #123 change Byteman roles and enable job restart in IT

commit ec45c74
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jul 26 23:51:34 2016 +0200

    #119 avoid using entity manager
    SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage.

commit deb2685
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jul 26 20:14:24 2016 +0200

    #117 add Java agent Byteman for arquillian integration test

commit 84a8909
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 21:42:39 2016 +0200

    #118 add "try catch finally" for all concerned classes

commit cbf94fa
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 20:58:55 2016 +0200

    #118 add "try catch finally" structure for PartitionMapper

commit 3462cb6
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 15:46:35 2016 +0200

    #118 add try finally in mapper

commit 289cc96
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 14:21:44 2016 +0200

    #115 add cacheable and fetchSize as batch property

commit d552c6b
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 12:06:40 2016 +0200

    #115 enable the entity MyDate.class in MassIndexerIT

commit 2556c8d
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 11:58:26 2016 +0200

    #115 cleanup partition mapper, delegate map to job context data

commit 925963d
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 10:43:50 2016 +0200

    #115 add toString() to entity type Person.class

commit 12f5a8f
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 10:42:38 2016 +0200

    #108 ID should be ordered before scrolling

commit 654aadf
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 25 03:23:03 2016 +0200

    #108 use range approach to define partition v1
    available for String ID (still have problem for criteria expression in item reader)

commit 6063c21
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 24 21:15:54 2016 +0200

    #108 use range approach to define partition v1
    available for integer ID

commit 13c8bd4
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 24 14:36:43 2016 +0200

    #107 add hashcode for entity and enable SQL

commit 981a371
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 24 12:30:51 2016 +0200

    #107 use the remainder approach for item reader
    Currently only available for integer ID.

commit f30a4ed
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 24 10:44:20 2016 +0200

    #115 format XML file to adapt Hibernate code style

commit 5d11e1c
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 24 10:38:03 2016 +0200

    #109 delete byteman config
    For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel.

commit ceacc8c
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 24 01:07:50 2016 +0200

    #117 add byteman to Java EE tests

commit b020c26
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 23 09:47:47 2016 +0200

    #116 update javadoc and cleanup code

commit dc45ece
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 22 23:51:53 2016 +0200

    #116 enhancement for partitioning
    code cleanup for better understanding

commit 3de0007
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 21 23:50:10 2016 +0200

    #116 define partitions instead of partitionCapacity in the job properties

commit 19c1a1c
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 20 18:35:39 2016 +0200

    #112 revert useless clean up

commit 306c922
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jul 19 23:59:15 2016 +0200

    #112 clear em before it is closed

commit 2b3831e
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jul 19 23:44:37 2016 +0200

    #112 inject emf instead of em to control the lifecycle

commit 4b7b77a
Author: mincong <mincong.h@gmail.com>
Date:   Tue Jul 19 20:26:36 2016 +0200

    #115 add class description for JobContextSetupListener

commit ef9e247
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 18 13:00:22 2016 +0200

    #106 enhance code & doc for internal classes

commit 3eba510
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 18 11:56:53 2016 +0200

    #106 clean up for class MassIndexerImpl

commit 5b86bcd
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 18 11:45:21 2016 +0200

    #106 add javadoc for interface MassIndexer

commit ff60058
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 23:11:16 2016 +0200

    #106 update image URL

commit d3b4a0a
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 23:06:32 2016 +0200

    #106 update description, add workflow diagram

commit fbf58e3
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 22:31:40 2016 +0200

    #106 delete PartitionReducer (we don't need it)

commit 3afd079
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 18:19:12 2016 +0200

    #25 add the afterChunk enhancement

commit 960a8ad
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 17:52:02 2016 +0200

    #24 rename package: purge -> beforeChunk

commit 02cb248
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 17:32:10 2016 +0200

    #24 separate the beforeChunk enhancements from the chunk step

commit b7765a9
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 15:56:26 2016 +0200

    #106 cleanup maven

commit c74e783
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 13:10:14 2016 +0200

    #89 add restart tests in MassIndexerIT under JavaEE

commit f9eef03
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 10:33:20 2016 +0200

    #106 clean up MassIndexerIT in Java EE

commit 3400718
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 17 00:25:15 2016 +0200

    #89 add entity type MyDate.class & test in IT (JEE)

commit 98e6e42
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 16 12:44:17 2016 +0200

    #24 purge at start enhancement

commit d49678a
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 15 22:58:53 2016 +0200

    #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup
    Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86

commit b93b2b1
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 15 21:59:38 2016 +0200

    #84 change step context usage & cleanup job params
    - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint.
    - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore
    - update related tests

commit b2e8377
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 15 18:32:49 2016 +0200

    #84 rename params inside the step context
    "partitionNumber" -> "scrollOffset"
    "partitionSize" -> "scrollInterval"

commit 905b8d7
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 15 18:13:53 2016 +0200

    #84 cleanup job params

commit 9940adc
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 15 00:17:21 2016 +0200

    #104 partitioning for multiple entity types

commit 21d9195
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 14 18:58:51 2016 +0200

    #104 multiple partitions for one entity type using scroll mechanism

commit e895220
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 14 12:56:59 2016 +0200

    #105 change author info

commit 36ff3c1
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 14 11:37:31 2016 +0200

    #102 increase target entities' number from 100 to 1000

commit 8bfe365
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 14 11:29:03 2016 +0200

    #67 increase TX in the customized config standalone-full-testqueues.xml

commit 3be9290
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 14 10:28:56 2016 +0200

    #102 Move creation of Lucene doc to ItemProcessor

commit 7154abb
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 13 22:31:14 2016 +0200

    #96 skip entity manager handling for Java SE + code clean up
    rename PartitionedContextData to StepContextData so that it is easier for memorising.
    The related part of the README file has been updated too.

commit 2705279
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 13 21:34:23 2016 +0200

    #96 skip tests in Java SE

commit de6338e
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 21:45:27 2016 +0200

    #86 avoid CDI injection before the job start

commit de9cb46
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 12:11:11 2016 +0200

    #89 change Person's ID into more complex string expression

commit 020eb91
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 12:07:41 2016 +0200

    #97 add order in item reader

commit eb3eb0f
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 11:39:04 2016 +0200

    #89 add Person.class into Java SE integration test

commit 43e8557
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 09:52:57 2016 +0200

    #100 increase README readability

commit fcc55fe
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 01:41:02 2016 +0200

    #100 update README composing

commit 74d909b
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 01:38:20 2016 +0200

    #100 update README composing

commit 1bd3e5e
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 01:29:30 2016 +0200

    #100 explain project's mechanism in README

commit 28942f9
Author: mincong <mincong.h@gmail.com>
Date:   Mon Jul 11 00:10:15 2016 +0200

    #92 rename BatchContextSetupListener to JobContextSetupListener

commit 09b7d03
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 10 23:54:16 2016 +0200

    #89 add entity type Person.class

commit b14d304
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 10 19:29:52 2016 +0200

    #82 move Java SE based test to core module

commit 22d67b6
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 10 16:21:51 2016 +0200

    #92 simply count process inside the transient / persistent data

commit bccdb05
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 10 07:31:09 2016 +0200

    #99 delete ContextSetupListener

commit 864feeb
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 10 01:17:50 2016 +0200

    #92 cleanup StepContext
    EntityType / EntityClazz will only be stored in JobContext using BatchContextData

commit 4711d52
Author: mincong <mincong.h@gmail.com>
Date:   Sun Jul 10 00:49:56 2016 +0200

    #92 exclude entity-sharding-strategy from context data
    The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one.

commit 9f857d3
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 9 18:25:01 2016 +0200

    #99 Disable AbstractItemReadListener

commit c1512d8
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 9 14:17:18 2016 +0200

    #98 use EntityName & EntityClazz

commit 3cda1ae
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 9 13:47:41 2016 +0200

    #85 get ID using EntityIndexBinding

commit a3c4bed
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 9 11:03:05 2016 +0200

    #81 clean up code in mass indexer

commit 08c06c0
Author: mincong <mincong.h@gmail.com>
Date:   Sat Jul 9 10:27:38 2016 +0200

    #90 enable customizable checkpoint algorithm

commit fb9c71e
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 8 23:12:51 2016 +0200

    #61 change project version to 5.5.4.Final

commit 37401bd
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 8 23:03:09 2016 +0200

    #87 add License LGPL

commit 19c0971
Author: mincong <mincong.h@gmail.com>
Date:   Fri Jul 8 22:34:46 2016 +0200

    #87 Apply Hibernate code style

commit 856c89a
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 7 23:42:24 2016 +0200

    #94 create dedicated sub-packages under internal.steps
    one package per step

commit 7ec9e14
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 7 23:05:55 2016 +0200

    #97 finish chunk step redesign & tests associated

commit 21fe2ea
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 7 21:10:25 2016 +0200

    #97 add stateless session (still failed)
    code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession"
    http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate

commit d31442e
Author: mincong <mincong.h@gmail.com>
Date:   Thu Jul 7 02:07:44 2016 +0200

    #97 redesign the step execution "chunk"

commit 691bced
Author: mincong <mincong.h@gmail.com>
Date:   Wed Jul 6 18:20:18 2016 +0200

    #78 embed final flush into ItemWriter
    So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
HSEARCH-2594 Initial work on JSR-352 - third part
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/
It is the result of squashing commits
576f105 to 40aff727a4ca70dd809ca72dcb6315d71f80a37f
(see git log below).

commit 40aff727a4ca70dd809ca72dcb6315d71f80a37f
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Jan 5 23:18:43 2017 +0100

    #168 test batch job runnability over a poorly-formed entity
    It has multiple ID-like fields, which make it difficult to identify the real identifier.

commit 3cd83d9b5d9c055ee133d66a5af7e9c027a572dc
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Jan 5 22:22:33 2017 +0100

    #165 delete unused import

commit 58fabd11044b64f82f69432061fe1a37ad8967a4
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Jan 5 21:55:17 2017 +0100

    #165 remove duplicate ID projection

commit d0c23f934851866ecfc797c889a9c3f69971208c
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Jan 5 21:49:01 2017 +0100

    #165 avoid using toString() method

commit 15d2952a30d14384ed9a1a514b7a9a72b2d30ae6
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Tue Jan 3 22:34:49 2017 +0100

    #165 use `Projections#id()`
    instead of `DocumentBuilderIndexedEntity#getIdPropertyName()` to avoid null reference issue for provided IDs.

commit 89624d52f7b6b955bbf45d576b538b8fc93445fb
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 21:40:45 2016 +0100

    #162 refactor entitiyType

commit c15633f0a2912b2cf50531c7742ae37b201d1229
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 21:13:59 2016 +0100

    #162 increase default itemCount to 200

commit 21be31a72a59e933006c1ccdc9e372b76a979098
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 20:27:12 2016 +0100

    #162 rename ID to Id

commit 81a39117cb8f9c8d0086c2252c2a3c759488fa4f
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 20:13:51 2016 +0100

    #162 rename entityClazz to entityType

commit 6a7f42a8b3f3ef23dc6df3d93ea0c8b69210b5cb
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 17:10:03 2016 +0100

    #162 rename PartitionUnit to PartitionBound

commit 58add35f70ff943c915be7aeaf03b411db1847e7
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 16:49:48 2016 +0100

    #162 refactor job context data setup

commit 0e942211a850a0d5c5b6c5a20f6e9a441123a8e0
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 15:56:51 2016 +0100

    #162 format code according to hibernate-ide-codestyles
    https://github.com/hibernate/hibernate-ide-codestyles

commit d0c9d69f2a702b997d40160f0aba6960fd22c4f8
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 12:10:43 2016 +0100

    #161 use getName() instead of toString()

commit d26d5c9acf9d08d303f1e2f1f728227a66bad87f
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Dec 8 11:15:29 2016 +0100

    #134 rename StepContextData to PartitionContextData
    to provide a better understanding

commit f4983aacf9b14faebcbe8f553eb68d5182caea0a
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Tue Dec 6 17:37:12 2016 +0100

    #134 persist indexing progress using `setPersistentData`

commit 3dd8b30bbaf623acddd3f240a9a0ca3583a5b488
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Tue Dec 6 12:48:42 2016 +0100

    #134 fix indexing progress error: avoid storing increment

commit a2c5ffcb60e99eff7245bd4999d81079f31db4e0
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Tue Dec 6 12:10:29 2016 +0100

    #134 delete unused attributes
    `isRestarted` and `rowsToIndex`

commit f4bcbd422a735364b9f90f71f1e1bf198d983eb5
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Tue Dec 6 11:22:37 2016 +0100

    #134 refactor process of indexing-progress-monitoring

commit 3c6b1d373a5096f993a77198e239acbe7a7fd844
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Mon Dec 5 21:42:45 2016 +0100

    #134 add class StepProgress, storing indexing progress at step level

commit bc037f453f1b25dd86b60f5d3e5e34f3eff584b1
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Fri Dec 2 09:44:05 2016 +0100

    #87 rename lgpl.txt to LICENSE.txt

commit 9ab00469a589cf9bdb118a6f952760c8e555836f
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Mon Nov 21 10:59:41 2016 +0100

    #160 add Travis CI status

commit cd5b933953dc4c523541709ccbe0100632823950
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Mon Nov 21 10:50:47 2016 +0100

    #160 set up Travis CI

commit 0229fdb56e9218994cdc4d7793553337ba93d476
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Fri Sep 30 21:39:53 2016 +0200

    #146 misc. enhance & use another test scenario for WF
    which demonstrates the possibility to index recent inserted entities

commit 13bc769a101c8f4cfa53fbe46433abdae2047a98
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Sep 29 16:29:48 2016 +0200

    #133 type and naming enhancement

commit b7ca7eab62eec0457ee56586c66c90946364c6f0
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Sep 29 15:44:26 2016 +0200

    #151 rename "massIndex" / "mass-index" -> "BatchIndexingJob"

commit 031e6a02fabc8fb7ee731fdf3a8258f88aab0e6a
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Sep 29 15:28:17 2016 +0200

    #156 #159 refactor the MassIndexer -> BatchIndexingJob
    - Make a second level Builder class Builder.
    - Avoid using constructor of MassIndexer.
    - Various arguments of type Class<?>, Class<?>... for builder to allow at least 1 argument.
    - Rename MassIndexer to BatchIndexingJob
    - Avoid a static reference to the EMF
    - Update all the test cases according to the changes

commit 950106e036309e24560a599b9132644d3b328e74
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Wed Sep 28 13:26:39 2016 +0200

    #133 the batch insertion counter need to be incremented

commit 4b8f5525a9ab716618bb42f0b90de0e5d77f5ec9
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Tue Sep 27 16:59:52 2016 +0200

    #133 require at least one element for MassIndexer#addRootEntities(...)

commit 556e88e74d5f39212af86712bd767821f7839de0
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Tue Sep 27 10:50:42 2016 +0200

    #146 Allow to select the entities through a HQL/JPQL query

commit ff9f4c6dfdee918d4f59d1e4a03b0765105746db
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sun Sep 25 22:10:50 2016 +0200

    #150 add test for validating the job start/stop in CLI

commit 4258265f34ddb69b0e14c15a9f30d20ca6170b9a
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sun Sep 25 17:37:00 2016 +0200

    #133 rename the war using the class's name

commit 038d53c0ac765f0ebd3826323c8c01a23b9255ad
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Sep 22 17:29:13 2016 +0200

    #136 refactor mocked tests
    Use real EntityManagerFactory instead of mocked one. Batch properties are now passed using a protected args-constructor. After the refactoring, there's no need to use PowerMock anymore.

commit 217d560424171e83864dfe55f6d08d955baeb763
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sun Sep 18 21:23:13 2016 +0200

    #158 store Object as job property, e.g. Hibernate criterion
    enable selection of entities to be re-indexed through criteria

commit e3a097a7eec8057b9c105684c9ab473645d08d3e
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Fri Sep 16 16:58:44 2016 +0200

    #154 banne caching option for PartitionMapper

commit 3bf541a6ebfd4aa7d10a6d1475e4977a2fbb706d
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Sep 15 18:36:22 2016 +0200

    #153 avoid SNAPSHOT versions among the dependencies

commit de4be0b1478c66237537847d27868059d38e981f
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Sep 15 18:09:03 2016 +0200

    #152 use java.lang.String for all the batch properties

commit e4e1ed4a33fcd616ccf5d58e065e77b85bcbf187
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Fri Aug 26 21:46:36 2016 +0200

    #132 remove dummy job xml

commit e52939897253a79320ad471645637275e1e3b6a8
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Fri Aug 26 21:41:38 2016 +0200

    #132 enable the module approach

commit a4fa8430b43ce138180eda8ce682d393ce55f473
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Wed Aug 24 23:47:53 2016 +0200

    #132 enable arquillian-managed WF's debug

commit 188fba5902ca1dc3777048db8039721ada32abb3
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 25 22:57:10 2016 +0200

    #132 WFLY-7000 Batch jobs from installed modules should be detected for non-batch app

commit 68ccad9abb1562f0f8cd0bbdc4b5057469fb7f77
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Wed Aug 24 23:47:53 2016 +0200

    #132 enable arquillian-managed WF's debug

commit 88c2dac3e7a7a7b2f8b7f4cc93d10d197b1b5244
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Mon Aug 22 23:55:34 2016 +0200

    #132 add org.jberet.jberet-core as a module dependency
    to fix unsatisfied dependencies issue

commit 3fb229edb54f78c79208391a722168751b7811e0
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 15:10:20 2016 +0200

    #132 implement an SPI like this ?

commit c44d5f7
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 23:54:34 2016 +0200

    #133 avoid dependence on the existing mass indexer
    for batchlets

commit 5c4add3
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 22:47:33 2016 +0200

    #143 change log level to INFO

commit e1b8229
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 22:47:06 2016 +0200

    #143 maven cleanup

commit 3c111e5
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 22:40:22 2016 +0200

    #140 change project version to 1.0

commit 1070a6d
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 22:08:44 2016 +0200

    #143 add issue management

commit 5ae4508
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 22:03:48 2016 +0200

    #143 maven clean up

commit e9bad75
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 20:59:43 2016 +0200

    #143 add license and developers info

commit 1b9f491
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 20:41:07 2016 +0200

    #139 separate simple integration test and performance test

commit 3a18b7b
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 18:43:59 2016 +0200

    #143 delete profiles

commit 35d8ba7
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 18:40:53 2016 +0200

    #143 revert WF module installation approach

commit a78086f
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sat Aug 20 17:27:29 2016 +0200

    #143 rename maven module javaee-wildfly to wildfly

commit d9fadc8
Merge: 50908f9 ca09828
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 18 21:15:00 2016 +0200

    #132 import meta-inf from dependency

commit 50908f9
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 18 02:06:07 2016 +0200

    #132 update for IT

commit 576f105
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Wed Aug 17 23:50:36 2016 +0200

    #132 package batch job as module for WildFly

commit ca09828
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Thu Aug 18 02:06:07 2016 +0200

    #132 update for IT

commit 69440de
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Wed Aug 17 23:50:36 2016 +0200

    #132 package batch job as module for WildFly
HSEARCH-2594 Adjustments on JSR-352 integration
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/
It is the result of squashing commits
94dc13e12b8477b357a4d6fcb3bff3008813e73a to 6a9df85c0d7351d83e05cc7062b9a18fbb6b3701
(see git log below).

commit 6a9df85c0d7351d83e05cc7062b9a18fbb6b3701
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 12 13:05:47 2017 +0100

    Fix a NPE when the conversion from EntityManagerFactory to SessionFactory fails

commit 596454f6d6bdfe89cf99646033e5b6a2b38d5934
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 11 17:35:15 2017 +0100

    Fix the dependencies in the WildFly module

commit f25e3a21327d301bae3f4f31a0062924a6f85f0a
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 11 17:31:41 2017 +0100

    Align the project version on the Hibernate Search version

    This avoids some weird issues with snippets of poms copied over
    from Hibernate Search.

commit 4830e5cf19a33a267c8bf7927910e0c21cda3d1c
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 11 11:43:39 2017 +0100

    Upgrade to Hibernate Search 5.7.0.Beta2 / Hibernate ORM 5.2.1.Final

    This will make it easier to work with SessionFactory if we have to,
    because starting from Hibernate ORM 5.2 SessionFactory extends
    EntityManagerFactory.

commit c903608a10f3c3db7db34ffbc693edcf7499d6a6
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Tue Jan 10 10:14:39 2017 +0100

    Reduce the use of JobSEEnvironment and @PersistenceUnit to a minimum

    That's because:

     * JobSEEnvironment will have to be replaced by something else, see #156 .
     * Injected persistence units do not work when CDI is disabled, so we
       might as well only use it where it is absolutely necessary.

commit 65aa7fec8d2a0e5c35f600cb5bad9ef822a08d45
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 11 14:48:46 2017 +0100

    i178 Remove "jobContextData" from the job parameters

commit db591e1c02deb1295306c9fdb7fb0bcb7cc85945
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 11 16:18:48 2017 +0100

    Add missing "static" keywords

commit 94dc13e12b8477b357a4d6fcb3bff3008813e73a
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 11 15:17:29 2017 +0100

    Also run integration tests in travis
HSEARCH-2594 Adjustments on JSR-352 integration
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/
It is the result of squashing commits
dba44895fa898554ddf635329d271faaf1ec5429 to 11eae4c207eac61e5ae105264fc02693db5f7faa
(see git log below).

commit 11eae4c207eac61e5ae105264fc02693db5f7faa
Author: Mincong HUANG <mincong.h@gmail.com>
Date:   Sun Jan 22 23:30:18 2017 +0100

    #184 avoid using fixed order for test executions

commit dba44895fa898554ddf635329d271faaf1ec5429
Author: Mincong HUANG <mincong-h@users.noreply.github.com>
Date:   Mon Jan 16 10:21:35 2017 +0100

    #179 clean up provided dependencies

    * #72 clean up dependencies

    * Clean up dependencies
    - "jboss-batch-api_1.0_spec" is replaced by "javax.batch-api"
    - "javax.batch-api" has scope provided, implementation depends on user's choice / platform
    - "javax.batch-api" version is managed in the parent POM

    * i72 Manage the scope "provided" of APIs in the parent pom

    * i72 Clean up some more dependencies

     * Manage the "provided" scope of APIs in the parent pom
     * Manage most dependency versions in the parent pom
     * Remove unneeded explicit dependencies (hibernate-entitymanager not
       needed since ORM 5.2, hibernate-jpa-2.1-api not needed since we
       depend on hibernate-core)

    I left the explicit versioning of jberet/byteman in -core for now.
HSEARCH-2594 Rework of EntityManagerFactory access in JSR-352 integra…
…tion + cleanup

This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/
It is the result of squashing commits
bb23592e2712e05427b019d7194ebed22e3678e9 to 5417e3bbeab1f07184022290a88a4a9ebb0b8876
(see git log below).

commit 5417e3bbeab1f07184022290a88a4a9ebb0b8876
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Feb 8 16:01:01 2017 +0100

    Remove unnecessary empty lines at the top of code blocks

commit b91226c5ea1e2e8ddc31dbba15b31a2b57a13163
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Feb 8 15:51:32 2017 +0100

    i175 Clearly separate massindexing-specific code using a dedicated package

commit 37299a8f92f88416bb0c562032d315e2ccd74756
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Feb 8 15:39:45 2017 +0100

    Remove useless <class> tags in persistence.xml

    Hibernate auto-detects entities by default, so let's not worry about
    those things: it's just tests, and fqcn in text files are a pain when
    refactoring.

commit 5353e39129791e7ea0f127ce684b768bfceb91e9
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Feb 8 15:00:53 2017 +0100

    i175 Rename packages to follow HSearch conventions

     1. No capital letters in package naming (Java convention)
     2. Implementation/internal packages are named "impl" (HSearch convention)

commit 3a8f030adabc46410297d18c507592a3f669675d
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Feb 8 12:27:07 2017 +0100

    i175 Set the parent project to hibernate-search-parent and take advantage of this

    This removes a bunch of dependency management entries, properties, etc.
    because they are already provided in hibernate-search-parent.

    Other changes:

     * Some forbidden uses of locale-sensitive methods had to be adapted
     * WildFly deployment structure files had to be adapted to support
       Hibernate ORM 5.2.7
     * The build-config project was removed (we use Hibernate Search's)

commit 0feac2208643844b4514febded855077595164a5
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Mon Jan 23 16:38:35 2017 +0100

    i156 Improve documentation around JobContextListener and EMFRegistries

commit 3ec1537bcce0c14c769eae8b162683a9d3ca9ac6
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Mon Jan 23 15:02:32 2017 +0100

    i156 Add a missing endline in an XML file

commit 84c617ad12bbf00c92454fa82ebcea4fd7e177bc
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 19 19:21:10 2017 +0100

    i156 Update the documentation regarding the entity manager factory selection.

commit 82835fe28b963ce6ce54da9de4c7afd0917637ec
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 19 19:17:35 2017 +0100

    Fix the syntax of code examples in the docs (this avoids issues with empty lines)

commit 387851570992bbb0a6dedffdbd11d56765c1d8c1
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 19 18:32:25 2017 +0100

    i156 Tweak the scope concept for EntityManagerFactory retrieval so that defaults are easier to the user

    Now when no scope is provided, we will try to retrieve an entity manager
    factory by any means that make sense for the current context (CDI/non-CDI):
    we won't be limited by the default scope. This means in particular that
    users who created a single SessionFactory using the Hibernate APIs will
    still be able to use this factory as a default (before this change, only
    factories created through the JPA APIs would be be detected by default).

    The main change is that there is now only one registry per context
    (CDI/non-CDI), allowing this registry to use more methods to get the
    default entity manager factory.

    Incidentally, this means that it's no longer possible to use non-CDI
    scopes in a CDI environment. It doesn't seem too bad, but if necessary
    we can restore the old behavior easily anyway (by making the CDI
    registry delegate to the non-CDI one).

commit e0ccbc1bea7631a41b7529ddead1e5e249f8c415
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 19 17:01:33 2017 +0100

    Update some obviously outdated code examples in the docs.

commit 1c3ed1d9719a5f7ece08a456140a5a2b82c0947f
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 19 16:35:39 2017 +0100

    When using CDI, retrieve the EntityManagerFactory using the @PersistenceUnit annotation if there is no EntityManagerFactory.

commit 67aebc648eda2c17f15df617f7a68c5350de0cfc
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 19 12:42:35 2017 +0100

    i156 Fix the behavior of the CDI bean-name scope for EntityManagerFactory retrieval when asking for the default EMF

    We used to have an annotated @PersistenceUnit field, but:

    1. This is not consistent with the behavior when asking for a specific
       factory, which requires an explicit configuration of entity manager
       factories.
    2. More importantly, this makes injection fail where there are multiple
       persistence units... Which defeats the purpose of this scope.

    Also, we now test the multiple entity manager factories case separately,
    so that we can notice regressions on the single entity manager factory case.

commit f23cdf607a8a3b725b5a9eaf3d229a043fd84c68
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Thu Jan 19 10:35:23 2017 +0100

    Remove duplicated copies of keepTestAlive and create a test util

commit 5bcde50c49a44012f25dcb68efff9e55515cc30c
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 17:36:30 2017 +0100

    Fix an incorrect use of @PersistenceContext(name = "h2") instead of @PersistenceContext(unitName = "h2")

commit 8f8892b876dccff672c957695e74aee6d860e16c
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 17:07:27 2017 +0100

    i156 When not using CDI, allow to reference the entity manager factory by its PU name and make this the default

commit da7bdd74625f2e697a5a5a405fd227f15f61b058
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 15:27:51 2017 +0100

    i156 Use our own session factory registry and avoid using the internal Hibernate ORM registry

commit 1b44d36099f4ee69f95f6f88071af29535c12793
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 12:13:47 2017 +0100

    i156 Introduce a notion of scope for entity manager factory registries

    This will allow to provide multiple options to retrieve the entity
    manager:

     * by session factory name
     * by persistence unit name
     * by bean name
     * etc.

commit deca536dc997838b7f359f58bdf570230b8625b2
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 11:14:27 2017 +0100

    i156 Revert back to using the fully-qualified name as batch artifact name for org.hibernate.search.jsr352.JobContextSetupListener

    We can't use the batch.xml to provide our own artifacts, because in
    standard, JavaSE applications, the user may provide its own batch.xml,
    which will erase our own and make the reference to the job context setup
    listener undefined.

    We work around this situation by using the fully-qualified class name as
    a batch artifact name, and by naming our CDI bean with this exact name.
    It's a bit strange to have a bean name be a fully-qualified class name,
    but it works.

commit 78c5f79c01de1a8bb25d5227bedd9c02b67a9905
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 10:53:39 2017 +0100

    i156 Move JobContextSetupListener to a public package

    ... since it's supposed to be extended.

commit b2f09f635ab00cb07c9b3feb7d1cf33888aac8bd
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 10:49:11 2017 +0100

    i156 Pass the EntityManagerFactory to the job through a string parameter

     * By default, we use the Hibernate registry to retrieve the entity
       manager factory, using the entityManagerFactoryReference as its name
     * When using CDI, the behavior is overridden. Either there is no
       entityManagerFactoryReference parameter and then we use the default entity
       manager factory (if any), or there is one and then we retrieve the bean
       with this name (caution, it's different from the persistence unit name!)

commit bb23592e2712e05427b019d7194ebed22e3678e9
Author: Yoann Rodière <yoann@hibernate.org>
Date:   Wed Jan 18 10:46:57 2017 +0100

    i86 Use fully-qualified class names for batch artifact references

    This way we can avoid making dependency injection mandatory.
HSEARCH-2594 Remove the JSR-352 performance tests
We'll add tests to the ORM perf tests later.
HSEARCH-2594 Upgrade the JSR-352 integration to HSearch 5.9
Also includes:

	HSEARCH-2898 Expose a JBeret JobXmlResolver that implements getJobXmlNames

	... so that it works with WildFly 11.

Also includes:

	HSEARCH-2620 Move JBeret specific file

	The file:

	    hibernate-search-jsr352-core/src/main/resources/META-INF/services/org.jberet.spi.JobXmlResolver

	should move to the `hibernate-search-jsr352-jberet` module, because it
	is an implementation-specific file. This file is required by WildFly
	because our JSR-352 job is installed via a module, and it not part of a
	deployment. It allows this job to be used in WildFly without asking the
	user to rebuild or modify an existing JAR.

Also includes:

	HSEARCH-2689 Do not use the Session in LuceneDocProducer

	Passing information from one batch job component to another seems more
	and more likely to be painful, so I'd rather do it only when strictly
	necessary. On top of that, the current use of the session context is
	dubious at best, since we put two different types of objects in
	stepContext.getTransientUserData (StepProgress and
	PartitionContextData). I am trying to untangle this.

	Besides, we should not need the session in LuceneDocProducer, as entity
	loading is supposed to be the EntityReader's responsibility.

	This change required to:

	 * get the SearchIntegrator from something else than the session
	(namely, from the EntityManagerFactory).
	 * not use HibernateSessionLoadingInitializer, which requires a specific
	session, but rather the default initializer.

	Luckily, both changes made sense regardless of the purpose of this
	commit: that's probably how the doc producer should have been in the
	first place.

Also includes:

	HSEARCH-2898 Update JSR-352 tests to target WildFly 11
HSEARCH-2594 Merge JSR-352 modules project with the main one
Also, add an alias for the JSR-352 integration WildFly module
HSEARCH-2619 MassIndexingJob: Rename the job to something more specif…
…ic to Hibernate Search

Also includes:

	HSEARCH-2708 JSR-352: Change the mass indexing job ID following the recent job renaming
yrodiere added 26 commits Sep 22, 2017
HSEARCH-2911 Fix scroll mode in JSR-352 PartitionMapper
We need ScrollMode.SCROLL_* because we don't just iterate over each and
every result, essentially we peek over each n-th element and thus
advance by blocks of n elements.
HSEARCH-2911 Avoid flushes and locks in JSR-352 mass indexer when rea…
…ding data

We do the same in the non-JSR-352 MassIndexer.
HSEARCH-2911 Use one session/scroll per chunk in JSR-352 mass indexer
The JSR-352 spec clearly states that one (JTA) transaction is created
per chunk, and this transaction is committed just after the chunk.
For that reason, re-using a scroll outside a chunk's boundaries will
lead to errors with some databases (PostgreSQL for example), because the
scroll is cleared on the server side when the transaction is committed.
Strangely, H2 seems to be supporting this just fine, but it's not
something we can rely on.

As a result, I had to adjust the way custom HQL queries are handled. In
that mode, we used to run a single query for the whole mass indexing,
but this is no longer possible since we have to split the processing in
as many transactions as there are chunks.
I ended up storing the index of the last result processed in each chunk,
and using that as an offset for the query for the next chunk. This is
admittedly not a perfect solution, but I can't see a better one for now.
HSEARCH-2914 Move the default JSR-352 mass indexer parameter values t…
…o constants

This way, we only define them in one place (and in the documentation,
obviously), instead of defining them in multiple places the job's
XML file *and* in comments in the parameter builder.
HSEARCH-2914 Make checkpoint interval validation/defaults smarter
1. A value equal to rowsPerPartition does make sense.
2. The parameter builder should take default values into account when
validating (as the JobContextSetupListener already does).
3. checkpointInterval should default to the value of rowsPerPartition
when it is smaller than the static default (currently 200)
HSEARCH-2609 Add periodic session clearing during chunks in JSR-352 m…
…ass indexer

Since we flush to the index at the end of each chunk, and an index flush
requires to hold a global lock on the index writer, it is not a sensible
strategy to just use small chunks.

Introducing periodic session clears during the processing of a chunk
will allow to raise the checkpoint interval without running out of memory,
and thus to flush to the index less often. This gives a huge
performance boost (indexing time divided by two on the ORM mass indexing
performance test with PostgreSQL on my machine).
HSEARCH-2917 Actually execute purges in JSR-352 mass indexing
They used to be skipped because the transaction context was never
closed.
HSEARCH-2609 Update default JSR-352 mass indexer parameter values fol…
…lowing performance tests

Turns out having a high chunk size provides a huge performance boost,
provided we also have a low session clear interval to keep memory
consumption low.

On my machine, running the ORM performance test against a PostgreSQL
database with the FileSystemJsr352MassIndexerTestScenario, the indexing
time dropped from 3min20s to 1min20s, becoming comparable to that of the
older mass indexer, so it's quite encouraging.
HSEARCH-2900 Move JSR-352 integration tests to a separate module
Hopefully this will shield us from cyclic dependency issues in the
future, like those we had with the Elasticsearch integration tests.
HSEARCH-2900 Move Elasticsearch plugin configuration to the parent pom
So that any module can use features such as purges in integration tests.
HSEARCH-2925 JSR-352: Use "sessionClearInterval" rather than "checkpo…
…intInterval" as a default for "entityFetchSize"

@yrodiere yrodiere force-pushed the yrodiere:HSEARCH-2594 branch from 4a6483b to bf3c1ad Oct 25, 2017

@Sanne

This comment has been minimized.

Copy link
Member

commented Oct 25, 2017

It's merged! Awesome job everyone 💯

@Sanne Sanne closed this Oct 25, 2017

@yrodiere yrodiere deleted the yrodiere:HSEARCH-2594 branch Jan 12, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.