-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HSEARCH-679 bis #74
HSEARCH-679 bis #74
Conversation
…n the root indexed entity
…re on unsafe cases
…h classbridges if they are deep enough
looking into it |
done |
Just to answer @Sanne
I just proposed reflectionManager, because it has the required method of comparing a XClass against a Class. My point was mainly about the readability. |
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). Also, a binary file (image) was removed. commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). Also, a binary file (image) was removed. commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). Also, a binary file (image) was removed. commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 hibernate#132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 hibernate#122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide hibernate#130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 hibernate#137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 hibernate#135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 hibernate#130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 hibernate#130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 hibernate#113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 hibernate#133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 hibernate#128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 hibernate#96 enable test RestartChunkIT - hibernate#131 batch status test - hibernate#123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 hibernate#131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 hibernate#133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 hibernate#133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 hibernate#133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 hibernate#133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 hibernate#133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 hibernate#122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 hibernate#127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 hibernate#127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 hibernate#125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 hibernate#125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 hibernate#119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 hibernate#125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 hibernate#125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 hibernate#126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 hibernate#96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 hibernate#125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 hibernate#122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 hibernate#114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 hibernate#125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 hibernate#125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 hibernate#125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 hibernate#121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 hibernate#125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 hibernate#121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 hibernate#122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 hibernate#109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 hibernate#122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 hibernate#116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 hibernate#124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 hibernate#120 hibernate#124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 hibernate#120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 hibernate#108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 hibernate#120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 hibernate#120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 hibernate#109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 hibernate#123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 hibernate#123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 hibernate#124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 hibernate#89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 hibernate#123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 hibernate#119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 hibernate#117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 hibernate#118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 hibernate#118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 hibernate#118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 hibernate#115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 hibernate#115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 hibernate#115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 hibernate#115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 hibernate#108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 hibernate#108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 hibernate#108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 hibernate#107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 hibernate#107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 hibernate#115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 hibernate#109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 hibernate#117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 hibernate#116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 hibernate#116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 hibernate#116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 hibernate#112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 hibernate#112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 hibernate#112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 hibernate#115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 hibernate#106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 hibernate#106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 hibernate#106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 hibernate#106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 hibernate#106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 hibernate#106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 hibernate#106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 hibernate#89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 hibernate#106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 hibernate#89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 hibernate#84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 hibernate#84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 hibernate#84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 hibernate#104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 hibernate#104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 hibernate#105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 hibernate#102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 hibernate#102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 hibernate#96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 hibernate#96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 hibernate#86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 hibernate#89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 hibernate#97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 hibernate#89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 hibernate#100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 hibernate#100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 hibernate#100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 hibernate#100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 hibernate#92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 hibernate#89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 hibernate#82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 hibernate#92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 hibernate#99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 hibernate#92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 hibernate#92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 hibernate#99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 hibernate#98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 hibernate#85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 hibernate#81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 hibernate#90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 hibernate#87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 hibernate#87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 hibernate#94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 hibernate#97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 hibernate#97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 hibernate#97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 hibernate#78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 691bced to 4ee5027 (see git log below). Also, a binary file (image) was removed. commit 4ee5027 Author: Mincong HUANG <mincong.h@gmail.com> Date: Wed Aug 17 08:11:36 2016 +0200 #132 add module "modules" v1 commit 0da6342 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 23:37:03 2016 +0200 #122 update doc - delete index-optimization doc - rename massindexer352 to user-guide - add developer-guide #130 commit 694f855 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 14:33:54 2016 +0200 #137 move the BM script to annotations commit 8b446c0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Mon Aug 15 13:59:13 2016 +0200 #135 Use more specific names for chunk classes commit 7c5e4f1 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 19:38:41 2016 +0200 #130 prove the indexed results commit ed7d35d Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 16:11:13 2016 +0200 #130 performance test commit 7eeefb0 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sun Aug 14 11:11:45 2016 +0200 #113 add transaction timeout to avoid em warning could not log warnings: java.sql.SQLException: IJ031070: Transaction cannot proceed: STATUS_COMMITTED commit d380e1e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 16:41:36 2016 +0200 #133 use purgeAll from FullTextEntityManager commit 106071e Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 14:20:23 2016 +0200 #128 enhance value assertion and util in test commit a709c10 Author: Mincong HUANG <mincong.h@gmail.com> Date: Sat Aug 13 12:55:43 2016 +0200 #96 enable test RestartChunkIT - #131 batch status test - #123 use byteman to stop the job commit 6848144 Author: Mincong HUANG <mincong.h@gmail.com> Date: Fri Aug 12 22:16:24 2016 +0200 #131 enable batch status test commit 91b52c3 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 23:23:55 2016 +0200 #133 clarification for emf and jobOperator under JavaSE commit 3495eaa Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:52:29 2016 +0200 #133 merge mass indexer interface and impl commit e76bd5e Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:34:18 2016 +0200 #133 rename analyzer and collector commit 72a6211 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 19:06:38 2016 +0200 #133 update comments in PartitionAnalyzer commit 6f55657 Author: Mincong HUANG <mincong.h@gmail.com> Date: Thu Aug 11 18:41:37 2016 +0200 #133 delete unused comparators commit f57d08e Author: mincong <mincong.h@gmail.com> Date: Thu Aug 11 00:49:30 2016 +0200 #122 add persistence configuration commit 76fdeac Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 08:55:19 2016 +0200 #127 prove the existence of root entities commit 7ba5450 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 9 07:48:23 2016 +0200 #127 use mock & avoid add getters / setters for tests commit 0f42691 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 21:11:34 2016 +0200 #125 attribute naming and position enhancement - use full upper case for static final attributes - rebase the order of different attributes in class scope : static final go first; final go second; annotations go third; classic attributes go last. commit f091d19 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 8 20:24:24 2016 +0200 #125 extends abstract class to decrease code complexity commit 88e89f4 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:51:09 2016 +0200 #119 unwrap session from emf directly commit 251ff72 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:43:16 2016 +0200 #125 separate obtention of ID name in an util class commit 5777c32 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 15:25:22 2016 +0200 #125 update javadoc and clean up code commit 967db61 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 14:54:49 2016 +0200 #126 fix boundary setting error commit 750b7f6 Author: mincong <mincong.h@gmail.com> Date: Sun Aug 7 10:52:18 2016 +0200 #96 enable mass indexer under Java SE commit 6a85592 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:50:25 2016 +0200 #125 migrate README content -> http://mincong-h.github.io/gsoc-hsearch commit 35ff8f6 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 18:10:11 2016 +0200 #21 rebuild the indexing monitor commit fc70ac8 Author: mincong <mincong.h@gmail.com> Date: Sat Aug 6 10:42:41 2016 +0200 #122 enhance documentation - introduction - example usage - different configurations commit 2bf1a50 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:59:01 2016 +0200 #114 delete utility class referenced from other project commit 896794d Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 22:09:23 2016 +0200 #55 add unit test for ItemReader commit 7e36592 Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:58:36 2016 +0200 #125 rename attribute partitionCapacity -> rowsPerPartition commit 4e0cb5a Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:30:07 2016 +0200 #74 accept single class type as entry for root entity(s) selection commit 35c2c1f Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:20:43 2016 +0200 #125 fix typo commit 66316bb Author: mincong <mincong.h@gmail.com> Date: Fri Aug 5 20:07:52 2016 +0200 #125 add javadoc & license for PartitionMapperTest commit dea58bd Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:45:05 2016 +0200 #121 mock for PartitionMapper commit e29fa33 Author: mincong <mincong.h@gmail.com> Date: Thu Aug 4 23:43:53 2016 +0200 #125 add toString to test entity types commit db2d444 Author: mincong <mincong.h@gmail.com> Date: Tue Aug 2 23:50:35 2016 +0200 #121 mock for PartitionMapper not finished commit 2c88280 Author: mincong <mincong.h@gmail.com> Date: Mon Aug 1 16:47:42 2016 +0200 #122 add documentation commit 296f621 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 12:38:09 2016 +0200 #109 prove the entire dataset is indexed using all context query commit 949c2c5 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 10:00:54 2016 +0200 #122 create document module commit 61619e4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 31 00:49:50 2016 +0200 #116 revert changes : use partitionCapacity again commit 1f304b0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 16:30:42 2016 +0200 #124 enhance code commit de7a96d Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 12:25:11 2016 +0200 #120 #124 enhance the boundary logic Currently, the boundary logic is not very clear. There're too many classes participating to the construction, which leads to a lack of abstraction. So here're some enhancements : - move embedded class PartitionMapper._Unit to PartitionUnit - embed PartitionBoundary class to PartitionUnit - rename partitionIndex to partitionID commit 0be59ab Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 09:22:44 2016 +0200 #120 enhance the partition boundary concept - cancel the "unique partition" concept, one entity type should have at least 2 partitions - change the closed interval to Right-Closed interval commit c64f4ec Author: mincong <mincong.h@gmail.com> Date: Sat Jul 30 00:05:35 2016 +0200 #108 several enhancements for partitioning - add an additional partition at the end of each entity type to handle new inserted entities - use N times more partitions than the user-defined partition (initial partition) in order to balance the thread computation charge commit bb69ee0 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 29 00:19:46 2016 +0200 #120 change R-closed interval to full closed interval commit ad7b077 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 23:56:29 2016 +0200 #120 replace firstID and lastID by divider Different dividers provide a partition boundary mechanism. commit a7ab1e3 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 28 00:23:21 2016 +0200 #109 prove the entire dataset is indexed in RestartIT commit 701555e Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:50:55 2016 +0200 #123 adapt MassIndexerIT to RestartIT so that the restartability can be proved inside this class. commit 818caa3 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:49:33 2016 +0200 #123 fixe typo error for Byteman script in maven pom.xml commit 5324146 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:48:09 2016 +0200 #124 delete unused method in Helper class commit ffb8edd Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 23:47:21 2016 +0200 #89 fix typo error for "family" commit 1aefe2a Author: mincong <mincong.h@gmail.com> Date: Wed Jul 27 19:40:56 2016 +0200 #123 change Byteman roles and enable job restart in IT commit ec45c74 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 23:51:34 2016 +0200 #119 avoid using entity manager SessionFactory can be unwrapped directly from EntityManagerFactory. Once factory is unwrapped, it can be used to create a session instance. Of course, Hibernate session need to be closed explicitly at the end of its usage. commit deb2685 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 26 20:14:24 2016 +0200 #117 add Java agent Byteman for arquillian integration test commit 84a8909 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 21:42:39 2016 +0200 #118 add "try catch finally" for all concerned classes commit cbf94fa Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 20:58:55 2016 +0200 #118 add "try catch finally" structure for PartitionMapper commit 3462cb6 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 15:46:35 2016 +0200 #118 add try finally in mapper commit 289cc96 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 14:21:44 2016 +0200 #115 add cacheable and fetchSize as batch property commit d552c6b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 12:06:40 2016 +0200 #115 enable the entity MyDate.class in MassIndexerIT commit 2556c8d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 11:58:26 2016 +0200 #115 cleanup partition mapper, delegate map to job context data commit 925963d Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:43:50 2016 +0200 #115 add toString() to entity type Person.class commit 12f5a8f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 10:42:38 2016 +0200 #108 ID should be ordered before scrolling commit 654aadf Author: mincong <mincong.h@gmail.com> Date: Mon Jul 25 03:23:03 2016 +0200 #108 use range approach to define partition v1 available for String ID (still have problem for criteria expression in item reader) commit 6063c21 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 21:15:54 2016 +0200 #108 use range approach to define partition v1 available for integer ID commit 13c8bd4 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 14:36:43 2016 +0200 #107 add hashcode for entity and enable SQL commit 981a371 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 12:30:51 2016 +0200 #107 use the remainder approach for item reader Currently only available for integer ID. commit f30a4ed Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:44:20 2016 +0200 #115 format XML file to adapt Hibernate code style commit 5d11e1c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 10:38:03 2016 +0200 #109 delete byteman config For instance, the byteman does not work properly, so everything related to byteman is deleted for instance. The previous configuration had been stored at branch issue-117. I need more time to figure out how it works, with or without Emmanuel. commit ceacc8c Author: mincong <mincong.h@gmail.com> Date: Sun Jul 24 01:07:50 2016 +0200 #117 add byteman to Java EE tests commit b020c26 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 23 09:47:47 2016 +0200 #116 update javadoc and cleanup code commit dc45ece Author: mincong <mincong.h@gmail.com> Date: Fri Jul 22 23:51:53 2016 +0200 #116 enhancement for partitioning code cleanup for better understanding commit 3de0007 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 21 23:50:10 2016 +0200 #116 define partitions instead of partitionCapacity in the job properties commit 19c1a1c Author: mincong <mincong.h@gmail.com> Date: Wed Jul 20 18:35:39 2016 +0200 #112 revert useless clean up commit 306c922 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:59:15 2016 +0200 #112 clear em before it is closed commit 2b3831e Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 23:44:37 2016 +0200 #112 inject emf instead of em to control the lifecycle commit 4b7b77a Author: mincong <mincong.h@gmail.com> Date: Tue Jul 19 20:26:36 2016 +0200 #115 add class description for JobContextSetupListener commit ef9e247 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 13:00:22 2016 +0200 #106 enhance code & doc for internal classes commit 3eba510 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:56:53 2016 +0200 #106 clean up for class MassIndexerImpl commit 5b86bcd Author: mincong <mincong.h@gmail.com> Date: Mon Jul 18 11:45:21 2016 +0200 #106 add javadoc for interface MassIndexer commit ff60058 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:11:16 2016 +0200 #106 update image URL commit d3b4a0a Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 23:06:32 2016 +0200 #106 update description, add workflow diagram commit fbf58e3 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 22:31:40 2016 +0200 #106 delete PartitionReducer (we don't need it) commit 3afd079 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 18:19:12 2016 +0200 #25 add the afterChunk enhancement commit 960a8ad Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:52:02 2016 +0200 #24 rename package: purge -> beforeChunk commit 02cb248 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 17:32:10 2016 +0200 #24 separate the beforeChunk enhancements from the chunk step commit b7765a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 15:56:26 2016 +0200 #106 cleanup maven commit c74e783 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 13:10:14 2016 +0200 #89 add restart tests in MassIndexerIT under JavaEE commit f9eef03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 10:33:20 2016 +0200 #106 clean up MassIndexerIT in Java EE commit 3400718 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 17 00:25:15 2016 +0200 #89 add entity type MyDate.class & test in IT (JEE) commit 98e6e42 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 16 12:44:17 2016 +0200 #24 purge at start enhancement commit d49678a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 22:58:53 2016 +0200 #24 optimize after purge (applied on ItemReader#open(...)) + code cleanup Enhancement before lucene work execution, according to https://github.com/hibernate/hibernate-search/blob/db0bd844315a26b0dcac1a032f7714c7cb1963fd/engine/src/main/java/org/hibernate/search/backend/impl/batch/DefaultBatchBackend.java#L80-L86 commit b93b2b1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 21:59:38 2016 +0200 #84 change step context usage & cleanup job params - change stepContext's persistent data methods to transient data, so that the data can be shared by reference in all classes in the same partition. Persistent data method will be used only on open (start) and close (end) to save as checkpoint. - params "arrayCapacity" was used for bulk of IDs previously, but it isn't used anymore - update related tests commit b2e8377 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:32:49 2016 +0200 #84 rename params inside the step context "partitionNumber" -> "scrollOffset" "partitionSize" -> "scrollInterval" commit 905b8d7 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 18:13:53 2016 +0200 #84 cleanup job params commit 9940adc Author: mincong <mincong.h@gmail.com> Date: Fri Jul 15 00:17:21 2016 +0200 #104 partitioning for multiple entity types commit 21d9195 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 18:58:51 2016 +0200 #104 multiple partitions for one entity type using scroll mechanism commit e895220 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 12:56:59 2016 +0200 #105 change author info commit 36ff3c1 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:37:31 2016 +0200 #102 increase target entities' number from 100 to 1000 commit 8bfe365 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 11:29:03 2016 +0200 #67 increase TX in the customized config standalone-full-testqueues.xml commit 3be9290 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 14 10:28:56 2016 +0200 #102 Move creation of Lucene doc to ItemProcessor commit 7154abb Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 22:31:14 2016 +0200 #96 skip entity manager handling for Java SE + code clean up rename PartitionedContextData to StepContextData so that it is easier for memorising. The related part of the README file has been updated too. commit 2705279 Author: mincong <mincong.h@gmail.com> Date: Wed Jul 13 21:34:23 2016 +0200 #96 skip tests in Java SE commit de6338e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 21:45:27 2016 +0200 #86 avoid CDI injection before the job start commit de9cb46 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:11:11 2016 +0200 #89 change Person's ID into more complex string expression commit 020eb91 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 12:07:41 2016 +0200 #97 add order in item reader commit eb3eb0f Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 11:39:04 2016 +0200 #89 add Person.class into Java SE integration test commit 43e8557 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 09:52:57 2016 +0200 #100 increase README readability commit fcc55fe Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:41:02 2016 +0200 #100 update README composing commit 74d909b Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:38:20 2016 +0200 #100 update README composing commit 1bd3e5e Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 01:29:30 2016 +0200 #100 explain project's mechanism in README commit 28942f9 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 11 00:10:15 2016 +0200 #92 rename BatchContextSetupListener to JobContextSetupListener commit 09b7d03 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 23:54:16 2016 +0200 #89 add entity type Person.class commit b14d304 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 19:29:52 2016 +0200 #82 move Java SE based test to core module commit 22d67b6 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 16:21:51 2016 +0200 #92 simply count process inside the transient / persistent data commit bccdb05 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 07:31:09 2016 +0200 #99 delete ContextSetupListener commit 864feeb Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 01:17:50 2016 +0200 #92 cleanup StepContext EntityType / EntityClazz will only be stored in JobContext using BatchContextData commit 4711d52 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 10 00:49:56 2016 +0200 #92 exclude entity-sharding-strategy from context data The stuff is already done by Hibernate Search's class EntityIndexBinding. No need to do it again, just use the existing one. commit 9f857d3 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 18:25:01 2016 +0200 #99 Disable AbstractItemReadListener commit c1512d8 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 14:17:18 2016 +0200 #98 use EntityName & EntityClazz commit 3cda1ae Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 13:47:41 2016 +0200 #85 get ID using EntityIndexBinding commit a3c4bed Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 11:03:05 2016 +0200 #81 clean up code in mass indexer commit 08c06c0 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 9 10:27:38 2016 +0200 #90 enable customizable checkpoint algorithm commit fb9c71e Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:12:51 2016 +0200 #61 change project version to 5.5.4.Final commit 37401bd Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 23:03:09 2016 +0200 #87 add License LGPL commit 19c0971 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 8 22:34:46 2016 +0200 #87 Apply Hibernate code style commit 856c89a Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:42:24 2016 +0200 #94 create dedicated sub-packages under internal.steps one package per step commit 7ec9e14 Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 23:05:55 2016 +0200 #97 finish chunk step redesign & tests associated commit 21fe2ea Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 21:10:25 2016 +0200 #97 add stateless session (still failed) code updated following article "Kohei Nozaki • Chunk-oriented Processing with Hibernate's StatelessSession" http://www.nailedtothex.org/roller/kyle/entry/articles-jbatch-hibernate commit d31442e Author: mincong <mincong.h@gmail.com> Date: Thu Jul 7 02:07:44 2016 +0200 #97 redesign the step execution "chunk" commit 691bced Author: mincong <mincong.h@gmail.com> Date: Wed Jul 6 18:20:18 2016 +0200 #78 embed final flush into ItemWriter So the FlushBatchJobListener can be deleted now. And the code complexity should be decrease thanks to this enhancement.
resping of HSEARCH-679,
including fixes for all previous comments.
thanks a lot!
Hardy, re "avoid annotation scanning on Object", I've proposed an alternative. you're right my code wasn't very readable, on the other hand I don't see why I need a reflectionManager to figure out if a class is java.lang.Object.
I believe the linkedList is better here than the arrayList, and as a plus I can create it efficiently in the proper (inverted) order, so iteration is cleaner as well.