-
Notifications
You must be signed in to change notification settings - Fork 243
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HSEARCH-667, HSEARCH-686, HSEARCH-699 #50
Closed
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commits still include some changes related to early faceting experiments with bobo browse. These changes will be removed later
…in org.hibernate.serch.query.facet Implementation of faceting is done via a custom Collector. Refactoring QueryHits Removing the last obsolete code from the bobo browse experiments
…s to access the session in tearDown and avoids exceptions in tearDown to swallow original exceptions. Also removing the fail() in handlingUnclosedResources()
Chatted with Hardy, this pull request will be split in two and we will continue to bake facet on a personal branch |
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Feb 20, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Feb 20, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Feb 20, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Feb 20, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
May 17, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Jun 19, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Oct 16, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Oct 17, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Oct 17, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Oct 25, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
yrodiere
referenced
this pull request
in yrodiere/hibernate-search
Oct 25, 2017
This commit has been extracted from https://github.com/mincong-h/gsoc-hsearch/tree/master/ It is the result of squashing commits 135a190 to f9d1716 (see git log below). commit f9d1716 Author: mincong <mincong.h@gmail.com> Date: Tue Jul 5 02:14:30 2016 +0200 #40 add IT for checkpoint: RestartChunkIT commit 5e5e177 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 16:38:27 2016 +0200 hibernate#81 correct spelling error commit d442d06 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:27:40 2016 +0200 #74 change unit test MassIndexerTest commit 236e1d8 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:21:42 2016 +0200 #74 add method `MassIndexer#addRootEntities(Class<?>... rootEntities)` commit 58f8f32 Author: mincong <mincong.h@gmail.com> Date: Mon Jul 4 09:07:54 2016 +0200 hibernate#81 clean up gitignore, parent pom, mysql module in WF commit 9d569b1 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:47:24 2016 +0200 hibernate#81 clean up IT in Java SE - properties are not used locally commit e9e1574 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:43:44 2016 +0200 hibernate#81 clean up IT in Java EE In pom.xml - Properties are not used. Properties should be provided by parent module or core module. - JBeret SE dependencies are useless. - We don't use absolute path here - There's not webapp to package. Arquillian create war archive with its own dependencies. - Exclude surefire because there's no unit test here. In persistence.xml - It should be placed in the META-INF folder as the standard. In MassIndexerIT - Change persistence.xml's location commit 5027a33 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 23:08:19 2016 +0200 hibernate#81 clean up core/pom.xml - The exclusion of dom4j is not necessary because hibernate search arm is in scope provide. So there's no conflict. - Servlet was used for the web app, but the project becomes a framework now. There's no servlet any more. - Same principal for JSTL. - Test resources were set for integration test. But now, ITs have been moved to independent modules. - Plugin for dependency path is not used since there isn't any absolute path in the project - Plugin for WAR is not used. Same reason as Servlet, not wabapp any more. - Unit test (surefire) does not need any additional requirement for instant - No IT in core module, so no failsafe, no WF unpack, no resources copy commit 32f6b3b Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 20:47:57 2016 +0200 #63 add value assertions in Java SE integration test commit e6a5699 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 16:32:24 2016 +0200 #63 deny JobOperator construction inside the MassIndexerImpl pass it through the setter method. commit 25adb82 Author: mincong <mincong.h@gmail.com> Date: Sun Jul 3 12:32:38 2016 +0200 #63 fix JobOperator instantiation issue In the previous version, the job operator cannot be instantiated and show the below error : java.util.ServiceConfigurationError: javax.batch.operations.JobOperator: Provider org.jberet.operations.JobOperatorImpl could not be instantiated at org.jboss.weld.environment.se.WeldContainer.initialize(WeldContainer.java:136) at org.jboss.weld.environment.se.Weld.initialize(Weld.java:589) at org.jberet.se.SEArtifactFactory.<init>(SEArtifactFactory.java:29) at org.jberet.se.BatchSEEnvironment.getArtifactFactory(BatchSEEnvironment.java:118) at org.jberet.operations.JobOperatorImpl.<init>(JobOperatorImpl.java:93) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at java.lang.Class.newInstance(Class.java:442) at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) at java.util.ServiceLoader$1.next(ServiceLoader.java:480) at javax.batch.runtime.BatchRuntime.getJobOperator(BatchRuntime.java:57) at org.hibernate.search.jsr352.se.MassIndexerIT.setup(MassIndexerIT.java:41) And I found out that if there's only one @test annotation in the integration test, the problem get fixed. I don't know why and it is very weird. commit 5a99766 Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:14:15 2016 +0200 hibernate#80 LuceneWork execution should be asynchronous commit 75bcf3e Author: mincong <mincong.h@gmail.com> Date: Sat Jul 2 19:11:35 2016 +0200 #73 change ref assignment for EntityManager inside the job commit bd0fe3a Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 19:14:12 2016 +0200 #63 replace entity `Address`, `Stock` by `Company` commit 36f8ce1 Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:35:57 2016 +0200 hibernate#76 update README commit 5ac099c Author: mincong <mincong.h@gmail.com> Date: Fri Jul 1 06:31:33 2016 +0200 #69 leave mysql and use only h2 commit cd6289f Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:42:18 2016 +0200 #69 code enhancement commit 18e2965 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:36:49 2016 +0200 #69 change addEntityCount to Synchronized mode commit ad3b5b9 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 23:16:44 2016 +0200 #67 extend timeout value the default timeout in JSR352 is 180 seconds according to v1.0 final release chapter §9.7 Transactionality commit accf7e1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 29 22:20:24 2016 +0200 #69 code enhancement commit 2151742 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 07:51:54 2016 +0200 #63 try to use my own MassIndexer (failed) revert to Gunnar's one in MassIndexerIT.java However, there're still some more dependencies added commit 1fc14a7 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 28 00:01:34 2016 +0200 #63 replace official indexer by my own indexer (Java SE) but this is not the finished version commit 889e782 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 21:29:30 2016 +0200 #68 use embedded H2 database for Java SE testing commit 7e49180 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 12:56:35 2016 +0200 #63 separate core module and integration tests modules and remove useless resources files, e.g. sql & csv commit 51e2636 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 11:45:11 2016 +0200 #63 create core module commit ed557c0 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 27 10:42:29 2016 +0200 #63 configure pom file for Java SE IT module commit 6ddbb7c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 26 23:54:12 2016 +0200 #63 start module hibernate-search-integrationtest-javase commit 931ca09 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 16:02:37 2016 +0200 #69 add batch status check commit 3a89e11 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 11:52:48 2016 +0200 #71 move persistence.xml to src/test/resources commit 64727ba Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:51:02 2016 +0200 #69 write count test commit ab9df58 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:50:16 2016 +0200 #70 IdProducerBatchlet produces less IDs than expected commit 917a011 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:49:17 2016 +0200 #43 add lucene files to .gitignore commit 9d8214c Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:06:38 2016 +0200 #69 read count test commit 7b61a5d Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:05:36 2016 +0200 #69 add BatchTestHelper, copied from javaee7-samples commit 4a3941f Author: mincong <mincong.h@gmail.com> Date: Sat Jun 25 10:04:22 2016 +0200 #49 change logger level commit 47aec79 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:40:32 2016 +0200 #60 some mixed enhancement commit be25a49 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:37:42 2016 +0200 #65 fix entity type usage use IndexingContext instead of batch properties commit 2f40c3c Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:34:56 2016 +0200 #49 change log level commit 1ff59ad Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:30:19 2016 +0200 #46 Rename package into org.hibernate.search.jsr352 commit 254d44f Author: mincong <mincong.h@gmail.com> Date: Thu Jun 23 23:28:45 2016 +0200 #66 use remote mysql server commit 1d4be37 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:12:10 2016 +0200 #60 add JBoss Deployment Structure File for class loading commit 29e5a24 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:11:23 2016 +0200 #60 add shrink-wrap resolver dependency for war archive creation commit 655e1ff Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:09:19 2016 +0200 #60 change WF to debug mode commit 73decd1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:06:36 2016 +0200 #60 fix PU naming error commit 8a0438e Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:05:23 2016 +0200 #60 move src/main/webapp to test/main/webapp commit e9c858c Author: mincong <mincong.h@gmail.com> Date: Wed Jun 22 00:04:18 2016 +0200 #60 remove wrong placed persistence.xml commit cd9aca8 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 21 19:01:26 2016 +0200 #64 disable IT log redirection commit 5426048 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 23:17:32 2016 +0200 #59 remove ds.xml and include mysql jar file commit f8d292c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 22:15:47 2016 +0200 #59 add mysql resources to src/wildflyConfig commit 00d45bc Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 11:17:46 2016 +0200 #58 add datasource for MySQL commit 64a1b5c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 20 09:46:27 2016 +0200 #57 clarify unit test and integration test - rename DeploymentTest.java to DeploymentIT.java - change pom.xml configuration commit 9a95b24 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 20:33:03 2016 +0200 #49 use Logger instead of System.out.println commit 0449388 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 19:38:25 2016 +0200 #28 change README content The previous demos have been deleted. The current README should only focus on 3 points : 1. What's new using JSR352 implementation ? 2. How to run it ? 3. Why we need it ? commit 7c6662c Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 17:09:57 2016 +0200 #50 delete MANIFEST.MF commit 95690a9 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:42:14 2016 +0200 #54 add unit test for mass indexer commit 522f178 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 15:40:46 2016 +0200 #48 separate interface and implementation commit 38c4f54 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 09:09:33 2016 +0200 #52 add deprecated class manually during war creation commit 31610a7 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 19 08:37:59 2016 +0200 #52 delete commented static configuration commit 4884f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 18 21:47:54 2016 +0200 #52 💩 only god knows why it works 💦 💦 💦 - pom.xml - add arquillian protocol support (not sure if necessary) - change WF from embedded to managed - add tag <testResources> to enable arquillian xml file lookup - add bit strings to find the absolute path to dependencies (not sure if necessary) - add detailed configuration to unit test plugin surefire (copied from HSEARCH) - add failsafe (not sure if necessary) - add WF runtime in a better way... (Actually, I've not idea how it works. Maybe it just download a target WF version from maven central and past it to the target output directory) - add resources plugin to transfer the defined WF config from src to test directory - DeploymentTest - disable the recursive package lookup because the deprecated class "org.hibernate.search.store.IndexShardingStrategy" is not found - arquillian.xml - disable the "javaVmArguments", otherwise the server cannot start - wildflyCondif/* - config for WF container commit 675c319 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 16 23:21:50 2016 +0200 #52 add Arquillian feature (failed) - add JUnit as a maven dependency - add arquillian components as maven dependencies (not finished, don't know exactly what to add) - add wildfly-arquillian-container-embedded, but not sure how it works - add class DeploymentTest commit 68082d7 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 21:31:09 2016 +0200 #48 split of public API and internal implementation packages commit d339328 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:48:51 2016 +0200 #37 move test classes from src/main/java to src/test/java - entities - sessions - servlets (should be deleted soon) - META-INF (except batch XML file mass-index.xml) commit a67ff0d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 15 20:07:30 2016 +0200 #45 Fetch WildFly for testing via maven-dependency-plugin There're other minor modifications for fixing warnings during the build. commit 47f2895 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 23:20:33 2016 +0200 #14 add dependency and modify artifact info commit 522f1ac Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:45:03 2016 +0200 #41 Move content of folder jsr352 to the root folder commit 5c46ff1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 22:13:57 2016 +0200 #41 remove jse-chunk and zoo-jpa commit 7fd6cd2 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:27:58 2016 +0200 #35 update project name in pom.xml commit ac276b9 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:26:28 2016 +0200 #42 change Deployment Assembly settings commit a7a3e93 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 14 02:24:05 2016 +0200 #41 add SQL script for importing CSV files commit d6024c6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:49:46 2016 +0200 #41 add SQL script for importing yahoo-finance-rht.csv commit c8f6634 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:44:51 2016 +0200 #41 change yahoo finance stock CSV file for redhat to lowercase commit 7ed4223 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:36:58 2016 +0200 #41 provide MySQL dump script for creating tables (without data) commit 917b64a Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:13:09 2016 +0200 #36 Move "./WebContents" to "./src/main/webapp" commit 81adb41 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 21:10:18 2016 +0200 #14 update pom.xml - add dependencies - rename artifactId (issue #35) - change folder location from ./WebContent to ./src/main/webapp (issue #36) commit b258cf4 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 19:45:57 2016 +0200 #35 rename "us-address" to "jsr352" commit 6f80618 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:46:02 2016 +0200 #28 change diagram for mass-indexer (overview) Prepare for the first tag. This image should have smaller width and all components up to date. commit fa1e12c Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 17:21:27 2016 +0200 #41 update javadoc and enhance code commit 1fb8ae6 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 16:26:46 2016 +0200 #41 update javadoc for chunk-step "produceLuceneDoc" Reader / Processor / Writer commit a8f7aaa Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 15:47:00 2016 +0200 #41 clean code for IdProducerBatchlet commit 9820a46 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 14:32:45 2016 +0200 #34 use IndexingContext for entity row count commit 430ea9e Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:12:27 2016 +0200 #33 change job to use dynamic configuration of partition plan commit e733a7f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 13 01:06:09 2016 +0200 #22 use generic type in BatchItemProcessor commit 0b28f46 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 21:55:24 2016 +0200 #32 add EntityPartitionMapper commit 518b035 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 20:45:06 2016 +0200 #31 get class type Class<?> from String commit b734016 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 16:08:47 2016 +0200 #30 add table `stock`, provided by yahoo finance commit 61b5406 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:13:42 2016 +0200 #24 #25 add <decision> tags in job "mass-index" work flow commit 6967351 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 11 15:08:57 2016 +0200 #28 change diagram for mass indexer (overview) The image provided in the commit is generated by the JBoss tool in Eclipse, so it helps to standardize the progress display commit c65f41d Author: mincong <mincong.h@gmail.com> Date: Wed Jun 8 20:07:03 2016 +0200 #24 change decision id in job xml commit d127e1e Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:42:49 2016 +0200 #28 add diagram for mass indexer (chunk) commit 2f314a1 Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 23:28:31 2016 +0200 #28 resize image commit e43b4ce Author: mincong <mincong.h@gmail.com> Date: Tue Jun 7 22:59:02 2016 +0200 #28 add diagram for mass indexer (overview) commit 36440be Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 21:46:24 2016 +0200 #24 add decider and purger batchlet before the doc production chunk Decider and purger batchlet provide a before-production enhancement. User can now have possibility to purge all index before started. However, this commit provides only the working model, method inside the batchlet is not implemented yet. commit 55f0137 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:52:29 2016 +0200 #26 delete IdReaderBatchlet This class belongs to the previous demo and it was not used since a moment. Delete it to avoid confusion. commit 7d2b5b5 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 19:47:28 2016 +0200 #26 delete previous demo's chunk classes AddressReader + AddressProcessor + AddressWriter + Job XML commit 4474125 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 17:26:53 2016 +0200 #23 replace LinkedList by ConcurrentLinkedQueue for idChunkQueue commit e4c9f04 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 12:27:42 2016 +0200 #22 Use generic type in BatchItemProcessor Now the method buildAddLuceneWorks(List, Class<T>) is generic. This implementation done thanks to Stack Overflow. I've asked a question this morning : How to use generic type in a map? [duplicate] <http://stackoverflow.com/questions/37654228/how-to-use-generic-type-in-a-map> commit 2a67c66 Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:55:21 2016 +0200 #4 performance test for indexing 1M entities 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeCollectorData(): 1000000 works processed (100.0%). 10:51:58,187 INFO [stdout] (Batch Thread - 1) #analyzeStatus(...) called. 10:51:58,187 INFO [stdout] (Batch Thread - 1) #beforePartitionedStepCompletion() called. 10:51:58,188 INFO [stdout] (Batch Thread - 1) #afterPartitionedStepCompletion(...) called. 10:51:58,466 INFO [stdout] (EJB default - 1) 52 rounds, delta T = 64441 ms. commit 9ccde4f Author: mincong <mincong.h@gmail.com> Date: Mon Jun 6 10:45:32 2016 +0200 #21 modify variable naming and javadoc to clarify the analyzer commit 911990f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 21:36:27 2016 +0200 #21 show mass index progress using PartitionAnalyzer commit 7072f6f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:59:08 2016 +0200 #19 decrease item-count to avoid OutOfMemory exception commit 392a40b Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:58:06 2016 +0200 #17 pass IndexShardingStrategy through IndexingContext commit de93ad6 Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 18:56:33 2016 +0200 #21 Collect and analyse items processed from different partitions Previously, items processed in different partitions were shown in separate monitor. Now, by using the StepContext + PartitionCollector + PartitionAnalyzer, these data are unionned together. So, we can see the total progress of this step. However, the monitor is not integrated yet. commit 6d2ae3f Author: mincong <mincong.h@gmail.com> Date: Sun Jun 5 17:59:34 2016 +0200 #20 disable the cache in a session commit 9b8faa8 Author: mincong <mincong.h@gmail.com> Date: Sat Jun 4 19:07:22 2016 +0200 #13 add AddLuceneWork execution in job "mass-index" The BatchItemWriter can now execute AddLuceneWork and a simple monitor has been added to monitoring the index progress. The BatchItemProcessor had a problem about the IndexShardingStrategy (issue #17). Now it is fixed using a customized context class for context-value-transfer. commit 81891b9 Author: mincong <mincong.h@gmail.com> Date: Thu Jun 2 20:54:21 2016 +0200 #13 change BatchItemWriter to do lucene work. The processor build AddLuceneWork using the entities obtained from JPA entity manager. The code has been separated into logical modules and different functions. commit 6d70db1 Author: mincong <mincong.h@gmail.com> Date: Wed Jun 1 23:33:05 2016 +0200 #15 fix unaware how to convert value commit c6e8fe1 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 23:48:06 2016 +0200 #13 add javadoc commit 4e7fcdc Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:44 2016 +0200 #13 add documentation for reader commit 0ec55aa Author: mincong <mincong.h@gmail.com> Date: Mon May 30 21:50:12 2016 +0200 #13 change processor's role and add documentation commit e636368 Author: mincong <mincong.h@gmail.com> Date: Mon May 30 13:43:45 2016 +0200 #13 mass indexer using JSR 352 - id production commit 382fe8c Author: mincong <mincong.h@gmail.com> Date: Sun May 29 20:38:25 2016 +0200 #7 Parallel processing for IdReaderBatchlet commit 941c3b0 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 22:51:08 2016 +0200 #10 Use customized context to pass properties between steps commit c361d22 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 21:25:37 2016 +0200 #8 change data structure from ArrayList to Array (4.0% faster) commit 5739a13 Author: mincong <mincong.h@gmail.com> Date: Sat May 28 12:00:02 2016 +0200 #9 Set batchlet properties via job xml Now the batchlet properties are defined through the Job XML, with Job XML substitution support described in §8.8 of JSR 352 v1.0 final release. commit 14a638a Author: mincong <mincong.h@gmail.com> Date: Fri May 27 18:29:37 2016 +0200 load ids v2 Now the class IdProducerBatchlet can load all entities' ids and print to terminal. commit 25c2014 Author: mincong <mincong.h@gmail.com> Date: Fri May 27 00:38:56 2016 +0200 add id producer batchlet (not finished) This batchlet is similar to org.hibernate.search.batchindexing.impl.IdentifierProducer commit eadadae Author: mincong <mincong.h@gmail.com> Date: Sun May 22 22:36:51 2016 +0200 execute LuceneWork in AddressProcessor commit 53881cd Author: mincong <mincong.h@gmail.com> Date: Sun May 22 20:15:21 2016 +0200 #1 move operations related to session from processor to reader commit aaf0441 Author: mincong <mincong.h@gmail.com> Date: Sat May 21 16:28:22 2016 +0200 change batch job from a print demo to lucene demo commit b72158b Author: mincong <mincong.h@gmail.com> Date: Mon May 16 00:40:24 2016 +0200 add demo batch job `print-addresses-job` commit 539c244 Author: mincong <mincong.h@gmail.com> Date: Sat May 14 15:21:23 2016 +0200 building demo jse-chunk commit a5db451 Author: mincong <mincong.h@gmail.com> Date: Thu May 5 21:50:33 2016 +0200 add different types of query - keyword - fuzzy - wildcard commit dcde70e Author: mincong <mincong.h@gmail.com> Date: Thu May 5 17:48:13 2016 +0200 add basic search feature (keyword query) commit 4a103d1 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 15:45:48 2016 +0200 optimise mass indexer settings https://docs.jboss.org/hibernate/search/5.6/reference/en-US/html_single/#search-batchindex-massindexer commit 2b2ac86 Author: mincong <mincong.h@gmail.com> Date: Sun May 1 14:21:29 2016 +0200 change hibernate search method `index()` into async mode commit 322c58a Author: mincong <mincong.h@gmail.com> Date: Sun May 1 09:50:00 2016 +0200 add Hibernate Search feature into `us-address` commit 05b0ee1 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 30 11:08:25 2016 +0200 add JPA example `us-address` The Twilio/Wigle.net Street Vector data set provides a complete database of US street names and address ranges mapped to zip codes and latitude/longitude ranges, with DTMF key mappings for all street names. This application `us-address` is a Java EE demo for JPA 2.1. It will be extended to Hibernate Search feature later. commit 135a190 Author: mincong <mincong.h@gmail.com> Date: Sat Apr 16 17:03:37 2016 +0200 add example zoo-jpa
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I squashed my commits for faceting into a just a few commits. It is still work in progress, but since this request also contains fixes for HSEARCH-686 and HSEARCH-699 I thought it might be a good time to get the current state into master.