Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New reindex spring batch operation #2845

Merged
merged 50 commits into from
Jul 29, 2021
Merged

New reindex spring batch operation #2845

merged 50 commits into from
Jul 29, 2021

Conversation

fil512
Copy link
Collaborator

@fil512 fil512 commented Jul 29, 2021

No description provided.

fil512 added 30 commits July 24, 2021 11:11
@fil512
first sweep to add new $reindex job
597f66d
@fil512
first cut done. next up integration test.
530f7b2
@fil512
start roughing out reindex everything
f12733b
@fil512
Failing test
d754ee4
@fil512
test passes
6af7d68
@fil512
add BundleUtil helper method
c038d5c
@fil512
rough in processEverything
6e901c2
@fil512
rough in processEverything
096c030
@fil512
fix mock test
5b8069f
@fil512
merge master
7a45fdd
@fil512
merge master
a8bae39
@fil512
Make BaseJpaTest.mySrd lenient.
b0a2be0
@fil512
fix test
c166181
@fil512
reindex everything works with FIXMEs
12d0b87
@fil512
reindex everything more tests
5846fa5
@fil512
moar tests
c2e849f
@fil512
moar tests
c04ec73
@fil512
fix test
61158cd
@fil512
fix test
3c42642
@fil512
fix regression and fix test
86f0117
@fil512
fix test cycle
db2dcff
@fil512
fixme
7519ce0
@fil512
fix test
bcda1b6
@fil512
test provider layer
141e2f1
@fil512
reindex everything partition aware
8429c36
@fil512
yay last fixme!
de33198
@fil512
Merge remote-tracking branch 'origin/master' into ks-20210724-reindex…
c97d21c 
…-job
@fil512
run reindex in partitioned threads
bb77fbb
@fil512
add fixmes for areas that require extra testing
3bf5d4c
@fil512
added transaction and log message
b3cfc3a
@lgtm-com
Copy link

lgtm-com bot commented Jul 29, 2021

This pull request introduces 1 alert and fixes 1 when merging 774e0e2 into 1b9761c - view on LGTM.com

new alerts:

  • 1 for Boxed variable is never null

fixed alerts:

  • 1 for Spurious Javadoc @param tags

@lgtm-com
Copy link

lgtm-com bot commented Jul 29, 2021

This pull request introduces 1 alert and fixes 1 when merging ff1516b into 1b9761c - view on LGTM.com

new alerts:

  • 1 for Boxed variable is never null

fixed alerts:

  • 1 for Spurious Javadoc @param tags

@lgtm-com
Copy link

lgtm-com bot commented Jul 29, 2021

This pull request introduces 1 alert and fixes 1 when merging d0da752 into 1b9761c - view on LGTM.com

new alerts:

  • 1 for Boxed variable is never null

fixed alerts:

  • 1 for Spurious Javadoc @param tags

@codecov
Copy link

codecov bot commented Jul 29, 2021 •
edited
Loading

Codecov Report

Merging #2845 (1bdd16c) into master (1b9761c) will increase coverage by 0.00%.
The diff coverage is 87.84%.

Impacted file tree graph

@@            Coverage Diff             @@
##             master    #2845    +/-   ##
==========================================
  Coverage     82.58%   82.58%            
- Complexity    19608    19687    +79     
==========================================
  Files          1311     1323    +12     
  Lines         69887    70100   +213     
  Branches      10714    10728    +14     
==========================================
+ Hits          57713    57889   +176     
- Misses         8069     8093    +24     
- Partials       4105     4118    +13
Impacted Files Coverage Δ
...in/java/ca/uhn/fhir/jpa/batch/BatchJobsConfig.java 100.00% <ø> (ø)
...a/uhn/fhir/jpa/batch/job/model/PartitionedUrl.java 66.66% <ø> (ø)
...r/jpa/batch/listener/PidReaderCounterListener.java 100.00% <ø> (ø)
...a/uhn/fhir/jpa/batch/writer/SqlExecutorWriter.java 100.00% <ø> (ø)
...fhir/jpa/bulk/export/job/ResourceToFileWriter.java 91.17% <ø> (ø)
...ir/jpa/dao/BaseHapiFhirResourceDaoObservation.java 80.95% <ø> (ø)
.../uhn/fhir/jpa/dao/FhirResourceDaoPatientDstu2.java 92.59% <ø> (ø)
...jpa/dao/dstu3/FhirResourceDaoObservationDstu3.java 22.22% <ø> (ø)
.../uhn/fhir/jpa/dao/r4/FhirResourceDaoPatientR4.java 90.90% <ø> (ø)
.../uhn/fhir/jpa/dao/r5/FhirResourceDaoPatientR5.java 4.54% <ø> (ø)
... and 55 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b9761c...1bdd16c. Read the comment docs.

@lgtm-com
Copy link

lgtm-com bot commented Jul 29, 2021

This pull request introduces 1 alert and fixes 1 when merging 38bb227 into 1b9761c - view on LGTM.com

new alerts:

  • 1 for Boxed variable is never null

fixed alerts:

  • 1 for Spurious Javadoc @param tags

michaelabuckley
michaelabuckley requested changes Jul 29, 2021
View reviewed changes
Copy link
Contributor

@michaelabuckley michaelabuckley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. A couple of substantive comments and several nits for you to pick.

hapi-fhir-base/src/main/java/ca/uhn/fhir/interceptor/model/RequestPartitionId.java Outdated Show resolved Hide resolved
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/batch/job/PartitionedUrlValidator.java Outdated Show resolved Hide resolved
hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/batch/job/model/RequestListJson.java Outdated Show resolved Hide resolved
...hir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/batch/reader/BatchDateThresholdUpdater.java Show resolved Hide resolved
...r-base/src/main/java/ca/uhn/fhir/jpa/batch/reader/CronologicalBatchAllResourcePidReader.java Show resolved Hide resolved
hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/provider/ProviderConstants.java Show resolved Hide resolved
hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/provider/ProviderConstants.java Show resolved Hide resolved
hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/provider/ReindexProvider.java Show resolved Hide resolved
hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/server/provider/ReindexProvider.java Show resolved Hide resolved
hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5_5_0/2845-reindex-operation.yaml Outdated Show resolved Hide resolved
fil512 and others added 12 commits July 29, 2021 16:23
@fil512
review feedback
9c25258
@fil512
review feedback
02571aa
@fil512
review feedback
09820ec
@fil512 @michaelabuckley
Update hapi-fhir-docs/src/main/resources/ca/uhn/hapi/fhir/changelog/5…
c163086 
…_5_0/2845-reindex-operation.yaml

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
@fil512
review feedback
eae6d16
@fil512
review feedback
c19c404
@fil512 @michaelabuckley
Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/delete/…
8153ed6 
…job/ReindexJobTest.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
@fil512 @michaelabuckley
Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/delete/…
b8cbb2e 
…job/ReindexJobTest.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
@fil512 @michaelabuckley
Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/provide…
034cfb1 
…r/r4/MultitenantBatchOperationR4Test.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
@fil512 @michaelabuckley
Update hapi-fhir-jpaserver-base/src/test/java/ca/uhn/fhir/jpa/provide…
68c165b 
…r/r4/MultitenantBatchOperationR4Test.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
@fil512 @michaelabuckley
Update hapi-fhir-jpaserver-searchparam/src/main/java/ca/uhn/fhir/jpa/…
6e3f709 
…searchparam/ResourceSearch.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
@fil512 @michaelabuckley
Update hapi-fhir-server/src/main/java/ca/uhn/fhir/rest/api/server/sto…
1bdd16c 
…rage/IDeleteExpungeJobSubmitter.java

Co-authored-by: michaelabuckley <michaelabuckley@gmail.com>
@fil512 fil512 enabled auto-merge (squash) July 29, 2021 21:49
@lgtm-com
Copy link

lgtm-com bot commented Jul 29, 2021

This pull request introduces 1 alert and fixes 1 when merging 1bdd16c into 1b9761c - view on LGTM.com

new alerts:

  • 1 for Boxed variable is never null

fixed alerts:

  • 1 for Spurious Javadoc @param tags

@fil512 fil512 disabled auto-merge July 29, 2021 23:10
@fil512 fil512 merged commit f5dfc3a into master Jul 29, 2021
@fil512 fil512 deleted the ks-20210724-reindex-job branch July 29, 2021 23:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@michaelabuckley michaelabuckley michaelabuckley requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@fil512 @michaelabuckley