issue #262 modify SQLQueryBuilder to improve performance.#308
issue #262 modify SQLQueryBuilder to improve performance.#308albertwang-ibm merged 10 commits intomasterfrom
Conversation
sync with master
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
|
I see that you added buildWhereClause2. Do we need both buildWhereClause and buildWhereClause2? Can we remove one of them? |
|
@albertwang-ibm do you have examples of the old sql vs the new sql? or the comments in this class still up-to-date for the queries being generated? |
|
@JohnTimm I had tried to use one unified optimized version for both, but unfortunately didn't work well even though I had spent a whole afternoon trying to make it work. the system one doesn't perform well for the extension search etc, and the other one failed about 10 test cases of the system search. and yes, I think I can spend one more day to try to create a workable unified version for both. @lmsurpre yes, after we agree with the PR, I will update the comments with the new generated sql. |
|
While reviewing this PR and our related functionality, I found #314 and I opened a competing pull request to fix that. I'm pretty sure there will be conflicts, so please review my pull request before proceeding with this one. |
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
|
@JohnTimm Done! successfully use unified from/where clausebuilder codes now! |
sync with master
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
|
Changes: |
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
lmsurpre
left a comment
There was a problem hiding this comment.
I still don't fully understand why the SQL query plan engines aren't smart enough to execute the subquery just once, but this LGTM pending the one last javadoc consistency change in my last 2 comments.
Signed-off-by: Albert Wang <xuwang@us.ibm.com>
JohnTimm
left a comment
There was a problem hiding this comment.
LGTM. FHIR when ready. :)
Changes made:
(1) Rewrite From/Where Clauses generators for the basic QuerySegmentAggregator;
(2) Simplify InclusionQuerySegmentAggregator for search _include query.
Test results:
Huge improvement of search performance, e.g,
(1) testSearchPatientWithBaseParametersAndExtensions of SearchExtensionsTest:
--Before the changes--
always timeout whenever there are about 50 patients created
--After the changes--
Even with more than 5000 patient created, this search test can still finish in just few seconds!
(2) testSearchObservationWithSubjectIncluded and testSearchObservationWithSubjectIncluded_filter_elements of SearchTest:
--Before the changes--
took more than 15 seconds if there were about 1000 patient created
always timeout whenever there are about 5000 patients created
--After the changes--
Even with more than 5000 patient created, this search test can still finish in just few seconds!