Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support JOIN query on object field with unexpanded name #1229

Merged
merged 4 commits into from
Jan 9, 2023

Conversation

dai-chen
Copy link
Collaborator

@dai-chen dai-chen commented Jan 5, 2023

Signed-off-by: Chen Dai daichen@amazon.com

Description

This PR is to fix JOIN query on object field which has unexpanded name. This is very common and convenient for user to create object field deep nested. Please check my comments in issue: #1135 (comment). In short, OpenSearch gives different response format, "account.name.first": "John" rather than "account": {"name": {"first": "John"}}}, if the doc was created in this way:

PUT join-object-test/_doc/1
{ 
  "account.name.first": "John"
}

POST localhost:9200/join-object-test/_search
{
  "took": 2,
   ...
    "hits": [
      {
        "_index": "join-object-test",
        "_id": "Ac00gIUBc6yO9RmmMeTE",
        "_score": 1.0,
        "_source": {
          "account.name.first": "John"
        }
      }
    ]
  }
}

TODO

This PR is a quick fix for upcoming 2.5.0 release. Afterwards, we need to check if V2 engine has same problem and fix it. We may want to document the limitation if complicated cases like #1135 (comment) is hard to support finally.

Issues Resolved

#1135

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen added bug Something isn't working backport 2.x labels Jan 5, 2023
@dai-chen dai-chen self-assigned this Jan 5, 2023
@dai-chen dai-chen added the SQL label Jan 5, 2023
@dai-chen dai-chen linked an issue Jan 5, 2023 that may be closed by this pull request
@codecov-commenter
Copy link

codecov-commenter commented Jan 5, 2023

Codecov Report

Merging #1229 (6cb5f25) into main (1b58f7d) will decrease coverage by 2.45%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main    #1229      +/-   ##
============================================
- Coverage     98.32%   95.86%   -2.46%     
- Complexity     3550     3575      +25     
============================================
  Files           344      354      +10     
  Lines          8762     9477     +715     
  Branches        554      678     +124     
============================================
+ Hits           8615     9085     +470     
- Misses          142      334     +192     
- Partials          5       58      +53     
Flag Coverage Δ
query-workbench 62.76% <ø> (?)
sql-engine 98.33% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...n/java/org/opensearch/sql/utils/DateTimeUtils.java 100.00% <0.00%> (ø)
...a/org/opensearch/sql/data/model/ExprDateValue.java 100.00% <0.00%> (ø)
...a/org/opensearch/sql/data/model/ExprTimeValue.java 100.00% <0.00%> (ø)
.../org/opensearch/sql/data/model/ExprValueUtils.java 100.00% <0.00%> (ø)
...g/opensearch/sql/data/model/ExprDatetimeValue.java 100.00% <0.00%> (ø)
...pensearch/sql/expression/function/FunctionDSL.java 100.00% <0.00%> (ø)
...arch/sql/expression/datetime/DateTimeFunction.java 100.00% <0.00%> (ø)
...h/sql/expression/function/BuiltinFunctionName.java 100.00% <0.00%> (ø)
...h/public/components/QueryLanguageSwitch/Switch.tsx 85.71% <0.00%> (ø)
workbench/public/components/SQLPage/SQLPage.tsx 100.00% <0.00%> (ø)
... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@dai-chen dai-chen changed the title Support sub object field in JOIN query Support object field with unexpanded name in JOIN query Jan 5, 2023
@dai-chen dai-chen changed the title Support object field with unexpanded name in JOIN query Support JOIN query on object field with unexpanded name Jan 5, 2023
Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen marked this pull request as ready for review January 6, 2023 18:40
@dai-chen dai-chen requested a review from a team as a code owner January 6, 2023 18:40
@dai-chen dai-chen merged commit 151f4cc into opensearch-project:main Jan 9, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 9, 2023
* Resolve sub object field in search hit source

Signed-off-by: Chen Dai <daichen@amazon.com>

* Rename to unexpanded object

Signed-off-by: Chen Dai <daichen@amazon.com>

* Update IT with where condition

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix test index mapping

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit 151f4cc)
opensearch-trigger-bot bot pushed a commit that referenced this pull request Jan 9, 2023
* Resolve sub object field in search hit source

Signed-off-by: Chen Dai <daichen@amazon.com>

* Rename to unexpanded object

Signed-off-by: Chen Dai <daichen@amazon.com>

* Update IT with where condition

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix test index mapping

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit 151f4cc)
penghuo pushed a commit that referenced this pull request Jan 9, 2023
* Resolve sub object field in search hit source

Signed-off-by: Chen Dai <daichen@amazon.com>

* Rename to unexpanded object

Signed-off-by: Chen Dai <daichen@amazon.com>

* Update IT with where condition

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix test index mapping

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit 151f4cc)

Co-authored-by: Chen Dai <daichen@amazon.com>
penghuo added a commit that referenced this pull request Jan 10, 2023
* Add BWC tests for running against distribution bundle.  (#1209)

Signed-off-by: Zelin Hao <zelinhao@amazon.com>

* Add Alternate Syntax For Match_Query And Other Functions (#1166)

Added Tests And Implementation For Match_Query, Match_Phrase, and Multi_Match Functions

Signed-off-by: GabeFernandez310 <gabrielf@bitquilltech.com>

Signed-off-by: GabeFernandez310 <gabrielf@bitquilltech.com>
Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

* Merge pull request #1241 from Bit-Quill/Failing-CI-Hot-Fix

Hot Fix For CI Build

(cherry picked from commit aae57a0)
Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

* Fixed error with single timestamp query (#1244) (#1246)

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>

Signed-off-by: vamsi-amazon <reddyvam@amazon.com>
(cherry picked from commit ee949cc)

Co-authored-by: vamsi-amazon <reddyvam@amazon.com>

* Add Second_Of_Minute Function As An Alias Of The Second Function (#1231) (#1237)

Added Testing And Implementation For Second_Of_Minute Function

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>
(cherry picked from commit dce7d0e)

Co-authored-by: GabeFernandez310 <gabrielf@bitquilltech.com>

* Add functions `ADDTIME` and `SUBTIME`. (#132) (#1194) (#1252)

* Add functions `ADDTIME` and `SUBTIME`. (#132)

Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
(cherry picked from commit 7630f87)

Co-authored-by: Yury-Fridlyand <yury.fridlyand@improving.com>

* Add Day_Of_Week Function As An Alias Of DayOfWeek (#190) (#1228) (#1239)

Added Implementation And Testing For Day_Of_Week Function

Signed-off-by: GabeFernandez310 <gabrielf@bitquilltech.com>

Signed-off-by: GabeFernandez310 <gabrielf@bitquilltech.com>
(cherry picked from commit bac9c37)

Co-authored-by: GabeFernandez310 <gabrielf@bitquilltech.com>

* [Backport 2.x] Add Minute_Of_Hour Function As An Alias Of Minute Function (#1253)

* Add Minute_Of_Hour Function As An Alias Of Minute Function (#196) (#1230)

Added Testing And Implementation For Minute_Of_Hour Function

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>
(cherry picked from commit 61e2374)

* Added Missing Imports

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

* Add support for long value return for CEIL, CEILING and FLOOR math functions (#1205) (#1255)

* Added long fix for CEIL, CEILING and FLOOR functions using LONG instead of INT for RETURN.

Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
Co-authored-by: Yury-Fridlyand <yury.fridlyand@improving.com>

* Support JOIN query on object field with unexpanded name (#1229) (#1250)

* Resolve sub object field in search hit source

Signed-off-by: Chen Dai <daichen@amazon.com>

* Rename to unexpanded object

Signed-off-by: Chen Dai <daichen@amazon.com>

* Update IT with where condition

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix test index mapping

Signed-off-by: Chen Dai <daichen@amazon.com>

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit 151f4cc)

Co-authored-by: Chen Dai <daichen@amazon.com>

* Remove unnecessary scripts after repo split (#1256)

Signed-off-by: Joshua Li <joshuali925@gmail.com>

* Add Support For `TIME` Type in "*_OF_YEAR" Functions (#199) (#1223) (#1258)

Added Support And Tests For Time Type in day_of_year, week_of_year, month_of_year Functions
Signed-off-by: GabeFernandez310 <gabrielf@bitquilltech.com>
(cherry picked from commit 6e72f18)

Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>

Signed-off-by: Zelin Hao <zelinhao@amazon.com>
Signed-off-by: GabeFernandez310 <gabrielf@bitquilltech.com>
Signed-off-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>
Signed-off-by: MitchellGale-BitQuill <mitchellg@bitquilltech.com>
Signed-off-by: Yury-Fridlyand <yury.fridlyand@improving.com>
Signed-off-by: Joshua Li <joshuali925@gmail.com>
Co-authored-by: Zelin Hao <87548827+zelinh@users.noreply.github.com>
Co-authored-by: GabeFernandez310 <gabrielf@bitquilltech.com>
Co-authored-by: YANGDB <yang.db.dev@gmail.com>
Co-authored-by: opensearch-trigger-bot[bot] <98922864+opensearch-trigger-bot[bot]@users.noreply.github.com>
Co-authored-by: vamsi-amazon <reddyvam@amazon.com>
Co-authored-by: Yury-Fridlyand <yury.fridlyand@improving.com>
Co-authored-by: GabeFernandez310 <Gabriel.Fernandez@improving.com>
Co-authored-by: Chen Dai <daichen@amazon.com>
Co-authored-by: Joshua Li <joshuali925@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] JOIN query with object field in joined condition return nothing
4 participants