Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bump org.json dependency version #1586

Merged
merged 4 commits into from
Apr 26, 2023

Conversation

dai-chen
Copy link
Collaborator

@dai-chen dai-chen commented Apr 26, 2023

Description

  1. Bumped org.json version in all submodules for CVE fix
  2. Fixed failed UT and IT assertion due to breaking changes in Change the underflying storage for floating point numbers to BigDecimal stleary/JSON-java#441 (more details below)

Impact on UT/IT

org.json use BigDecimal to interpret floating point text in JSON. This caused the return value of JSONObject/JSONArray.get() and query() get changed from double to BigDecimal class.

Impact on User Interface

Minor changes I noticed in our response format which seems an improvement:

Previously:

POST _plugins/_sql
{
  "query": "SELECT 123.0"
}

{
  "schema": [
    {
      "name": "123.0",
      "type": "double"
    }
  ],
  "datarows": [
    [
      123  <-- the decimal part was missing
    ]
  ],
  "total": 1,
  "size": 1,
  "status": 200
}

Now:

curl -XPOST 'http://localhost:9200/_plugins/_sql' \
--header 'Content-Type: application/json' \
--data '{
  "query": "select 123.0"
}
'

{
  "schema": [
    {
      "name": "123.0",
      "type": "double"
    }
  ],
  "datarows": [
    [
      123.0  <-- exact the same as what user provided
    ]
  ],
  "total": 1,
  "size": 1,
  "status": 200
}

Issues Resolved

CVE-2022-45688

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen added the dependencies Pull requests that update a dependency file label Apr 26, 2023
@dai-chen dai-chen self-assigned this Apr 26, 2023
rupal-bq
rupal-bq previously approved these changes Apr 26, 2023
Yury-Fridlyand
Yury-Fridlyand previously approved these changes Apr 26, 2023
@codecov-commenter
Copy link

codecov-commenter commented Apr 26, 2023

Codecov Report

Merging #1586 (cc43408) into 2.7 (6cfe354) will not change coverage.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff            @@
##                2.7    #1586   +/-   ##
=========================================
  Coverage     97.18%   97.18%           
  Complexity     4104     4104           
=========================================
  Files           371      371           
  Lines         10459    10459           
  Branches        706      706           
=========================================
  Hits          10165    10165           
  Misses          287      287           
  Partials          7        7           
Flag Coverage Δ
sql-engine 97.18% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen dismissed stale reviews from Yury-Fridlyand and rupal-bq via 7b8aef3 April 26, 2023 21:51
Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen marked this pull request as ready for review April 26, 2023 23:36
@dai-chen dai-chen merged commit 0bc8628 into opensearch-project:2.7 Apr 26, 2023
13 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-1586-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0bc8628395b8443f6e1a611345901a7f2c2561fd
# Push it to GitHub
git push --set-upstream origin backport/backport-1586-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-1586-to-2.x.

@opensearch-trigger-bot
Copy link
Contributor

The backport to main failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-main main
# Navigate to the new working tree
pushd ../.worktrees/backport-main
# Create a new branch
git switch --create backport/backport-1586-to-main
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0bc8628395b8443f6e1a611345901a7f2c2561fd
# Push it to GitHub
git push --set-upstream origin backport/backport-1586-to-main
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-main

Then, create a pull request where the base branch is main and the compare/head branch is backport/backport-1586-to-main.

dai-chen added a commit to dai-chen/sql-1 that referenced this pull request May 1, 2023
* Bump org.json version for CVE

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix assertion by json array similar method

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix more assertions which failed on query path return BigDecimal

Signed-off-by: Chen Dai <daichen@amazon.com>

* Fix legacy expr value factory

Signed-off-by: Chen Dai <daichen@amazon.com>

---------

Signed-off-by: Chen Dai <daichen@amazon.com>
@dai-chen dai-chen deleted the fix-org-json-cve branch May 1, 2023 22:07
Yury-Fridlyand pushed a commit that referenced this pull request May 3, 2023
* Bump org.json version for CVE



* Fix assertion by json array similar method



* Fix more assertions which failed on query path return BigDecimal



* Fix legacy expr value factory



---------

Signed-off-by: Chen Dai <daichen@amazon.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request May 3, 2023
* Bump org.json version for CVE

* Fix assertion by json array similar method

* Fix more assertions which failed on query path return BigDecimal

* Fix legacy expr value factory

---------

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit ff2050d)
dai-chen added a commit that referenced this pull request May 5, 2023
* Bump org.json version for CVE

* Fix assertion by json array similar method

* Fix more assertions which failed on query path return BigDecimal

* Fix legacy expr value factory

---------

Signed-off-by: Chen Dai <daichen@amazon.com>
(cherry picked from commit ff2050d)

Co-authored-by: Chen Dai <daichen@amazon.com>
@opensearch-trigger-bot
Copy link
Contributor

The backport to 1.3 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-1.3 1.3
# Navigate to the new working tree
pushd ../.worktrees/backport-1.3
# Create a new branch
git switch --create backport/backport-1586-to-1.3
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 0bc8628395b8443f6e1a611345901a7f2c2561fd
# Push it to GitHub
git push --set-upstream origin backport/backport-1586-to-1.3
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-1.3

Then, create a pull request where the base branch is 1.3 and the compare/head branch is backport/backport-1586-to-1.3.

pjfitzgibbons added a commit that referenced this pull request May 8, 2023
@pjfitzgibbons pjfitzgibbons mentioned this pull request May 8, 2023
6 tasks
pjfitzgibbons added a commit that referenced this pull request May 8, 2023
Signed-off-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
@pjfitzgibbons pjfitzgibbons mentioned this pull request May 8, 2023
6 tasks
pjfitzgibbons added a commit that referenced this pull request May 9, 2023
Signed-off-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
opensearch-trigger-bot bot pushed a commit that referenced this pull request May 9, 2023
Signed-off-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
(cherry picked from commit 0c2ba70)
dai-chen pushed a commit that referenced this pull request May 10, 2023
Signed-off-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
(cherry picked from commit 0c2ba70)

Co-authored-by: Peter Fitzgibbons <peter.fitzgibbons@gmail.com>
MitchellGale pushed a commit to Bit-Quill/opensearch-project-sql that referenced this pull request Jun 12, 2023
…h-project#1598)

* Bump org.json version for CVE

* Fix assertion by json array similar method

* Fix more assertions which failed on query path return BigDecimal

* Fix legacy expr value factory

---------

Signed-off-by: Chen Dai <daichen@amazon.com>
Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport main backport 1.3 backport 2.x dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants