Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create new anonymizer for new engine #1665

Merged
merged 4 commits into from
May 30, 2023

Conversation

matthewryanwells
Copy link
Contributor

Description

Created a new listener for the SQLSyntaxParser to anonymize all queries going through the new engine, calling the old anonymizer only if the query falls back to the legacy engine.

Through the update of the anonymizer there are a few differences between between the current and old anonymization including:

  • Dots are used to access elements of a table, e.g. identifier.identifier instead of simplifying as just identifier
  • All functions are capitalized
  • Adds a space between all nodes except for before and after a dot, before a comma, and between compound comparison symbols (<=, >=, <>, !=)
  • Includes the negative symbol when anonymizing numbers, e.g. number / - number instead number / number

Issues Resolved

#821

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

* Created anonymizer listener for anonymizing SQL queries through the new engine
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
@codecov
Copy link

codecov bot commented May 29, 2023

Codecov Report

Merging #1665 (2055d00) into main (9824b1c) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##               main    #1665   +/-   ##
=========================================
  Coverage     97.16%   97.17%           
- Complexity     4120     4146   +26     
=========================================
  Files           371      372    +1     
  Lines         10373    10407   +34     
  Branches        704      711    +7     
=========================================
+ Hits          10079    10113   +34     
  Misses          287      287           
  Partials          7        7           
Flag Coverage Δ
sql-engine 97.17% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...g/opensearch/sql/sql/antlr/AnonymizerListener.java 100.00% <100.00%> (ø)
.../org/opensearch/sql/sql/antlr/SQLSyntaxParser.java 100.00% <100.00%> (ø)

MaxKsyunz
MaxKsyunz previously approved these changes May 29, 2023
Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>
acarbonetto
acarbonetto previously approved these changes May 30, 2023
MaxKsyunz
MaxKsyunz previously approved these changes May 30, 2023
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
@acarbonetto acarbonetto merged commit 62120fd into opensearch-project:main May 30, 2023
15 checks passed
@acarbonetto acarbonetto deleted the integ-sql-anonymizer branch May 30, 2023 19:48
opensearch-trigger-bot bot pushed a commit that referenced this pull request May 30, 2023
* Create new anonymizer for new engine (#266)

* Created anonymizer listener for anonymizing SQL queries through the new engine
Signed-off-by: Matthew Wells <matthew.wells@improving.com>

* Update for review comments

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>

* added missing file header, change public variable to private

Signed-off-by: Matthew Wells <matthew.wells@improving.com>

---------

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>
(cherry picked from commit 62120fd)
opensearch-trigger-bot bot pushed a commit that referenced this pull request May 30, 2023
* Create new anonymizer for new engine (#266)

* Created anonymizer listener for anonymizing SQL queries through the new engine
Signed-off-by: Matthew Wells <matthew.wells@improving.com>

* Update for review comments

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>

* added missing file header, change public variable to private

Signed-off-by: Matthew Wells <matthew.wells@improving.com>

---------

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>
(cherry picked from commit 62120fd)
@Yury-Fridlyand Yury-Fridlyand mentioned this pull request May 30, 2023
6 tasks
Yury-Fridlyand pushed a commit that referenced this pull request May 30, 2023
* Create new anonymizer for new engine (#266)

* Created anonymizer listener for anonymizing SQL queries through the new engine
Signed-off-by: Matthew Wells <matthew.wells@improving.com>

* Update for review comments

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>

* added missing file header, change public variable to private

Signed-off-by: Matthew Wells <matthew.wells@improving.com>

---------

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>
(cherry picked from commit 62120fd)

Co-authored-by: Matthew Wells <matthew.wells@improving.com>
acarbonetto pushed a commit that referenced this pull request May 30, 2023
* Create new anonymizer for new engine (#266)

* Created anonymizer listener for anonymizing SQL queries through the new engine
Signed-off-by: Matthew Wells <matthew.wells@improving.com>

* Update for review comments

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>

* added missing file header, change public variable to private

Signed-off-by: Matthew Wells <matthew.wells@improving.com>

---------

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>
(cherry picked from commit 62120fd)

Co-authored-by: Matthew Wells <matthew.wells@improving.com>
MitchellGale pushed a commit to Bit-Quill/opensearch-project-sql that referenced this pull request Jun 12, 2023
* Create new anonymizer for new engine (#266)

* Created anonymizer listener for anonymizing SQL queries through the new engine
Signed-off-by: Matthew Wells <matthew.wells@improving.com>

* Update for review comments

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>

* added missing file header, change public variable to private

Signed-off-by: Matthew Wells <matthew.wells@improving.com>

---------

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>
MitchellGale pushed a commit to Bit-Quill/opensearch-project-sql that referenced this pull request Jun 12, 2023
* Create new anonymizer for new engine (#266)

* Created anonymizer listener for anonymizing SQL queries through the new engine
Signed-off-by: Matthew Wells <matthew.wells@improving.com>

* Update for review comments

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>

* added missing file header, change public variable to private

Signed-off-by: Matthew Wells <matthew.wells@improving.com>

---------

Signed-off-by: Andrew Carbonetto <andrewc@bitquilltech.com>
Signed-off-by: Matthew Wells <matthew.wells@improving.com>
Co-authored-by: Andrew Carbonetto <andrewc@bitquilltech.com>
Signed-off-by: Mitchell Gale <Mitchell.Gale@improving.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants