Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix handling of special characters in categorical values #757

Merged
merged 1 commit into from
May 17, 2024

Conversation

kaituo
Copy link
Collaborator

@kaituo kaituo commented May 15, 2024

Description

This PR addresses a bug that occurred when categorical values contained special characters such as colons. Previously, entities were stored as a comma-separated name-value pair string in the heatmap cell's custom data field. When a cell was clicked, the custom data was retrieved and the string was split via colons. This caused errors if the categorical value itself contained colons or if multiple name-value pairs included commas.

To fix this issue, this PR changes the storage format to a JSON string in the cell's custom data. When a cell is clicked, the JSON object is restored. Since JSON naturally handles special characters, this eliminates the need for additional special character handling.

Additionally, the previous implementation stored a newline-separated name-value pair in the y-axis and a comma-separated name-value pair in the cell tooltip. With entities now stored as JSON in custom data, we no longer have the flexibility to store these different formats. This PR standardizes the format to newline-separated name-value pairs for both the y-axis and cell tooltip.

Testing Done:

  • Conducted end-to-end testing in preview, historical, and real-time use cases.
  • Added unit tests.

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

This PR addresses a bug that occurred when categorical values contained special characters such as colons. Previously, entities were stored as a comma-separated name-value pair string in the heatmap cell's custom data field. When a cell was clicked, the custom data was retrieved and the string was split via colons. This caused errors if the categorical value itself contained colons or if multiple name-value pairs included commas.

To fix this issue, this PR changes the storage format to a JSON string in the cell's custom data. When a cell is clicked, the JSON object is restored. Since JSON naturally handles special characters, this eliminates the need for additional special character handling.

Additionally, the previous implementation stored a newline-separated name-value pair in the y-axis and a comma-separated name-value pair in the cell tooltip. With entities now stored as JSON in custom data, we no longer have the flexibility to store these different formats. This PR standardizes the format to newline-separated name-value pairs for both the y-axis and cell tooltip.

Testing Done:
* Conducted end-to-end testing in preview, historical, and real-time use cases.
* Added unit tests.

Signed-off-by: Kaituo Li <kaituo@amazon.com>
Copy link

codecov bot commented May 15, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 50.05%. Comparing base (e5cf2d8) to head (c2780a4).
Report is 6 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #757      +/-   ##
==========================================
- Coverage   50.31%   50.05%   -0.27%     
==========================================
  Files         166      166              
  Lines        5595     5922     +327     
  Branches     1075     1186     +111     
==========================================
+ Hits         2815     2964     +149     
- Misses       2508     2644     +136     
- Partials      272      314      +42     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@kaituo
Copy link
Collaborator Author

kaituo commented May 15, 2024

Not sure why coverage CI failed as I only added tests and didn't delete tests. Will leave it as it is.

@jackiehanyang
Copy link
Collaborator

Thanks for the fix and adding tests. LGTM!

@kaituo kaituo merged commit 9cb2b00 into opensearch-project:main May 17, 2024
12 of 13 checks passed
opensearch-trigger-bot bot pushed a commit that referenced this pull request May 17, 2024
This PR addresses a bug that occurred when categorical values contained special characters such as colons. Previously, entities were stored as a comma-separated name-value pair string in the heatmap cell's custom data field. When a cell was clicked, the custom data was retrieved and the string was split via colons. This caused errors if the categorical value itself contained colons or if multiple name-value pairs included commas.

To fix this issue, this PR changes the storage format to a JSON string in the cell's custom data. When a cell is clicked, the JSON object is restored. Since JSON naturally handles special characters, this eliminates the need for additional special character handling.

Additionally, the previous implementation stored a newline-separated name-value pair in the y-axis and a comma-separated name-value pair in the cell tooltip. With entities now stored as JSON in custom data, we no longer have the flexibility to store these different formats. This PR standardizes the format to newline-separated name-value pairs for both the y-axis and cell tooltip.

Testing Done:
* Conducted end-to-end testing in preview, historical, and real-time use cases.
* Added unit tests.

Signed-off-by: Kaituo Li <kaituo@amazon.com>
(cherry picked from commit 9cb2b00)
kaituo added a commit that referenced this pull request May 20, 2024
This PR addresses a bug that occurred when categorical values contained special characters such as colons. Previously, entities were stored as a comma-separated name-value pair string in the heatmap cell's custom data field. When a cell was clicked, the custom data was retrieved and the string was split via colons. This caused errors if the categorical value itself contained colons or if multiple name-value pairs included commas.

To fix this issue, this PR changes the storage format to a JSON string in the cell's custom data. When a cell is clicked, the JSON object is restored. Since JSON naturally handles special characters, this eliminates the need for additional special character handling.

Additionally, the previous implementation stored a newline-separated name-value pair in the y-axis and a comma-separated name-value pair in the cell tooltip. With entities now stored as JSON in custom data, we no longer have the flexibility to store these different formats. This PR standardizes the format to newline-separated name-value pairs for both the y-axis and cell tooltip.

Testing Done:
* Conducted end-to-end testing in preview, historical, and real-time use cases.
* Added unit tests.

Signed-off-by: Kaituo Li <kaituo@amazon.com>
(cherry picked from commit 9cb2b00)

Co-authored-by: Kaituo Li <kaituo@amazon.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants