Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable Integration Test:test_get_json_object_single_quotes on DB 10.4 #10423

Merged

Conversation

SurajAralihalli
Copy link
Collaborator

@SurajAralihalli SurajAralihalli commented Feb 14, 2024

This PR skips test_get_json_object_single_quotes test on Databricks 10.4 as it shows incorrect behavior when single quotes are used in the input.

Resolves 10422

Signed-off-by: Suraj Aralihalli <suraj.ara16@gmail.com>
Copy link
Collaborator

@revans2 revans2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is the right fix and we need to do more testing here.

First of all in your testing [r'''{'key':'A'}'''], was parsed correctly. If we disable single quotes in our code is it still going to be parsed correctly? Second this feels like it is a bug in databricks, and we might want to contact them and report it instead.

We also have no tests to cover json_tuple with single quotes and I am not sure if the same issue exits there or not. I would like to see us first gather come information about what each version of databricks does that we support for both get_json_object, from_json, and json_tuple, when dealing with single quotes for the test cases you listed. After that we can decide what is the best course of action. In the short term if you want to disable the test for databricks feel free to do that with a reason pointing to the test failures that we are seeing.

@SurajAralihalli
Copy link
Collaborator Author

You're right! Thanks for pointing it out Bobby. I'll drop these changes and update this PR to skip this test on DB 10.4.

Signed-off-by: Suraj Aralihalli <suraj.ara16@gmail.com>
@SurajAralihalli SurajAralihalli changed the title Fix single quote get_json_object issue on DB 10.4 Disable Integration Test:test_get_json_object_single_quotes on DB 10.4 Feb 15, 2024
@SurajAralihalli SurajAralihalli marked this pull request as ready for review February 15, 2024 21:20
@SurajAralihalli
Copy link
Collaborator Author

build

1 similar comment
@revans2
Copy link
Collaborator

revans2 commented Feb 15, 2024

build

@SurajAralihalli SurajAralihalli merged commit 21ffabc into NVIDIA:branch-24.04 Feb 16, 2024
39 of 40 checks passed
@sameerz sameerz added the test Only impacts tests label Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Only impacts tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] test_get_json_object_single_quotes failure
3 participants