Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] port CSV tests from Spark and univocity #6927

Open
Tracked by #2063
revans2 opened this issue Oct 27, 2022 · 0 comments
Open
Tracked by #2063

[FEA] port CSV tests from Spark and univocity #6927

revans2 opened this issue Oct 27, 2022 · 0 comments
Labels
test Only impacts tests

Comments

@revans2
Copy link
Collaborator

revans2 commented Oct 27, 2022

Is your feature request related to a problem? Please describe.
Spark does not have a lot of really good CSV tests. They use a third party CSV parser https://github.com/uniVocity/univocity-parsers/ and end up relying on it for a lot of the testing. We should try to port over as many tests as possible from these to our tests to help augment corner case testing. We should concentrate on tests that are for the default settings in Spark, but we should also add in as many tests as possible and verify that we fall back if they do not work, or mark them as xfail with the issues that would fix that problem.

@revans2 revans2 added feature request New feature or request ? - Needs Triage Need team to review and classify test Only impacts tests labels Oct 27, 2022
@revans2 revans2 mentioned this issue Oct 27, 2022
38 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Only impacts tests
Projects
None yet
Development

No branches or pull requests

1 participant