Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

importccl: fix MYSQLOUTFILE edge cases #26275

Merged
merged 2 commits into from Jun 1, 2018
Merged

importccl: fix MYSQLOUTFILE edge cases #26275

merged 2 commits into from Jun 1, 2018

Conversation

maddyblue
Copy link
Contributor

No description provided.

@maddyblue maddyblue requested review from dt and a team May 31, 2018 07:05
@cockroach-teamcity
Copy link
Member

This change is Reviewable

opts roachpb.MySQLOutfileOptions
conv rowConverter
opts roachpb.MySQLOutfileOptions
debugRow func(tree.Datums)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this used?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

was during debugging. removed.

}
for query, res := range tc.query {
sqlDB.CheckQueryResults(t, query, res)
}
})
}

t.Run("mysqlout multpile", func(t *testing.T) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: multiple

Previously, the MYSQLOUTFILE file reader would convert its data to
CSV and use the CSV workers to convert it to datums. This worked for
most data, but was not able to correctly handle all NULL encodings.

To fix this, decouple MYSQLOUTFILE from CSV so that it can parse
datums in its own way and thus handle NULLs correctly. This does lose
parallelization, but we can come up with ways to do that in the future.

As a side effect, this fixes the listed bug, which occurred because
the csvRecord field wasn't being correctly initialized since readFile
was shadowing the CSV readFile.

Fixes #26153

Release note (bug fix): correct NULL handling during IMPORT of
MYSQLOUTFILE.
@maddyblue
Copy link
Contributor Author

bors r+

craig bot pushed a commit that referenced this pull request Jun 1, 2018
26275: importccl: fix MYSQLOUTFILE edge cases r=mjibson a=mjibson



Co-authored-by: Matt Jibson <matt.jibson@gmail.com>
@craig
Copy link
Contributor

craig bot commented Jun 1, 2018

Build succeeded

@craig craig bot merged commit fe81c74 into cockroachdb:master Jun 1, 2018
@maddyblue maddyblue deleted the mysqlout-fixes branch June 2, 2018 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants