Handle file names with special characters #25
Conversation
| err = csvWriter.Write([]string{"Path", "BlobId", "IsFile"}) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to write file headers '%v': %v", optionalIndexFilePath, err) | ||
| } |
There was a problem hiding this comment.
did we write headers previously also?
There was a problem hiding this comment.
no, but this helps in C# to deserialize the data
git/git_test.go
Outdated
| @@ -510,7 +510,7 @@ func (gitSuite *gitTestSuite) TestSnapshotWithIndexPath() { | |||
| }) | |||
|
|
|||
| gitSuite.Nil(err) | |||
| gitSuite.verifyIndexFile(40, indexFilePath) | |||
| gitSuite.verifyIndexFile(41, indexFilePath) | |||
There was a problem hiding this comment.
Is it 41 cause you added a file with special characters in the test repo?
There was a problem hiding this comment.
nope. because of the headers line...
There was a problem hiding this comment.
Might be a good idea to add a test for it then
There was a problem hiding this comment.
I meant a test for the special characters
There was a problem hiding this comment.
this requires adding these files to dc-heacth github.com/apiirolab (the repo that the test is running on).
I don't have permissions to, and I am not sure what this could break...
we can also get back to this later.
| return fmt.Errorf("failed to read paths file from location: '%v', error: '%v'", opts.PathsFileLocation, err) | ||
| } | ||
| for i := range lines { | ||
| provider.fileListToSnap[lines[i][0]] = true |
There was a problem hiding this comment.
Do you think we need some validation that lines[i] isn't empty? or do we trust the flow?
There was a problem hiding this comment.
So I am writing this as a CSV from C#, and I didn't see any way that this broke when I tested it...
Do you have a specific case in mind you want me to test?
|
LGTM, but do you want to wait for Ariel? |
What
Use the Golang built-in CSV library to escape special characters in file names automatically
Why
Some customers (like EA, Shell, and Included Health) have files with special characters like
\nor\rin their names.Currently, since we're not really writing a CSV this breaks the file format.
Example:
Say the customer has a file called
weird\nname.txt.Right now, the output file will be:
using the CSV library this becomes:
if we read this using a CSV reader, it should be able to handle this correctly (tested in C#)
Same for the input file list.
How
Use CSV readers and writers on both ends instead of self-parsing the CSV