Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Git does not work with escaped unicode characcters #2418

Closed
rgmz opened this issue Feb 11, 2024 · 1 comment · Fixed by #2585
Closed

Git does not work with escaped unicode characcters #2418

rgmz opened this issue Feb 11, 2024 · 1 comment · Fixed by #2585
Labels

Comments

@rgmz
Copy link
Contributor

rgmz commented Feb 11, 2024

Please review the Community Note before submitting

TruffleHog Version

3.67.5

Trace Output

2024-02-11T17:22:52-05:00       error   trufflehog      error waiting for command       {"source_manager_worker_id": "sIEwQ", "repo": "https://github.com/appleboy/DeepLearning-500-questions.git", "command": "/usr/bin/git -C /tmp/trufflehog-169189-937274496/.git cat-file blob f71e35a836b7f97301a23fe6af6a8305e285467c:\\346\\267\\261\\345\\272\\246\\345\\255\\246\\344\\271\\240500\\351\\227\\256-Tan-00\\347\\233\\256\\345\\275\\225.docx", "stderr": "fatal: Not a valid object name f71e35a836b7f97301a23fe6af6a8305e285467c:\\346\\267\\261\\345\\272\\246\\345\\255\\246\\344\\271\\240500\\351\\227\\256-Tan-00\\347\\233\\256\\345\\275\\225.docx\n", "commit": "f71e35a836b7f97301a23fe6af6a8305e285467c", "error": "exit status 128"}

(The corresponding file is ]文件500页-Tan-00数据统计.docx)

Expected Behavior

The git cat-file command should handle escaped filenames.

cmd := exec.Command("git", "-C", gitDir, "cat-file", "blob", commitHash.String()+":"+path)

Actual Behavior

The command should pass in unescaped characters.

e.g.,

git cat-file blob e48902879384ff53812c6420440db9ac1c50bb99:"$(printf '\320\244\320\270\320\267\320\270\320\272\320\260/\320\255\320\272\320\267\320\260\320\274\320\265\320\275/\320\236\321\202\321\201\320\276\321\200\321\202\320\270\321\200\320\276\320\262\320\260\320\275\320\275\321\213\320\265 \321\200\320\265\321\210\320\265\320\275\320\275\321\213\320\265 \320\261\320\270\320\273\320\265\321\202\321\213 2017/1 \320\261\320\270\320\273\320\265\321\202.pdf')"

Steps to Reproduce

Attempt to scan https://github.com/scutan90/DeepLearning-500-questions

Environment

N/A

Additional Context

  1. This is similar to GitParse error: Expected binary line to be in 'Binary files a/fileA and b/fileB differ' format. #2384. Need to figure out if the solution is to add quotes or to decode the name. (https://stackoverflow.com/a/69817595)
  2. PR [feat] - Remove go-git dependency #2174 replaced go-git with direct calls to Git. It's worth testing the capabilities of go-git to see if it handles edge cases like this.

References

N/A

@rgmz rgmz added the bug label Feb 11, 2024
@rgmz
Copy link
Contributor Author

rgmz commented Feb 12, 2024

Another failing file: https://github.com/Appverse/appverse-web-showcases/blob/c2c7f9b4568607f933e139525e393383f5738e17/C:%5Cdevelopment%5Crepositories%5Cmaven/com/adobe/flex/compiler/asdoc/3.2.0.3958/asdoc-3.2.0.3958-template.zip

2024-02-11T21:49:54-05:00       error   trufflehog      error waiting for command       {"source_manager_worker_id": "MugHm", "repo": "https://github.com/iromu/appverse-web-showcases.git", "command": "/usr/bin/git -C /tmp/trufflehog-435612-893384690/.git cat-file blob c2c7f9b4568607f933e139525e393383f5738e17:C:\\\\development\\\\repositories\\\\maven/com/adobe/flex/compiler/asdoc/3.2.0.3958/asdoc-3.2.0.3958-template.zip", "stderr": "fatal: Not a valid object name c2c7f9b4568607f933e139525e393383f5738e17:C:\\\\development\\\\repositories\\\\maven/com/adobe/flex/compiler/asdoc/3.2.0.3958/asdoc-3.2.0.3958-template.zip\n", "commit": "c2c7f9b4568607f933e139525e393383f5738e17", "error": "exit status 128"}

https://github.com/LionZXY/Technopark/blob/e50b135fd29e91b2fbb25923797f5ecffe59f359/C%2B%2B/1%20%D0%A3%D1%80%D0%BE%D0%BA/cmake-build-debug/1_

2024-02-12T16:16:51-05:00       error   trufflehog      error waiting for command       {"source_manager_worker_id": "LGnot", "repo": "https://github.com/LionZXY/Technopark.git", "command": "/usr/bin/git -C /tmp/trufflehog-1292053-2945290590/.git cat-file blob e50b135fd29e91b2fbb25923797f5ecffe59f359:C++/1 \\320\\243\\321\\200\\320\\276\\320\\272/cmake-build-debug/1_", "stderr": "fatal: Not a valid object name e50b135fd29e91b2fbb25923797f5ecffe59f359:C++/1 \\320\\243\\321\\200\\320\\276\\320\\272/cmake-build-debug/1_\n", "commit": "e50b135fd29e91b2fbb25923797f5ecffe59f359", "error": "exit status 128"}

diff --git "a/C++/1 \320\243\321\200\320\276\320\272/.idea/workspace.xml" "b/C++/1 \320\243\321\200\320\276\320\272/.idea/workspace.xml"
index 85bfb17..89b08b5 100644
--- "a/C++/1 \320\243\321\200\320\276\320\272/.idea/workspace.xml"
+++ "b/C++/1 \320\243\321\200\320\276\320\272/.idea/workspace.xml"

@rgmz rgmz changed the title Git.handleBinary does not work with escaped unicode characcters Git does not work with escaped unicode characcters Mar 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging a pull request may close this issue.

1 participant