Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some text files are processed with null bytes #2212

Closed
pombredanne opened this issue Sep 7, 2020 · 0 comments · Fixed by #2213
Closed

Some text files are processed with null bytes #2212

pombredanne opened this issue Sep 7, 2020 · 0 comments · Fixed by #2213
Labels
bug text extraction

Comments

@pombredanne
Copy link
Contributor

There are some rare cases where a file pretends to be a text file, but contains null bytes further past its start and well beyond the horizon of file type checking. In these cases, we still want to strip null bytes as they can cause all kind of problems downstream and are not super interesting.
As an example the attached ReadMe.txt
from https://master.dl.sourceforge.net/project/passwordnotify/PasswordNotify/PasswordNotify%201.1b2/PasswordNotify-1.1b2.tgz
pombredanne added a commit that referenced this issue Sep 7, 2020
@pombredanne
Do not carry null bytes in text #2212
51307dd 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne pombredanne mentioned this issue Sep 7, 2020
Merged
4 tasks
pombredanne added a commit that referenced this issue Sep 7, 2020
@pombredanne
Merge pull request #2213 from nexB/2212-no-null-bytes
138abda 
Do not carry null bytes in text #2212
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug text extraction
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Do not carry null bytes in text #2212 aboutcode-org/scancode-toolkit
1 participant
@pombredanne