Skip to content

Commit

Permalink
Address review comments
Browse files Browse the repository at this point in the history
  • Loading branch information
LysandreJik committed Apr 18, 2022
1 parent 6d27a15 commit 9fe6bb6
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions src/huggingface_hub/repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -238,11 +238,15 @@ def is_binary_file(filename: Union[str, Path]) -> bool:
`bool`: `True` if the file passed is a binary file, `False` otherwise.
"""
try:
with open(filename) as f:
with open(filename, "rb") as f:
content = f.read()

# Check for the presence of the null character in the string
return "\x00" in content
# Code sample taken from the following stack overflow thread
# https://stackoverflow.com/questions/898669/how-can-i-detect-if-a-file-is-binary-non-text-in-python/7392391#7392391
text_chars = bytearray(
{7, 8, 9, 10, 12, 13, 27} | set(range(0x20, 0x100)) - {0x7F}
)
return bool(content.translate(None, text_chars))
except UnicodeDecodeError:
return True

Expand Down

0 comments on commit 9fe6bb6

Please sign in to comment.