Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

strftime produces corrupted characters in windows #2961

Open
baterflyrity opened this issue Dec 3, 2023 · 7 comments
Open

strftime produces corrupted characters in windows #2961

baterflyrity opened this issue Dec 3, 2023 · 7 comments

Comments

@baterflyrity
Copy link

baterflyrity commented Dec 3, 2023

Describe the bug
Using strftime function in Windows produces unreadable symbols in console.

To Reproduce
Taken from

strftime("%A, %B %e, %Y")
and

jq/tests/jq.test

Line 1562 in 693a726

strftime("%A, %B %d, %Y")
:

$ echo 1435677542.822351 | jq "strftime(\"%A, %B %d, %Y\")"
"�������, ���� 30, 2015"

Using both windows 1251 and utf 8 encodings gives same corrupted output.

Expected behavior
The output should be readable.

Environment (please complete the following information):

  • OS and Version: Windows 10 x64
  • jq version: jq-1.7-dirty

Additional context

Both files contain same test case.

@wader
Copy link
Member

wader commented Dec 3, 2023

Huh weird, possibly to pipe it thru hexdump -C or possibly redirect to a file and attach? but maybe it will just show replacement characters hmm

@itchyny
Copy link
Contributor

itchyny commented Dec 3, 2023

Did you try chcp 65001 ?

@baterflyrity
Copy link
Author

Huh weird, possibly to pipe it thru hexdump -C or possibly redirect to a file and attach? but maybe it will just show replacement characters hmm

Sure.
Here it is:
jq#2961.txt

echo 1435677542.822351 | jq "strftime(\"%A, %B %d, %Y\")" > jq#2961.txt

22efbfbdefbfbdefbfbdefbfbdefbfbdefbfbdefbfbd2c20efbfbdefbfbdefbfbdefbfbd2033302c2032303135220d0a

Did you try chcp 65001 ?

Yes, of course. Same for any codepage.

$ chcp 65001
$ echo 1435677542.822351 | jq "strftime(\"%A, %B %d, %Y\")"
"�������, ���� 30, 2015"

@itchyny itchyny added the windows label Dec 3, 2023
@wader
Copy link
Member

wader commented Dec 3, 2023

Managed to kind of produce it with wine (no windows dev setup atm) and jq-windows-amd64.exe

$ docker run --platform linux/amd64 -ti --rm debian:bookworm
$ apt-get update
$ apt-get install -y curl wine
$ curl -OLJ https://github.com/jqlang/jq/releases/download/jq-1.7/jq-windows-amd64.exe

# same with LC_ALL=ru_RU.UTF-8
$ LC_ALL=ru_RU.CP1251 wine jq-windows-amd64.exe -n '1435677542.822351 | strftime("%A, %B %d, %Y")'
<lots-of-warnings>
"???????, ???? 30, 2015"
$ LC_ALL=ru_RU.UTF-8 wine jq-windows-amd64.exe -n '1435677542.822351 | strftime("%A, %B %d, %Y")'
<lots-of-warnings>
"???????, ???? 30, 2015"

# default local (if i run locale is says "POSIX")
$ wine jq-windows-amd64.exe -n '1435677542.822351 | strftime("%A, %B %d, %Y")'
"Tuesday, June 30, 2015"

# swedish seems to work
$ LC_ALL=sv.SE.CP1251 wine jq-windows-amd64.exe -n '1435677542.822351 | strftime("%A, %B %d, %Y")'
"tisdag, juni 30, 2015"

No idea how compareable this is to a proper windows installation but maybe there is something with msys2 and some locales etc?

@nicowilliams
Copy link
Contributor

I wonder if we have to make stdout binary on Windows. It seems like something is interpreting the output.

@Biswa96
Copy link

Biswa96 commented Feb 13, 2024

I can not reproduce the issue with jq from msys2 https://packages.msys2.org/base/mingw-w64-jq

$ echo 1435677542.822351 | jq "strftime(\"%A, %B %d, %Y\")"
"Tuesday, June 30, 2015"

No locale variable set, system language is US English.

@baterflyrity
Copy link
Author

Seems like non-latin characters are corrupted.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

5 participants