Non latin symbols replaced with ???? after formatting in Windows 10.

[Real-Gecko]

If template contains non ASCII symbols after formatting they're replaced with ? characters. Running djLint directly from command line does not have this issue. So I guess it's somewhere in extension. File encoding is utf-8, OS is Windows 10. In Linux and MacOS everything works as expected.

[monosans]

Solution: djlint/djLint#243 (comment).

EDIT: This solution is bad. Update djLint to version 1.1.1 or higher to fix the issue.

[Real-Gecko]

Yikes it worked :D
Maybe it should be reflected in a README? so that no one ever asks this question again?

[christopherpickering]

Thats windows for ya 🤣

[Real-Gecko]

Nah I work on Linux and MacOS, but my colleague are not. I've tired of seeing unformatted templates so :D

[Real-Gecko]

OK, after enabling utf-8 in winduz some other programs has stopped working as expected. So option was turned off again. So, running from command line works as expected, running through extension does not. Maybe we can tweak something on extension side?
Maybe we can add an option to do formatting inplace and ask editor for reload instead of asking it to replace the text in here?
https://github.com/monosans/djlint-vscode/blob/89073365e968effce95dffd99bbf2dc75d2bb579/src/extension.ts#L247
AFAIK some extensions do that, because I have formatting enabled on save, and when I save the file for some time I see file not being formatted, and then suddenly it changes and is resaved. Noticable when you have a lot of files to save at the same time.

[christopherpickering]

I'll dust of my windows and take a look tomorrow as well. Do you have a small sample code? I'll try w/ 必須 from the other issue otherwise.

[Real-Gecko]

{% extends 'base.html' %}

{% load i18n django_bootstrap5 %}

{% block content %}
<div class="col-4 offset-4 card">
<div class="card card-body">
<form method="post">
{% csrf_token %}
{% bootstrap_form form %}
<button class="btn btn-primary" type="submit">{% trans "Вход" %}</button>
</form>
</div>
</div>
{% endblock content %}

This one will do I guess notice the {% trans "Вход" %} part.

[monosans]

Perhaps this will help solve the issue. I'll do it today when I'm at home.

[christopherpickering]

I tested on my windows > using the output from stdout from djlint does make the "??":

It does seem to be from the default windows encoding for cmd.

Git bash it works :D

And running click directly works:

So it must be something in the process of writing the temp file to reading it in djlint.

I popped open the temp file that stdin was using... it has the dreaded ??

So I followed the ?? upstream in djlint until I got to the stdin and realized that I was reading it as the systems encoding.. which on windows, is not utf8.

After setting it to read as utf8, and also changing windows shell to utf8, it works:

Do djlint needs to be updated to read stdin as utf8. Then djlint-vscode will need to be updated to spawn with utf8.
I'll do a new djlint release w/ this update soon.

[monosans]

@christopherpickering
I see that you have released an update that fixes this issue, cool. Can you please check if I need to make any changes to the extension on my end to fix this problem too. It's just that I don't use Windows so I can't test it.

[monosans]

AFAIK they use PowerShell as VSCode shell. Don't know what is encoding for this one.

The extension does not launch a shell, but launches Python directly.

[christopherpickering]

Windows is being pretty persistent. The colorize library was also allowing windows to tweak the encoding.. so I need to do another update the djlint to ensure the output is passed to the printer as utf8.

I was able to get it to work in vscode, I'll test a bit more and see if anything needs to be pr'ed.

[christopherpickering]

I published djlint 1.1.1, which seems to fix this 🚀 No changes to djlint-vscode

.

[monosans]

I published djlint 1.1.1, which seems to fix this No changes to djlint-vscode

Amazing! Thank you very much!

I will later write in README.md that djLint version 1.1.1 or higher is required to fix this issue.

[christopherpickering]

No problem, thank you!

[monosans]

Solution - update djLint to version 1.1.1 or higher.

[Real-Gecko]

Hey, guys! We've just checked the update on two machines, one with winduz 10 another with 11. In both cases everything works flawlessly. Thanks for your effort! Really appreciated.

[christopherpickering]

Nice!