Windows: Coala replaces some special caracters to unkown character

[tims0]

When I run coala, it replaces some special characters like this:

When I want to start it a second time, I get these error messages, because it can't handle with the ?-Character:

[Makman2]

Seems to be an internal encoding issue somewhere. cmd uses a special encoding format (something with cp and a number) that is not really compatible to unicode that would explain this more or less^^

If you need to work with coala, try to use Powershell, I believe it uses unicode as default^^

I'll investigate this, but I have not that much time now so this could take a while^^

[sils]

This is not a usability issue, it's a bug in coala. We shouldn't destroy his data like this. @Makman2 are we even sure this doesn't happen on linux?

Scheduling importance to high since this is connected to actual data loss.

@TimSo94 are you free to share the original file where this occurred with us?

[tims0]

Yes, it happend in 2 files.
In the functions.py in line 75 and line 78 and in the models.py in line 108 and 109.

pys.zip

[sils]

@TimSo94 we'll try to look into this soon. I hope we can get to fix this till next week.

[Makman2]

@TimSo94 could you also provide the .coafile? First reproduction on fedora 23 didn't yield an error, my assumed coafile was:

[Default]
bears = PEP8Bear
files = *.py

[sils]

Mischa: you could just introduce an error and see if this occurs for now.
On 24 Jan 2016 00:51, "Mischa Krüger" notifications@github.com wrote:

@TimSo94 https://github.com/TimSo94 could you also provide the .coafile?
First reproduction on fedora 23 didn't yield an error, my assumed coafile
was:

[Default]
bears = PEP8Bear

—
Reply to this email directly or view it on GitHub
#1238 (comment)
.

[Makman2]

Introducing an error (in this case trailing whitespaces) did nothing.
(okay on fedora it marked the completely file as deleted and added, but I assume this is because of CRLF file endings that get completely rewritten when a patch is applied due to pythons universal-newlines conversion)

[Makman2]

Currently I have problems reproducing this on windows since coala doesn't take up the files there...

[Makman2]

Confirmed on Windows! Since this error only happened after apply, I believe @TimSo94 you have a setting like default_actions = PEP8Bear: AutoApplyAction so coala automatically fixes errors, that's why it replaces directly with these weird signs. After that it can't read them again.

[Makman2]

Also this doesn't come from the PEP8Bear, the according diff only showed me the error I introduced^^
Looks like the write-back is responsible for this...

[tims0]

I only have the "Standard" Settings, here:
[Default]
bears = PEP8Bear
files = */.py

[sils]

@Makman2 he applied it manually which does the same thing I guess, makes no difference

[sils]

@AbdealiJK can you look into this one? We've had another occurrence too:

One thing I noticed is that all the UTF-8 characters have been replaced in our JSON files -- I'd like for them to stay as-is.

duckduckgo/zeroclickinfo-goodies#2289 (comment)

[sils]

That one is rather #1527 .

[AbdealiLoKo]

I do not have a windows machine to test - and hence it'd be difficult for me to do this

[sils]

maybe this can be solved with #1541 (comment) , @AbdealiJK can you try reproducing this and solving it?

[AbdealiLoKo]

I am able to reproduce the. I opened the files in a hex editor and noticed 4 chars were removed when coala wrote back to it.

The symbol ° is B0 in the file after coala ran on it.
The same thing in the one before coala ran on it is @#° - 40 23 C2 B0

ALSO - This happens in git bash too. (Im not sure if it's windows specific anymore)

[AbdealiLoKo]

Changing the writing of the file to with open(filename, mode='w', encoding='utf-8') as file: in coalib.results.result_actions.ApplyPatchAction (line 50) fixes this

[Makman2]

@TimSo94 is the bug solved for you?

[tims0]

Yes, now it works fine. Thanks! :)