Make Emacs use the encoding and line endings configured at the Git level

[kostafey]

Since I have to edit files in cp1251 encoding, I'd like to versioning them.

Magit diff buffer shows cp1251-encoded text wrong.

To fix it in bare git via git diff I can use:

[core]
    pager = iconv -f windows-1251 -t utf-8 | less

in my .git/config.
So, is it possible to solve this problem in magit diff buffers?

[tarsius]

Try whether removing --no-pager from magit-git-standard-options and possibly setting pager.diff either true or what you use for core.pager helps.

[kostafey]

When I set

(setq magit-git-standard-options
      '("-c"
        "core.preloadindex=true"
        "pager.diff=true"))

magit can't find git repository for this file (without "pager.diff=true" ok, but incorrect cp1251-encoded symbols displaying).

[npostavs]

I think you need one -c per setting:

(setq magit-git-standard-options
      '("-c" "core.preloadindex=true"
        "-c" "pager.diff=true"))

[kostafey]

Ok, now magit works well thank you, but for both
"pager.diff=true" and "pager.diff=iconv -f windows-1251 -t utf-8 | less" cp1251-encoded symbols remain mangled.

[tarsius]

I am afraid it will be a while until I can do something about this. Currently Magit just encodes all of its buffers in unicode and it would be hard to change that. Especially when considering that the same may contain snippets from various files, not all of which necessarily use the same encoding.

I recommend you instead tell Git to checkout as unicode, if that is possible. Something similar can be done for line endings, I don't know whether the equivalent is possible for the coding system. If it is support then I would expect that also to affect diff output, not just checked out files.

[tarsius]

... if that is feasible.

[tarsius]

@kostafey could you please point me to a public repository that I can use to experiment.

[tarsius]

The output of one git diff may contain output from files with various encodings. So while it might be possible to figure out the encodings of files before calling git diff, it would not be possible to tell Emacs what the encoding of the process output is - "the" encoding might be several different encodings.

So this has to be done at the git level. I believe the pager.diff does not work because Magit doesn't use a pager even when pager.diff=true. But this should work (you might have to recreate the working directory contents before it starts working):

• .git/config

[filter "windows-1251"]
    clean  = iconv -f utf-8 -t windows-1251
    smudge = iconv -f windows-1251 -t utf-8

• .gitattributes

* filter=windows-1251

Of course you might have to be more precise than just * (all files). Read all about it in the gitattributes(5) manpage.

I cannot actually verify that this works until I get my hands onto a real-word repository containing files using such encodings.

[kostafey]

I recommend you instead tell Git to checkout as unicode, if that is possible.

That is the main problem, because of my-very-strange compiler restrictions, I have to use cp1251.

could you please point me to a public repository that I can use to experiment.

https://github.com/kostafey/test_cp1251

But this should work...

This doesn't solve the problem for me.

[tarsius]

This doesn't solve the problem for me.

Does it not work at all, or does it work but you cannot use it because of the compiler restriction?

Edit: Just tried myself. Doesn't seem to work at all...

[kostafey]

It seems, I'm doing something wrong. Should I change the file encodings manually?

[tarsius]

It seems, I'm doing something wrong.

What are you doing wrong?

This is what I get:

$ file file.txt 
file.txt: UTF-8 Unicode text, with CRLF line terminators
$ git diff     
diff --git a/file.txt b/file.txt
index b8f917d..497db3a 100644
--- a/file.txt
+++ b/file.txt
@@ -1,2 +1,2 @@
-<F2><E5><EA><F1><F2> <F2><E5><EA><F1><F2> <F2><E5><EA><F1><F2>
+<F2><E5><EA><F1><F2> <F2><E5><EA><F1><F2> <F2><E5><EA><F1><F2>foo^M
 text text text
$ git -c "pager.diff=iconv -f windows-1251 -t utf-8" diff       
diff --git a/file.txt b/file.txt
index b8f917d..497db3a 100644
--- a/file.txt
+++ b/file.txt
@@ -1,2 +1,2 @@
-текст текст текст
+текст текст текстfoo
 text text text

After adding the filter definition to .git/config and recreating the files in the working directory.

I also found msysgit/msysgit#113, which I believe means that this approach is supposed to work.

[kostafey]

I've got the same result. But it's a git console. And what about magit?

[tarsius]

Same in Magit.

I suggest you post the filter configuration above on (e.g.) Stackoverflow. Maybe someone there can help you figure out why it doesn't work. Or maybe this isn't the right approach at all. I just don't know.

The issue isn't really specific to Magit so I would not mention that. By the way the core.pager=iconv ... thing seems like a hack to me, but you might still want to mention that this works.