Add support for .gitattributes

[martinvonz]

We'll probably want to support at least the eol attribute. I noticed while using jj in a working copy shared with git that some files appeared modified because they had LF in the commit and CRLF in the working copy.

[WonderBeat]

And also there are tools like git-crypt that use gitattributes.
jj marks all git-crypt decrypted files as modified :(

[martinvonz]

And also there are tools like git-crypt that use gitattributes. jj marks all git-crypt decrypted files as modified :(

That seems to rely on the smudge/clean filters. Perhaps we'll want to support those too. I haven't thought much about how to implement that or what consequences it would have. I also haven't yet checked when Git runs them. I specifically wonder if they're only used when checking out from the index to the working copy or if they're also used e.g. when doing operations in memory.

[xelxebar]

Spitballing a bit here, but git diffing between commits works just fine on encrypted files, so clean at least has got to be working on in-memory ops.

[martinvonz]

Speaking of that, there's an annoying special case that I don't think we're going to support: A file can have the same content in two commits but different .gitattributes, which ideally should make it show up in diffs. For example, let's say a file is committed with LF line endings, and in commit A it has a .gitattributes that says to convert those to CRLF and in commit B it doesn't have any .gitattributes. When diffing that file, you could argue that you should see the change in line endings. That would make diffing slow, however, since we can't say that a tree/file is unchanged if there are changes in .gitattributes in a parent directory. I asked the Git team at Google about this case a long time ago, and my recollection is that Git doesn't handle that case. It's of course easy to check but I haven't.

[kamalmarhubi]

For example, let's say a file is committed with LF line endings, and in commit A it has a .gitattributes that says to convert those to CRLF and in commit B it doesn't have any .gitattributes. When diffing that file, you could argue that you should see the change in line endings.

I tested this out, and I'm pretty sure git doesn't handle it.

Click to see transcript of testing it out

Create a test repo

❯ git init test-eol-attr
Initialized empty Git repository in /private/tmp/test-eol-attr/.git/
❯ cd test-eol-attr/
❯ git commit --allow-empty -m'Root commit (empty)'
[main (root-commit) 21742c4] Root commit (empty)

Now add a file with LF line endings:

❯ seq 1 5 > test
❯ git add -p
No changes.
❯ git add test
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a                    |1.2.3.4.5.|
0000000a
❯ git commit -m'Add 1-10 with LF line endings (\x0a)'
[main 8f2afec] Add 1-10 with LF line endings (\x0a)
 1 file changed, 5 insertions(+)
 create mode 100644 test

Now set eol=crlf in .gitattributes

❯ echo 'test text eol=crlf' >> .gitattributes
❯ git checkout test
Updated 0 paths from the index
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a                    |1.2.3.4.5.|
0000000a
❯ git add .gitattributes
❯ git commit -m'Add .gitattributes setting eol to CRLF (\x0d\x0a)'
[main fe06662] Add .gitattributes setting eol to CRLF (\x0d\x0a)
 1 file changed, 1 insertion(+)
 create mode 100644 .gitattributes

At this point, git see no differences, and the file in the working copy still has LF line endings:

❯ git checkout -p
No changes.
❯ git checkout -p test
No changes.
❯ git checkout test
Updated 0 paths from the index
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a                    |1.2.3.4.5.|
0000000a

Adding two more lines still leaves the file with LF line endings:

❯ seq 6 7 >> test
❯ git add -p
warning: in the working copy of 'test', LF will be replaced by CRLF the next time Git touches it
warning: in the working copy of 'test', LF will be replaced by CRLF the next time Git touches it
diff --git a/test b/test
index 8a1218a..06e567b 100644
--- a/test
+++ b/test
@@ -3,3 +3,5 @@
 3
 4
 5
+6
+7
(1/1) Stage this hunk [y,n,q,a,d,e,?]? y
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a 36 0a 37 0a        |1.2.3.4.5.6.7.|
0000000e
❯ git commit -m'Add two more lines'
[main bf9e4b4] Add two more lines
 1 file changed, 2 insertions(+)
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a 36 0a 37 0a        |1.2.3.4.5.6.7.|
0000000e

Even trying to checkout the file leaves it unchanged:

❯ git checkout test
Updated 0 paths from the index
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a 36 0a 37 0a        |1.2.3.4.5.6.7.|
0000000e
❯ git checkout -f test
Updated 0 paths from the index
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a 36 0a 37 0a        |1.2.3.4.5.6.7.|
0000000e

It's only when I take an action where git itself modifies the working copy file that it changes the line endings:

❯ git checkout -p HEAD~
diff --git b/test a/test
index 06e567b..8a1218a 100644
--- b/test
+++ a/test
@@ -3,5 +3,3 @@
 3
 4
 5
-6
-7
(1/1) Apply this hunk to index and worktree [y,n,q,a,d,e,?]? y

❯ hexdump -C test
00000000  31 0d 0a 32 0d 0a 33 0d  0a 34 0d 0a 35 0d 0a     |1..2..3..4..5..|
0000000f

Finally, git diff doesn't show a change in line endings in the diff, even against the working copy after running dos2unix (it does print a warning though):

❯ dos2unix test
dos2unix: converting file test to Unix format...
❯ git diff
warning: in the working copy of 'test', LF will be replaced by CRLF the next time Git touches it
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a 36 0a 37 0a        |1.2.3.4.5.6.7.|
0000000e
❯ git checkout -p test
warning: in the working copy of 'test', LF will be replaced by CRLF the next time Git touches it
No changes.
❯ hexdump -C test
00000000  31 0a 32 0a 33 0a 34 0a  35 0a 36 0a 37 0a        |1.2.3.4.5.6.7.|
0000000e
❯ git checkout test
Updated 1 path from the index
❯ hexdump -C test
00000000  31 0d 0a 32 0d 0a 33 0d  0a 34 0d 0a 35 0d 0a 36  |1..2..3..4..5..6|
00000010  0d 0a 37 0d 0a                                    |..7..|
00000015

[martinvonz]

Thanks for checking!

[omasanori]

Indeed the lack of eol=crlf support prevents me from using jj with Windows-specific files like .cmd files for AppVayor CI...

[martinvonz]

Indeed the lack of eol=crlf support prevents me from using jj with Windows-specific files like .cmd files for AppVayor CI...

Just to make sure you know, you should still be able to use jj in a non-colocated workspace, i.e. if you do jj init --git-repo=<path to git repo> outside of the git repo's working copy.