--commit-callback fails when trying to add files - Git version 2.24

[galsi]

Hi
Trying to add file to the root of the branch fails with following error:

git filter-repo --force --commit-callback "if not commit.parents: commit.file_changes.append(FileChange(b'M', 'C:\MDC\MDC.7z', $(git hash-object -w 'C:\MDC\MDC.7z'), 100644))"
Traceback (most recent call last):
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 3839, in <module>
    filter = RepoFilter(args)
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 2661, in __init__
    self._handle_arg_callbacks()
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 2763, in _handle_arg_callbacks
    handle('commit')
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 2756, in handle
    setattr(self, callback_field, make_callback(type, code_string))
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 2741, in make_callback
    exec('def callback({}, _do_not_use_this_var = None):\n'.format(argname)+
  File "<string>", line 2
    if not commit.parents: commit.file_changes.append(FileChange(b'M', 'C:\MDC\MDC.7z', 3d5fb68077a1d627a7ec3b18f335713c4262fbf0, 100644))
                                                                                         ^
SyntaxError: invalid syntax

Any advise would be great
Tried on 3 different repos

[phdru]

Discussed and fixed at https://stackoverflow.com/a/59033952/7976758. Recommend closing.

[newren]

While lack of quotes around the blob object id is the first obvious problem, I'd say there are a few more:

• Filenames passed to the FileChange constructor should be relative to the repository root. A name like C:\MDC\MDC.7z makes no sense. But it's worse because that's not what you passed...
• backslashes within strings will be interpreted by bash and/or python as control characters and would thus need to be escaped. Since you need to escape from both bash and python, that would look really ugly. Except that you don't even want backslashes because...
• Although path names on windows use backslashes, path names in git don't. filter-repo doesn't do any \ -> / conversion for you. I suspect attempting to use backslashes will yield broken results (unless fast-import does something special I'm not aware of?)
• Everything in filter-repo is supposed to be bytestrings. Thus, you'd need b'$(git hash-object ...)' rather than `'$(git hash-object ...)'.
• The final argument to FileChange is supposed to be a bytestring too rather than a bare integer. Thus b'100644' rather than 100644. Yeah, I know, my example in the comment in contrib/file-repo-demos/insert-beginning made this mistake; I'll fix it up.

[galsi]

Hi
i tried changing the command to us bytestring also add this to the path - is that correct?

git filter-repo --force --commit-callback "if not commit.parents: commit.file_changes.append(FileChange(b'M', b'/MDC.7z', b'$(git hash-object -w '/C/MDC/MDC.7z')', b'100644'))"
Parsed 1 commitsfatal: Empty path component found in input
fast-import: dumping crash report to .git/fast_import_crash_15656
Traceback (most recent call last):
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 3840, in <module>
    filter.run()
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 3777, in run
    self._parser.run(self._input, self._output)
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 1396, in run
    self._parse_commit()
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 1249, in _parse_commit
    self._commit_callback(commit, aux_info)
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 3335, in _tweak_commit
    self._record_remapping(commit, orig_parents)
  File "C:/Program Files/Git/mingw64/libexec/git-core\git-filter-repo", line 3128, in _record_remapping
    self._output.flush()
OSError: [Errno 22] Invalid argument

[newren]

This is looking better, but b'/MDC.7z' is still an absolute path (due to the leading slash). It should be the path within the repo you want the file known as. If you want it at the toplevel, then it's just b'MDC.7z'.

Does $(git hash-object -w '/C/MDC/MDC.7z') return the expected hash? Sorry, haven't used Windows in decades so I don't know how that side works.

Can you provide the output of .git/fast_import_crash_15656?

I suspect though that it's related to the "Empty path component found in input". i.e. it thinks you want the file stuck in $DIRNAME/$FILENAME (since you had one slash) where $DIRNAME is the empty string (an error) and $FILENAME is MDC.7z.

[galsi]

thanks for your great & quick support
"b'/MDC.7z' is still an absolute path" - if i need it inside a path i use /pathname/mdc.7z?

With the following command it is running!!!!
git filter-repo --force --commit-callback "if not commit.parents: ```
commit.file_changes.append(FileChange(b'M', b'MDC.7z', b'$(git hash-object -w '/C/MDC/MDC.7z')', b'100644'))"
Parsed 8937 commits

[newren]

If you wanted /C/MDC/MDC.7z to appear within your repo in some subdirectory, let's say you wanted it under sources/archives/MDC.7z, then you'd use b'sources/archives/MDC.7z' for the path. If you want it at the toplevel, then you'd use b'MDC.7z', as you did above.

Glad you've got it working. Does this answer all your questions?

[newren]

I pushed up 4de3861 to try to avoid anyone else tripping over the same problems.

[galsi]

If you wanted /C/MDC/MDC.7z to appear within your repo in some subdirectory, let's say you wanted it under sources/archives/MDC.7z, then you'd use b'sources/archives/MDC.7z' for the path. If you want it at the toplevel, then you'd use b'MDC.7z', as you did above.

Glad you've got it working. Does this answer all your questions?

Hi
thank you very much , if i would like to add several files what would be the syntax?
Can i do it in the same command?

[newren]

Yes, you can do it in the same command, but python makes it difficult to put multiple statements onto a single line so you may need some line breaks. It'd look something like:

git filter-repo --force --commit-callback "if not commit.parents:
    commit.file_changes.append(FileChange(b'M', b'MDC.7z', b'$(git hash-object -w '/C/MDC MDC.7z')', b'100644'))
    commit.file_changes.append(FileChange(b'M', b'some/other/file.txt', b'$(git hash-object -w '/C/Path/To/file.txt')', b'100644'))
    "

or

git filter-repo --force --commit-callback "if not commit.parents: commit.file_changes += [
    FileChange(b'M', b'MDC.7z', b'$(git hash-object -w '/C/MDC MDC.7z')', b'100644'), 
    FileChange(b'M', b'some/other/file.txt', b'$(git hash-object -w '/C/Path/To/file.txt')', b'100644')]"

The latter form could perhaps be put on one line, though it gets hard to read.

[newren]

I'm assuming that has answered your questions, so I'll go ahead and close out. Let me know if that's not the case.