Question for commit filtering

[ngeor]

Hello, we're evaluating using git-cliff for our monorepo, in order to automate the generation of the CHANGELOG. The tool looks really powerful!

We have a couple of questions:

• is it possible to ignore commits, if they all concern files under a common path? To explain the use case, our monorepo holds both libraries and apps. The apps live under the same folder, "apps". The changelog is only supposed to contain the changes of the libraries, which live in multiple folders. So if a commit only contains changes against paths under "apps/*", we'd like to filter them out if possible.
• is it possible to only process merge commits? We always work with merge requests and this ends up in the default branch as either one squashed commit or multiple commits + merge commit. I believe it is possible in Git to determine if a commit belongs to a merge commit by checking its parents (I don't remember it 100%). Can git-cliff identify and exclude commits that are referenced by a merge commit? Otherwise we'll end up with duplicate messages in the changelog.

[orhun]

Hello, we're evaluating using git-cliff for our monorepo, in order to automate the generation of the CHANGELOG. The tool looks really powerful!

Hello! Thank you 🐻

• is it possible to ignore commits, if they all concern files under a common path? To explain the use case, our monorepo holds both libraries and apps. The apps live under the same folder, "apps". The changelog is only supposed to contain the changes of the libraries, which live in multiple folders. So if a commit only contains changes against paths under "apps/*", we'd like to filter them out if possible.

It is not currently possible. However, you specify the --commit-path argument for using git-cliff inclusively:

$ git cliff --commit-path libs/

But now that I think of it, I can probably modify this argument to take a regular expression as a value so you can include/exclude any path. e.g.

$ git cliff --commit-path "^(?!apps\/*)(.+)$"

How does that sound? If it's applicable, I can work on implementing it.

is it possible to only process merge commits? We always work with merge requests and this ends up in the default branch as either one squashed commit or multiple commits + merge commit. I believe it is possible in Git to determine if a commit belongs to a merge commit by checking its parents (I don't remember it 100%). Can git-cliff identify and exclude commits that are referenced by a merge commit? Otherwise we'll end up with duplicate messages in the changelog.

I don't think it is currently possible to differentiate the merge commits from other commits if the commit message is the same. Otherwise I'd say it can easily be done using commit_parsers.

Can you post an example git history with commit messages along with the merge commits to make things a bit clear here?

[ngeor]

Unfortunately we also care for file changes at the root of the repo which can't be relocated under the libs folder. The regex idea would work I think. Another idea could be to support an additional flag --exclude-commit-path. Or maybe allow multiple usages of these flags --commit-path X --commit-path Y --exclude-commit-path Y/node_modules whatever, just thinking out loud :-) Whatever makes more sense to you.

For the merge commits, I'm not a git guru, but I think we need to check how many parents the commit has. Merge commits have two parents or so https://git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---merges https://stackoverflow.com/questions/9059335/how-can-i-get-the-parents-of-a-merge-commit-in-git

I went through our git history and I see this mentioned in the metadata, if I do git log in our repo I see:

commit 7d5d9fb999051f8ca440b3620f5fd7aebd5d8e3c (HEAD -> master, origin/master, origin/HEAD)
Merge: 1e40c1939 7d3d983f0
Author: John Doe
Date:   Wed Nov 24 14:59:41 2021 +0100

    Merge branch 'docs' into 'master'
    
    Some commit message
    
    See merge request blah!2960

commit 1e40c193992be7a4d8d80a6ebfd3d783e34a126b
Merge: a515e6fec a9c9821e9
Author: John Doe
Date:   Wed Nov 24 14:35:57 2021 +0100

    Merge branch 'swagger' into 'master'
    
    enable swagger.
    
    See merge request blah!2958

commit 7d3d983f01e952e21a7938f6001afc198e93c830
Author: John Doe
Date:   Wed Nov 24 14:28:59 2021 +0100

    docs

Notice that the first two commits are merge commits and they have a "Merge" field in their metadata. These are two different merge requests that got merged into master branch. The last commit in the example is referenced by the first MR, so one of them should be excluded to avoid duplication.

As I'm typing these lines, I'm thinking we should probably be excluding the merge commits, because they have the extra text in the commit message "Merge branch 'blah' into 'master'" which violates conventional commit message style...

So maybe it's possible to do this by setting conventional commits to true (which is something we'd like to enforce) and add a commit parser like

{ message: "^Merge '[^']+' into 'master'", skip: true }

?

[ngeor]

Edit: please ignore new question, found an example https://github.com/orhun/git-cliff/blob/main/examples/scoped.toml seconds after I posted the question 🤦

@orhun can I ask you one extra question? I was trying to utilize the conventional commit message scope in the commit parsers but it doesn't seem to be possible i.e.

commit_parsers = [
    { message = "^feat\\((.+?)\\)", group = "Features of $1"},
    { message = "^feat", group = "Features"},

Since we have a monorepo of multiple libraries, I was hoping people could have commit messages like "feat(my-cool-library): Adding my cool feature" and I'll end up with a dedicated group in the changelog for "my-cool-library".

[orhun]

Hello again! Sorry for the late reply :/

Unfortunately we also care for file changes at the root of the repo which can't be relocated under the libs folder. The regex idea would work I think. Another idea could be to support an additional flag --exclude-commit-path. Or maybe allow multiple usages of these flags --commit-path X --commit-path Y --exclude-commit-path Y/node_modules whatever, just thinking out loud :-) Whatever makes more sense to you.

I implemented the following arguments:

--include-path <PATTERN>...   Sets the path to include related commits [env: INCLUDE_PATH=]
--exclude-path <PATTERN>...   Sets the path to exclude related commits [env: EXCLUDE_PATH=]

You can use them multiple times or in conjunction. They also support glob patterns:

git cliff --include-path "**/*.toml" --include-path "*.md"
git cliff --exclude-path ".github/*"

I hope that will be sufficient for your use case 🙂

P.S. I renamed the old --commit-path argument to --include-path.

For the merge commits, I'm not a git guru, but I think we need to check how many parents the commit has. Merge commits have two parents or so git-scm.com/docs/git-rev-list#Documentation/git-rev-list.txt---merges stackoverflow.com/questions/9059335/how-can-i-get-the-parents-of-a-merge-commit-in-git

I went through our git history and I see this mentioned in the metadata, if I do git log in our repo I see:

commit 7d5d9fb999051f8ca440b3620f5fd7aebd5d8e3c (HEAD -> master, origin/master, origin/HEAD)
Merge: 1e40c1939 7d3d983f0
Author: John Doe
Date:   Wed Nov 24 14:59:41 2021 +0100

    Merge branch 'docs' into 'master'
    
    Some commit message
    
    See merge request blah!2960

commit 1e40c193992be7a4d8d80a6ebfd3d783e34a126b
Merge: a515e6fec a9c9821e9
Author: John Doe
Date:   Wed Nov 24 14:35:57 2021 +0100

    Merge branch 'swagger' into 'master'
    
    enable swagger.
    
    See merge request blah!2958

commit 7d3d983f01e952e21a7938f6001afc198e93c830
Author: John Doe
Date:   Wed Nov 24 14:28:59 2021 +0100

    docs

Notice that the first two commits are merge commits and they have a "Merge" field in their metadata. These are two different merge requests that got merged into master branch. The last commit in the example is referenced by the first MR, so one of them should be excluded to avoid duplication.

As I'm typing these lines, I'm thinking we should probably be excluding the merge commits, because they have the extra text in the commit message "Merge branch 'blah' into 'master'" which violates conventional commit message style...

So maybe it's possible to do this by setting conventional commits to true (which is something we'd like to enforce) and add a commit parser like

{ message: "^Merge '[^']+' into 'master'", skip: true }

?

Edit: please ignore new question, found an example main/examples/scoped.toml seconds after I posted the question facepalm

Of course, you can skip the merge commits like that. Other than that, I'm not really sure how to "remove" duplicate commit messages. Skipping seems like a good solution to me. However, let me know if you need anything else regarding this.

@orhun can I ask you one extra question? I was trying to utilize the conventional commit message scope in the commit parsers but it doesn't seem to be possible i.e.

commit_parsers = [
    { message = "^feat\\((.+?)\\)", group = "Features of $1"},
    { message = "^feat", group = "Features"},

Since we have a monorepo of multiple libraries, I was hoping people could have commit messages like "feat(my-cool-library): Adding my cool feature" and I'll end up with a dedicated group in the changelog for "my-cool-library".

Ah, good point. Parser regex'es does not support grouping yet. But instead you can achieve the same thing by using a template like this:

{% for group, commits in commits | group_by(attribute="scope") %}
    ### {{ group | upper_first }}
    {% for commit in commits %}
        - ({{ commit.group | lower }}) {{ commit.message | upper_first }}\
    {% endfor %}
{% endfor %}

So if you commit something like feat(my-cool-library): Adding my cool feature, it will show up in the changelog like this:

### My-cool-library

- (feat) Adding my cool feature

About the regex grouping, feel free to submit a separate issue for tracking and I'm more than happy to work on it 🐻

[ngeor]

Thank you very much @orhun ! Let's close this issue whenever you like, either now or when you make a new release that has the new features --include-path and --exclude-path. Thanks for implementing it so quickly!

[orhun]

0.5.0 is out now. Closing! ⛱️