Error when creating and merging changelog entries with projects

[Valgard]

Description
I encountered issues when using changie to create and merge changelog entries:

1. Creating a Changelog Entry:

   • When I attempt to create a new changelog entry with the command changie new, I receive the following error message: Error: project missing but required.
   • This error occurs despite having projects defined in the configuration file.
   • However, if I specify the project using the -j parameter, I can successfully create the changelog entry without any issues.

2. Merging Changelog Entries:

   • When I attempt to merge changelog entries using the command changie merge, I encounter the following error message: Error: reading files from '.changes/backend': open .changes/backend: no such file or directory.

Reproduction Steps

1. Run changie new without specifying a project using -j.
2. Observe the error message regarding the missing project.
3. Run changie new -j <project-name> to create an entry successfully.
4. Attempt to merge entries with changie merge.
5. Observe the error regarding the missing directory.

Expected behavior

• The changie new command should create a changelog entry without requiring the -j parameter
• The changie merge command should successfully merge changelog entries without directory errors.

Additional context

• changie version v1.21.0
• Environment: macOS 15 / Homebrew
• .changie.yml

  changesDir: .changes
  unreleasedDir: unreleased
  headerPath: header.tpl.md
  changelogPath: CHANGELOG.md
  versionExt: md
  versionFormat: '## {{.Version}} - {{.Time.Format "2006-01-02"}}'
  kindFormat: '### {{.Kind}}'
  changeFormat: '* {{.Body}}'
  envPrefix: CHANGIE_
  projectsVersionSeparator: ': '
  projects:
    - label: Backend
      key: backend
      changelog: apps/backend/CHANGELOG.md
    - label: Frontend
      key: frontend
      changelog: apps/frontend/CHANGELOG.md
    - label: Common
      key: common
      changelog: packages/common/CHANGELOG.md
  kinds:
    - label: Added
      auto: minor
    - label: Changed
      auto: patch
    - label: Deprecated
      auto: minor
    - label: Removed
      auto: major
    - label: Fixed
      auto: patch
    - label: Security
      auto: patch
  newlines:
    afterChangelogHeader: 1
    beforeChangelogVersion: 1
    endOfVersion: 1

[miniscruff]

Hmm, your config seems to be working fine for me locally. I wonder what could be causing your problem.

Your issue with merge is likely cause you did not run changie batch first, so there are no files to merge together. I am not sure its giving the appropriate error all things considered, so there could be an issue with trying to merge if you don't have the proper project folder structure pre-created, lets focus on your first issue first though.

[Valgard]

Hmm, I've double-checked everything again and it's still not working for me. I tried it again from scratch in an empty directory:

1. mkdir -p changie-test && cd changie-test
2. changie init
3. Modified .changie.yaml as mentioned above
4. changie new
   • Selected a project name
   • Results in error Error: project missing but required
5. changie new -j backend
   • Selected kind
   • Entered body
   • Entry is created
6. changie merge
   • Results in error Error: reading files from '.changes/backend': open .changes/backend: no such file or directory
7. changie merge -u "## Unreleased"
   • Results in error Error: reading files from '.changes/backend': open .changes/backend: no such file or directory
8. The resulting directory structure (tree -a --dirsfirst)

   .
   ├── .changes
   │   ├── unreleased
   │   │   ├── .gitkeep
   │   │   └── backend-Added-20250207-142221.yaml
   │   └── header.tpl.md
   ├── apps
   │   ├── backend
   │   │   └── CHANGELOG.md
   │   └── frontend
   ├── packages
   │   └── common
   ├── .changie.yaml
   └── CHANGELOG.md  
   

Additional information (if it helps):

• changie installed with brew install changie
• using ZSH with OhMyZSH!

[miniscruff]

Hmm, I've double-checked everything again and it's still not working for me. I tried it again from scratch in an empty directory:

1. mkdir -p changie-test && cd changie-test
2. changie init
3. Modified .changie.yaml as mentioned above
4. changie new
   • Selected a project name
   • Results in error Error: project missing but required
5. changie new -j backend
   • Selected kind
   • Entered body
   • Entry is created
6. changie merge
   • Results in error Error: reading files from '.changes/backend': open .changes/backend: no such file or directory
7. changie merge -u "## Unreleased"
   • Results in error Error: reading files from '.changes/backend': open .changes/backend: no such file or directory
8. The resulting directory structure (tree -a --dirsfirst)

   .
   ├── .changes
   │   ├── unreleased
   │   │   ├── .gitkeep
   │   │   └── backend-Added-20250207-142221.yaml
   │   └── header.tpl.md
   ├── apps
   │   ├── backend
   │   │   └── CHANGELOG.md
   │   └── frontend
   ├── packages
   │   └── common
   ├── .changie.yaml
   └── CHANGELOG.md  
   

Additional information (if it helps):

• changie installed with brew install changie
• using ZSH with OhMyZSH!

Between steps 5 and 6 you need to run the changie batch command to build the release notes before merge combines the release notes into the changelog.

[Valgard]

• changie batch patch gives the same error as changie new: Error: project missing but required
• changie batch patch -j backend runs without any issues
• changie merge then gives Error: reading files from '.changes/frontend': open .changes/frontend: no such file or directory
• Only when I have run changie batch at least once for all projects does changie merge work
• But why do I need to execute changie batch?
  • I can always run changie merge without projects without any errors
  • With changie merge -u, I explicitly want to include unversioned changes in the changelog
• I believe the error Error: reading files from occurs because the directories for the projects under .changes do not yet exist
  • With changie batch -j, they are created, and changie merge can be executed for the project without errors
  • A check to see if the project directory exists should fix the error
• However, the error Error: project missing but required with changie new or changie batch remains unaffected by this

[miniscruff]

• changie batch patch gives the same error as changie new: Error: project missing but required
• changie batch patch -j backend runs without any issues

Yes, if you are in the mono repo style project setup you have to tell us which project you were batching. Maybe this could be documented better.

• changie merge then gives Error: reading files from '.changes/frontend': open .changes/frontend: no such file or directory
• Only when I have run changie batch at least once for all projects does changie merge work
• But why do I need to execute changie batch?

Yes, you need to batch at least once to generate your first release in order to merge your changelog file. Batch combines the unreleased changes into a new version, without it you have only unreleased fragments. Think of the yaml files as only one change, and the release notes as all the changes for a version, and the changelog as well the versions.

• I can always run changie merge without projects without any errors

Yes, it just merges the version files together, but without version files it does nothing.

• With changie merge -u, I explicitly want to include unversioned changes in the changelog
• I believe the error Error: reading files from occurs because the directories for the projects under .changes do not yet exist

Hmm, I think the -u flag may not work for the initial release, that could be an issue. Can't say I have tried this before.

• With changie batch -j, they are created, and changie merge can be executed for the project without errors

Yes, this is the expected flow for mono repos. The new and merge commands work as normal but for batch you do have to specify the project. Does this flow not work for you?

• A check to see if the project directory exists should fix the error

I do recall considering this, however, I went with this error as I felt it you ran merge and we didn't generate any file, that should tell the user we failed rather then silently doing nothing.

• However, the error Error: project missing but required with changie new or changie batch remains unaffected by this

Right, you do still need to tell us which project your change belongs to and which project you are batching. I do recall someone asking to batch all projects but due to the way other flags work it was deemed unsafe of an option. But you can use yq pretty easily if you have the minimum scenario.

It sounds like the expected scenario does work, maybe you are expecting changie to support an alternative scenario we do not.

[Valgard]

• changie batch patch gives the same error as changie new: Error: project missing but required
• changie batch patch -j backend runs without any issues

Yes, if you are in the mono repo style project setup you have to tell us which project you were batching. Maybe this could be documented better.

With changie batch without the -j parameter, I expected a selection like with changie new. But I can also live with having to explicitly specify the project in the command prompt.

• changie merge then gives Error: reading files from '.changes/frontend': open .changes/frontend: no such file or directory
• Only when I have run changie batch at least once for all projects does changie merge work
• But why do I need to execute changie batch?

Yes, you need to batch at least once to generate your first release in order to merge your changelog file. Batch combines the unreleased changes into a new version, without it you have only unreleased fragments. Think of the yaml files as only one change, and the release notes as all the changes for a version, and the changelog as well the versions.

• I can always run changie merge without projects without any errors

Yes, it just merges the version files together, but without version files it does nothing.

I'm afraid that's not quite accurate.

• changie merge without projects simply recreates the changelog file with the header and the existing versions. If there are no versions, it is just the header. I would expect the same behavior with changie merge with projects.
• changie merge -u without projects simply recreates the changelog file with the header, the unversioned changes, and the existing versions. If there are no versions, it is the header with the unversioned changes. I would expect the same behavior with changie merge with projects.

• With changie merge -u, I explicitly want to include unversioned changes in the changelog
• I believe the error Error: reading files from occurs because the directories for the projects under .changes do not yet exist

Hmm, I think the -u flag may not work for the initial release, that could be an issue. Can't say I have tried this before.

• With changie batch -j, they are created, and changie merge can be executed for the project without errors

Yes, this is the expected flow for mono repos. The new and merge commands work as normal but for batch you do have to specify the project. Does this flow not work for you?

• A check to see if the project directory exists should fix the error

I do recall considering this, however, I went with this error as I felt it you ran merge and we didn't generate any file, that should tell the user we failed rather then silently doing nothing.

• If I manually create the directories for the projects under .changes and then execute changie merge or changie merge -u, the behavior is as expected. This is also applicable to the initial release.
• So a check for the project directory and assuming no versions exist if the directory is missing would probably solve the issue. This is the same behaviour as when no projects are defined.

• However, the error Error: project missing but required with changie new or changie batch remains unaffected by this

Right, you do still need to tell us which project your change belongs to and which project you are batching. I do recall someone asking to batch all projects but due to the way other flags work it was deemed unsafe of an option. But you can use yq pretty easily if you have the minimum scenario.

It sounds like the expected scenario does work, maybe you are expecting changie to support an alternative scenario we do not.

Okay, the error Error: project missing but required with changie new is my mistake. I just realized after analyzing the code that you need to select a project with the spacebar. Until now, I had simply "selected" a project with the arrow keys and then pressed the enter key. 🤦🏻

[miniscruff]

With changie batch without the -j parameter, I expected a selection like with changie new. But I can also live with having to explicitly specify the project in the command prompt.

Only changie new uses prompts, all other commands are meant to be more admin/automation commands and are not given the same level of flexibility. This was a design decision a while ago, could be changed but more or less was the thinking.

I'm afraid that's not quite accurate.

* `changie merge` without projects simply recreates the changelog file with the header and the existing versions. If there are no versions, it is just the header. I would expect the same behavior with `changie merge` with projects.

* `changie merge -u` without projects simply recreates the changelog file with the header, the unversioned changes, and the existing versions. If there are no versions, it is the header with the unversioned changes. I would expect the same behavior with `changie merge` with projects.

Ok, yes, I see, I believe you are correct and I agree. Having changie merge fail with an error with projects when the non-projects version is basically just a no-op feels weird.

Number 2 is definitely the core issue, and I can fix that real fast.

Okay, the error Error: project missing but required with changie new is my mistake. I just realized after analyzing the code that you need to select a project with the spacebar. Until now, I had simply "selected" a project with the arrow keys and then pressed the enter key. 🤦🏻

ok nice, that sounds right, number 1 in your original comment seems to be resolved now.

[miniscruff]

@Valgard thanks to your report I also found out that changie next fails in a similar way for projects, so also fixing that while I am here.