Updating Recipe Fails When Using Git Sub-Modules

[PythooonUser]

Symfony version(s) affected

5.4.4

Description

Project Setup

We are using a project structure that leverages git sub-modules, i.e.

root
  -> project1
  -> project2 (checked-out)

The .git "folder" is actually a file thereof, pointing to the "parent" .git folder:

# cat root/project2/.git
gitdir: ../.git/modules/project2

Problem

When I now run the command to update a recipe (it doesn't matter which one it is), I'm greeted with this error:

user@computer:/root/project2$ composer recipes:update symfony/security-bundle
  Updating recipe for symfony/security-bundle...

There was an error applying the recipe update patch
Failed to create "./.git/objects/e9": mkdir(): Not a directory.

Update the recipe by re-installing the latest version with:
  composer recipes:install symfony/security-bundle --force -v

Issue: Composer tries to work with the .git folder as usual, not being aware of the concept about git sub-modules.

The provided solution here is to just nuke the existing code and "start fresh". That works of course, but could be "cleaner".

How to reproduce

I think the description says enough already, but:

• Create a git repository
• Add a sub-module with a Symfony project
• Try to update a recipe

Possible Solution

Not sure about a solution here.

We would probably need to make composer aware about the git sub-module concept and make it follow the link to the "actual" .git folder.

But the provided solution to just "nuke" the existing code is also fine, when the above cannot be achieved easily.

Additional Context

No response

[nicolas-grekas]

Would you mind working on a fix and sending a PR?

[PythooonUser]

Sure, I can try. Could you notch me into the right direction?

[nicolas-grekas]

src/Update/RecipePatcher.php might be a good start, that's where the .git folder is referenced.

[weaverryan]

I’d be happy to help also - you can also ping me on symfony slack. It seems like you’re likely correct that the code needs to find the “real” .git directory.

I’m guessing this exact error comes from here: https://github.com/symfony/flex/blob/1.x/src/Update/RecipePatcher.php#L195 - but you could get a better stack trace if you update with -vvv. In that class, you can see a lot of references to rootDir. It’s possible that this needs to change in some or all cases - but I’m doing a lot of guessing :)

[PythooonUser]

Okay, I tried creating a reproducer first. Another thing that I found out:

Install from sub-project

When you have not yet installed a package (e.g. symfony/security-bundle) in your sub-project, checkout the sub-project from your root project and then try to install it, the recipe will not be installed (without any error message).

$ symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project> composer recipes symfony/security-bundle
name             : symfony/security-bundle
version          : n/a
status           : recipe not installed
latest recipe    : https://github.com/symfony/recipes/tree/master/symfony/security-bundle/5.3

Trying to update results in a failure, because it has never been installed.

$ symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project> composer recipes:update symfony/security-bundle
Package not found inside symfony.lock. It looks like it's not installed?
Try running composer recipes:install symfony/security-bundle --force -v to re-install the recipe.

[PythooonUser]

Huh boy. It took some time to get an installable Symfony project where I could upgrade a recipe.

I now created a sub-project and a root project.

When I run $ composer recipes:update symfony/console -vvv in the sub-module I get the following output. But it does not seem to contain additional information.

$ symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project> composer recipes:update symfony/console -vvv
Reading ./composer.json (symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project\composer.json)
Loading config file AppData/Roaming/Composer/config.json
Loading config file ./composer.json (symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project\composer.json)
Checked CA file /etc/pki/tls/certs/ca-bundle.crt does not exist or it is not a file.
Checked directory /etc/pki/tls/certs/ca-bundle.crt does not exist or it is not a directory.
Checked CA file /etc/ssl/certs/ca-certificates.crt does not exist or it is not a file.
Checked directory /etc/ssl/certs/ca-certificates.crt does not exist or it is not a directory.
Checked CA file /etc/ssl/ca-bundle.pem does not exist or it is not a file.
Checked directory /etc/ssl/ca-bundle.pem does not exist or it is not a directory.
Checked CA file /usr/local/share/certs/ca-root-nss.crt does not exist or it is not a file.
Checked directory /usr/local/share/certs/ca-root-nss.crt does not exist or it is not a directory.
Checked CA file /usr/ssl/certs/ca-bundle.crt does not exist or it is not a file.
Checked directory /usr/ssl/certs/ca-bundle.crt does not exist or it is not a directory.
Checked CA file /opt/local/share/curl/curl-ca-bundle.crt does not exist or it is not a file.
Checked directory /opt/local/share/curl/curl-ca-bundle.crt does not exist or it is not a directory.
Checked CA file /usr/local/share/curl/curl-ca-bundle.crt does not exist or it is not a file.
Checked directory /usr/local/share/curl/curl-ca-bundle.crt does not exist or it is not a directory.
Checked CA file /usr/share/ssl/certs/ca-bundle.crt does not exist or it is not a file.
Checked directory /usr/share/ssl/certs/ca-bundle.crt does not exist or it is not a directory.
Checked CA file /etc/ssl/cert.pem does not exist or it is not a file.
Checked directory /etc/ssl/cert.pem does not exist or it is not a directory.
Checked CA file /usr/local/etc/ssl/cert.pem does not exist or it is not a file.
Checked directory /usr/local/etc/ssl/cert.pem does not exist or it is not a directory.
Checked CA file /usr/local/etc/openssl/cert.pem does not exist or it is not a file.
Checked directory /usr/local/etc/openssl/cert.pem does not exist or it is not a directory.
Checked CA file /usr/local/etc/openssl@1.1/cert.pem does not exist or it is not a file.
Checked directory /usr/local/etc/openssl@1.1/cert.pem does not exist or it is not a directory.
Checked CA file /etc/pki/tls/certs does not exist or it is not a file.
Checked directory /etc/pki/tls/certs does not exist or it is not a directory.
Checked CA file /etc/ssl/certs does not exist or it is not a file.
Checked directory /etc/ssl/certs does not exist or it is not a directory.
Checked CA file /etc/ssl does not exist or it is not a file.
Checked directory /etc/ssl does not exist or it is not a directory.
Checked CA file /usr/local/share/certs does not exist or it is not a file.
Checked directory /usr/local/share/certs does not exist or it is not a directory.
Checked CA file /usr/ssl/certs does not exist or it is not a file.
Checked directory /usr/ssl/certs does not exist or it is not a directory.
Checked CA file /opt/local/share/curl does not exist or it is not a file.
Checked directory /opt/local/share/curl does not exist or it is not a directory.
Checked CA file /usr/local/share/curl does not exist or it is not a file.
Checked directory /usr/local/share/curl does not exist or it is not a directory.
Checked CA file /usr/share/ssl/certs does not exist or it is not a file.
Checked directory /usr/share/ssl/certs does not exist or it is not a directory.
Checked CA file /etc/ssl does not exist or it is not a file.
Checked directory /etc/ssl does not exist or it is not a directory.
Checked CA file /usr/local/etc/ssl does not exist or it is not a file.
Checked directory /usr/local/etc/ssl does not exist or it is not a directory.
Checked CA file /usr/local/etc/openssl does not exist or it is not a file.
Checked directory /usr/local/etc/openssl does not exist or it is not a directory.
Checked CA file /usr/local/etc/openssl@1.1 does not exist or it is not a file.
Checked directory /usr/local/etc/openssl@1.1 does not exist or it is not a directory.
Checked CA file AppData\Local\Temp\opeB4F6.tmp: valid
Executing command (symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project): git branch -a --no-color --no-abbrev -v
Failed to initialize global composer: Composer could not find the config file: AppData/Roaming/Composer/composer.json

Reading symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project/vendor/composer/installed.json (symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project\vendor\composer\installed.json)
Loading plugin Symfony\Flex\Flex (from symfony/flex)
Running 2.2.6 (2022-02-04 17:00:38) with PHP 8.1.2 on Windows NT / 10.0
Executing command (.): git status --porcelain --untracked-files=no
Reading AppData/Local/Composer/repo/flex/symfony-recipes-flex-main-index.json from cache
Reading AppData/Local/Composer/repo/flex/symfony-recipes-contrib-flex-main-index.json from cache
Downloading https://raw.githubusercontent.com/symfony/recipes/flex/main/index.json
Downloading https://raw.githubusercontent.com/symfony/recipes-contrib/flex/main/index.json
[304] https://raw.githubusercontent.com/symfony/recipes/flex/main/index.json
[304] https://raw.githubusercontent.com/symfony/recipes-contrib/flex/main/index.json
Reading ./composer.lock (symfony-contributing\symfony-flex-reproducer-864-root\symfony-flex-reproducer-864-sub-project\composer.lock)
Reading AppData/Local/Composer/repo/flex/symfony-recipes-flex-main-archived-symfony.console-c6d02bdfba9da13c22157520e32a602dbee8a75c.json from cache
Downloading https://raw.githubusercontent.com/symfony/recipes/flex/main/archived/symfony.console/c6d02bdfba9da13c22157520e32a602dbee8a75c.json
[304] https://raw.githubusercontent.com/symfony/recipes/flex/main/archived/symfony.console/c6d02bdfba9da13c22157520e32a602dbee8a75c.json
Reading AppData/Local/Composer/repo/flex/symfony-recipes-flex-main-symfony.console.4.4.json from cache
Downloading https://raw.githubusercontent.com/symfony/recipes/flex/main/symfony.console.4.4.json
[304] https://raw.githubusercontent.com/symfony/recipes/flex/main/symfony.console.4.4.json
  Updating recipe for symfony/console...

Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git init
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git config commit.gpgsign false
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git config user.name "Flex Updater"
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git config user.email ""
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git add -A
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git commit -m "original files"
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git hash-object bin/console
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git add -A
Executing command (AppData\Local\Temp/_flex_recipe_update833775946620fd03c26c283.93890470): git diff --cached
There was an error applying the recipe update patch
Failed to create "./.git/objects/8f": mkdir(): No such file or directory.

Update the recipe by re-installing the latest version with:
  composer recipes:install symfony/console --force -v