Integration tests failing during Magento setup

[sergeynezbritskiy]

Preconditions (*)

1. Fresh Magento 2.4.3 installation
2. PHP 7.3 or PHP 7.4
3. Composer version 2.1.5 (also tried with the highest composer 1 version
4. no RabbitMQ

Steps to reproduce (*)

1. Basically I follow the steps in this doc https://devdocs.magento.com/guides/v2.4/test/integration/integration_test_execution.html
2. composer create-project --repository-url=https://repo.magento.com/ magento/project-community-edition
3. bin/magento module:enable --all
4. update the configuration in dev/tests/integration/etc/install-config-mysql.php` (no RabbitMQ configuration)
5. mysql -e "DROP DATABASE IF EXISTS magento_integration_tests"
6. mysql -e "CREATE DATABASE magento_integration_tests"
7. cd dev/tests/integration && php ../../../vendor/bin/phpunit

Expected result (*)

1. Magento test instance should be configured in dev/tests/integration/tmp/sandbox-{hash} dir
2. Tests defined in dev/tests/integration/phpunit.xml.dist should start running

Actual result (*)

1. When running magento installation command, this command fails during the step "Disabling Maintenance Mode" with an error

In ResourceConnection.php line 148:

  [DomainException]
  Connection "default" is not defined

Log file is attached below

---

Please provide Severity assessment for the Issue as Reporter. This information will help during Confirmation and Issue triage processes.

• Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.
  log.txt

[m2-assistant]

Hi @sergeynezbritskiy. Thank you for your report.
To help us process this issue please make sure that you provided the following information:

• Summary of the issue
• Information on your environment
• Steps to reproduce
• Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.4-develop instance - upcoming 2.4.x release

For more details, please, review the Magento Contributor Assistant documentation.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

⚠️ According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting.

🕙 You can find the schedule on the Magento Community Calendar page.

📞 The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, please join the Community Contributions Triage session to discuss the appropriate ticket.

🎥 You can find the recording of the previous Community Contributions Triage on the Magento Youtube Channel

✏️ Feel free to post questions/proposals/feedback related to the Community Contributions Triage process to the corresponding Slack Channel

[sergeynezbritskiy]

Additional information regarding this issue:

1. The issue is not reproducible on current 2.4-develop branch
2. If I run integration tests second time and TESTS_CLEANUP config in phpunit.xml is set to disabled, the tests are running fine, so currently as a workaround I wrapped this command in try/catch in my Jenkinsfiles and run tests second time in catch

script {
    try {
        sh 'cd ${TMP_DIR}/dev/tests/integration && php ../../../vendor/bin/phpunit'
    } catch (err) {
        sh 'cd ${TMP_DIR}/dev/tests/integration && php ../../../vendor/bin/phpunit'
    }
}

[sergeynezbritskiy]

During the investigation I've found few issues:

1. Some objects, which should be cached in object manager and shared btw objects, are not actually shared and there are multiple instances of the same class:

• \Magento\Framework\App\ResourceConnection
• \Magento\Framework\App\DeploymentConfig

When object manager first creates ResourceCollection object, it has a dependency DeploymentConfig $deploymentConfig with

$data = [
    'cache_types' => [
        'compiled_config' => 1
    ]
]

And this object is cached, but then I see that another instances of ResourceConnection is created with another instance of deploymentConfig as its dependency. The data of this new deploymentConfig get updated multiple times during installation, but when we go to the step "Disable maintenance mode" I see that at some place there is a call to the old object of resource connection with an old instance of deployment config. And when we try to get db connection, there is no info about db connection in the old version of deployment config.
https://github.com/magento/magento2/blob/2.4-develop/lib/internal/Magento/Framework/App/ResourceConnection.php#L141

2. Second issue is that deployment config at some point gets cached inside the instance of DeploymentConfig, and every time we need reset it manually. so even if it doesn't return anything from cached $this->data, we presume that this configuration doesn't exist at all

I see 3 possible solutions:

1. if we can't get db configuration from deployment config, then we need to reset data and try to get db configuration again. This is mostly a workaround for a single case, so doesn't solve the issue.
2. ensure that shared objects are indeed shared all around the app. Currently I wasn't able to figure out why is it happening and how to solve is. Probably this should even work as it works now, so don't think that it would be a good idea to debug this.
3. extend the functionality of \Magento\Framework\App\DeploymentConfig. Every time when the requested configuration was not found in $this->data or $this->flattenedData, we reload the configuration.

I think that the third solution is preferable in this case, but think that someone from Adobe team need to approve this solution, no ?

[m2-assistant]

Hi @engcom-Hotel. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[m2-assistant]

Hi @engcom-Echo. Thank you for working on this issue.
In order to make sure that issue has enough information and ready for development, please read and check the following instruction: 👇

• 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).

  Details

  If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.

• 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.

• 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.

• 4. Verify that the issue is reproducible on 2.4-develop branch

  Details

  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!

• 5. Add label Issue: Confirmed once verification is complete.

• 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-Echo]

Hi @sergeynezbritskiy
We tried to reproduce this issue on 2.4-develop & were unable to do so.
Also as per your comments, you have confirmed that issue is not reproducible on 2.4-develop, Hence we are closing this issue.
We request you to create separate issues for any new issues identified during development/debugging.
Thank you.

[ihor-sviziev]

@engcom-Echo, please double-check on 2.4.3, as this release wasn't yet merged to the 2.4-develop branch, or we can wait and then re-check again after merging.

[t-heuser]

I can confirm that this issue exists. I just updated to magento 2.4.3 and then tried to run integration tests. I get the exact same error as @sergeynezbritskiy stated in the issue.

The same bug also appears when trying to install magento with the file app/etc/config.php present.
To reproduce try the following:

• Install magento from scratch
• Drop and recreate the magento database
• remove the file app/etc/env.php
• run the command bin/magento setup:install ... again
• You'll get the the error:

In ResourceConnection.php line 148:

  [DomainException]
  Connection "default" is not defined

I think that these errors are related as the setup process for integration tests tries to install magento with an env.php and config.php present, so I think both errors are fixable at once.

Should I still create an extra issue for it or not, what do you think @ihor-sviziev?

[ihor-sviziev]

No need to create a separate issue. I reopened this one

[github-jira-sync-bot]

✅ Jira issue https://jira.corp.magento.com/browse/AC-1405 is successfully created for this GitHub issue.

[m2-assistant]

✅ Confirmed by @engcom-Alfa. Thank you for verifying the issue.
Issue Available: @engcom-Alfa, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

[mrodespin]

We are struggling with the same issue, and i can confirm that the PR #33803 fixes it.

[astephanh]

i too can confirm, that the PR #33803 fixes the issue.

[jreinke]

For a reason I do not have time to investigate for, I have removed the package magento/composer-root-update-plugin from my integration tests pipeline and it works now:

composer remove magento/composer-root-update-plugin

Maybe it will help someone!

[BorisovskiP]

Any updates on this issue? I can reproduce it in a pipeline on Magento 2.4.4.

[ilnytskyi]

I also confirm that #33803 fixes the issue with installation in pipelines in our case

[pawan-mittal]

Any updates on this issue? I can reproduce it in a pipeline on Magento 2.4.4.

[engcom-Hotel]

Hello @pawan-mittal,

The related PR #33803 is in Merge in progress status.

Thanks

[Vinai]

Reproduced on 2.4.5

[norgeindian]

Does anybody by chance already have a patch file to fix this?

[hostep]

Here's one for Magento 2.4.5 which I've created earlier today, it's based on #33803 but had to fix some merge conflicts, so hopefully those were solved correctly, but it seems to work on our end at least.

PR-33803-Framework.txt

[norgeindian]

@hostep , thanks a lot, awesome. Works fine for me.

[peterjaap]

Reproduced on 2.4.5-p1.

The patch provided by @hostep works.

[peterjaap]

The current pull request here does not work on 2.4.3 currently since the load() function differs between 2.4.3 and 2.4.4;

• https://github.com/magento/magento2/blob/2.4.3/lib/internal/Magento/Framework/App/DeploymentConfig.php#L150
• https://github.com/magento/magento2/blob/2.4.4/lib/internal/Magento/Framework/App/DeploymentConfig.php#L167

[convenient]

FWIW i've just got this on 2.4.7-beta1

My CI tooling can fix it by re-running composer install --no-interaction (which I discovered when playing with this comment #33802 (comment))

https://github.com/AmpersandHQ/magento-docker-test-instance/pull/12/files#diff-79a3628f21c70a32e21bfe89edb38a4f158b1707f6c1bea98083204d0934484fR143

[hostep]

This problem has indeed returned in 2.4.7-beta1, but also in 2.4.6-p2 (which is now available in pre-release and will be available from the 8th of August for everyone).

I've opened a new issue for this: #37805

[bhaveshseepossible]

Here's one for Magento 2.4.5 which I've created earlier today, it's based on #33803 but had to fix some merge conflicts, so hopefully those were solved correctly, but it seems to work on our end at least.

PR-33803-Framework.txt

@hostep your patch is very useful.
Thank you