Project YAML files not being processed

[jripmeester]

Describe the bug

We added Campaign to a existing website. We rely heavy on the project config files. After updating the database from the live enviroment to our dev enviroment we run the project config update tool. All changes are made perfectly except we needed to re-configure all Campaign settings. Campaign itself was installed though.

To reproduce

Steps to reproduce the behaviour:

1. Take a website without campaign
2. Export database
3. Install campaign and setup campaigns and mailing lists
4. rebuild project config (to be sure all settings are saved to the config)
5. replace current db with old exported one
6. run project config tool to re configure settings
7. No campaign types or mailing list type are set

Expected behaviour

We expect that all campaign settings present in the project config files are processed when project config tool is run.

Versions

• Plugin version: 1.16.4
• Craft version:3.5.14

[bencroker]

It sounds like you might be overriding the plugin settings in step 5 (replace current db with old exported one). Can you try migrating the database before you install and configure the Campaign plugin?

[jripmeester]

@bencroker Yes I’m overwriting the settings on purpose so that the settings can be re-configured from the project config. All other db changes are re-configured again but those of campaign aren't.

[bencroker]

Ok, I'll do a review of the project config rebuild process in the Campaign plugin as a lot has changed in recent Craft versions, and will let you know what I find.

[bencroker]

Things are looking ok in the plugin. Looking at your steps, it looks to me like maybe the plugin needs to be installed, so the DB tables are created, before you rebuild project config.

[jripmeester]

Hi @bencroker, sounds fair. We do install plugin and configure the settings in one go.
But isn't that a order issue in the Craft config handling?

[bencroker]

Possibly. Perhaps @andris-sevcenko can chime in on this?

[brandonkelly]

If I’m reading this correctly, the DB backup workflow is really just to simulate deploying the project config changes to a new environment. In which case, expected behavior would be that the plugin gets installed and then its existing project config YAML settings get applied.

@bencroker take a look at https://craftcms.com/docs/3.x/extend/migrations.html#setting-default-project-config-data which shows how your install migration should be handling potential incoming project config data.

[bencroker]

Thanks for the response @brandonkelly. I'm not making any project config changes in the safeUp method in https://github.com/putyourlightson/craft-campaign/blob/1.16.5/src/migrations/Install.php

Here is where I'm listening for changes to project config:
https://github.com/putyourlightson/craft-campaign/blob/1.16.5/src/Campaign.php#L512-L541

6. run project config tool to re configure settings

@jripmeester How exactly are you re-applying project config?

[jripmeester]

@bencroker We take the following steps in our development enviroment.

1. We copy the live enviroment to our dev (data, database and project config)
2. We install campaign and configure mailinglist types en campaign types and add some fields to sections
3. We run a project config rebuild in our dev env (just to be sure all settings are saved, all campaign settings are nicely saved to yaml files)
4. We empty the db and re-import the live env database
5. We visit our admin / utilities and run the "apply all config changes".
6. After that campaign is nicely installed, fields are added to the sections but the mailinglist types and campaign types aren't set

After typing this i realise you only asked where do we run the project config changes; so that's step 5. In the admin / utilities / project config section.

Shall i share the project config folder with you? This contains some API keys so i would preferably e-mail it to you.

[bencroker]

Thanks for clarifying, no need to send anything over, I'll run some local tests and let you know what I find. Do you happen to use Commerce? I'm wondering if the process you followed is working with Commerce product types.

[jripmeester]

@bencroker We don't use commerce.

This is our composer file

{
"require": {
"adigital/cookie-consent-banner": "^1.2",
"angellco/spoon": "^3.5",
"anubarak/craft-relabel": "2.0.2",
"born05/craft-assetusage": "^2.0",
"brikdigital/assetfield-preview": "^1.0",
"brikdigital/assets-to-entries": "^1.0",
"brikdigital/entry-picker-list": "^1.0",
"craftcms/cms": "^3.5.5",
"craftcms/feed-me": "^4.2.0",
"craftcms/postmark": "^2.0",
"craftcms/redactor": "^2.4.0",
"craftcms/store-hours": "2.1.1.1",
"dolphiq/redirect": "^1.0",
"fruitstudios/linkit": "^1.1",
"goldinteractive/craft3-sitecopy": "^0.6.0",
"hillholliday/craft-user-manual": "^2.0",
"nav33d/craft-relations": "^1.2",
"nystudio107/craft-cookies": "^1.1",
"nystudio107/craft-minify": "^1.2",
"nystudio107/craft-seomatic": "^3.2",
"ostark/craft-async-queue": "^2.1",
"putyourlightson/craft-blitz": "^3.1",
"putyourlightson/craft-campaign": "^1.16",
"putyourlightson/craft-dashboard-begone": "^1.0",
"putyourlightson/craft-log-to-file": "^1.1",
"rias/craft-position-fieldtype": "^1.0",
"rias/craft-width-fieldtype": "^1.0",
"robuust/craft-sentry-log-target": "^1.1",
"spacecatninja/imager-x": "^3.0",
"superbig/craft-mjml": "^1.0",
"unionco/craft-related-entry-types": "^0.2.2",
"venveo/craft-bulkedit": "^2.0",
"verbb/navigation": "^1.3.27",
"verbb/super-table": "^2.4",
"verbb/wishlist": "^1.2",
"vlucas/phpdotenv": "^3.4.0",
"wrav/oembed": "^1.3",
"yiisoft/yii2-redis": "~2.0.0"
},
"autoload": {
"psr-4": {
"modules\collectionmodule\": "modules/collectionmodule/src/"
}
},
"config": {
"sort-packages": true,
"optimize-autoloader": true
},
"scripts": {
"post-root-package-install": [
"@php -r "file_exists('.env') || copy('.env.example', '.env');""
]
},
"repositories": [
{
"type":"path",
"url": "./plugins/*"
}
]
}

[bencroker]

So after investigating further, it turns out that @brandonkelly was, unsurprisingly, right about it being caused by a conflict in the incoming project config data. In this case, it wasn't in the install migration but in the main plugin file, which threw me off initially.

Fixed in 2229110 and released in 1.17.5 for your Friday updating pleasure.