drush config-import --partial doesn't create field tables in DB

[siolfyr]

Sorry if this isn't an issue, but it feels like one.

In D8, if using Drupal's config sync from the UI, new fields create their tables in the DB. If I do the following:

1. Make a DB dump
2. Make a new field
3. Config-export with Drush so the new field configs will be imported next time
4. Delete DB and restore from dbdump
5. Drush config-import

...the tables in the DB aren't created, but the config for this field exists in the DB. What's worse, if I also changed the config of a view to rely on the new field I made, the site will crash trying to load the field from the tables that don't exist.

I greatly appreciate things that automate what could be done in the UI. If drush cim is not made for this, is there a way to tell Drupal, "Hey, you know those fields in your config? I want those."

Tried with D8 8.0.5 and Drush 8.0.5. Thanks for reading.

[Berdir]

drush just calls core API's. This should work and we have tests in core that it does. I've also imported fields like this.

[siolfyr]

Any idea why it's not working in my case, then? I'm using the --source and --partial options. I'm very certain that it isn't working, I've tried the test a number of times.

[weitzman]

Can you try without --partial? That is, do a config-export, add/update the new files, and config-import from there.

[siolfyr]

When I didn't use the partial option, the node__field and node__revision tables are created. It seems like it must only be the --partial option that this is happening for, then.

[weitzman]

Thanks for testing that.

--partial uses a very low level API and apparently bypasses a lot of the config event system. We either need to put warnings on --partial, remove it, change it, or something else. I have long thought that--partial should

1. config-export into a temp dir
2. copy the contents of the --partial dir into the temp dir
3. config-import from the temp dir

You can see us bypassing the API by comparing these:

• _drush_config_import_partial
• _drush_config_import

ping @jhedstrom since he added it in #1170.

[podarok]

The thing is that config-import removing configs from active storage if they aren't included to importing one.
So here is a use-case why --partial should be kept
User creating Contact Form (Config Entity) and working with it
DEV team trying to push their own changes to the user's environment and drush config-import removes created Contact Forms...

[weitzman]

With the 3 steps I posted above, --partial would never remove anything.

[podarok]

Back in stone age... Before every deploy to prod, developer should grab all configs from there, resolve hundreds of conflicts, and deploy everything again...

[weitzman]

I dont understand that last comment.

If we replace the current --partial with the steps above (which are the same steps we do for config-edit already), then we need to decide about backward compat. Its arguably a bug fix but some folks may be surprised by new validation failures, new database fields getting created, etc.

[siolfyr]

Yeah, I understand. Those three steps are a good workaround for now. Thanks.

[weitzman]

Would be great if someone using --partial could confirm that this is fixed for them. I added a unit test which is passing so we should be good.

[siolfyr]

I've tested this and it's working to create the tables. But, there are a couple of import scenarios that worked before that don't work after the change:

1. Installing a new site & then importing a previous site (same code, just different installation) used to work with --partial, now errors out with:
   The import failed due for the following reasons: [error]
   Site UUID in source storage does not match the target storage.
   ...but this seems like it would be fixed by the --force option mentioned here: Add option to config-import to force installation with mismatched site UUID. #1625 -- Is there a reason this --force option isn't in the main branch yet?
2. Pre-existing entities complain that they need to be deleted before importing:
   Entities exist of type Taxonomy term and Vocabulary category. These entities need to be
   deleted before importing.
   Entities exist of type Shortcut link and Default. These entities need to be deleted before
   importing.
   -- This was done in the same scenario as Scenario 1, so not sure if it's because configs were being deleted & recreated that caused this error.

[MatthieuScarset]

Almoste same issue here: #2278
Not sure why the current issue is closed then...

[weitzman]

What version of Drush? We made a relevant fix in 8.1.0

[MatthieuScarset]

Hi,

I'm using 8.0.5 :(

Here's the full details of drush core-status:

Drupal version : 8.1.6
Site URI : http://default
Database driver : mysql
Database hostname : 127.0.0.1
Database port : 33067
Database username : drupaluser
Database name : database
Drupal bootstrap : Successful
Drupal user :
Default theme : bartik
Administration theme : seven
PHP executable : php.exe
PHP configuration : C:\tools\devdesktop\php5_5\php.ini
PHP OS : WINNT
Drush script : C:\tools\devdesktop\drush\vendor\drush\drush\drush.php
Drush version : 8.0.5
Drush temp directory : D:\Users\mscarset\AppData\Local\Temp
Drush configuration : C:\tools\devdesktop\drush\vendor\drush\drush\drushrc.php
Drush alias files : D:\Users\mscarset/.acquia/DevDesktop/Drush/Aliases/aliases.drushrc.php
Install profile : standard
Drupal root : D:\devdesktop\cmne
Drupal Settings File : sites/default/settings.php
Site path : sites/default
File directory path : sites/default/files
Temporary file directory path : c:\windows\temp
Sync config path : sync

[weitzman]

This is fixed in recent Drush releases.

[MatthieuScarset]

Hi,
I've updated Drush to 8.1.2 and I'm still not able to import Fields
Everything else seems OK (content type, modules etc...) but fields are still not created in database.

Here's my new drush details:
Drupal version : 8.1.6
Site URI : http://default
Database driver : mysql
Database hostname : 127.0.0.1
Database port : 33067
Database username : drupaluser
Database name : test
Drupal bootstrap : Successful
Drupal user :
Default theme : bartik
Administration theme : seven
PHP executable : php.exe
PHP configuration : C:\tools\devdesktop\php5_5\php.ini
PHP OS : WINNT
Drush script : C:\tools\devdesktop\drush\vendor\drush\drush\drush.php
Drush version : 8.1.2
Drush temp directory : D:\Users\mscarset\AppData\Local\Temp
Drush configuration :
Drush alias files :
Install profile : standard
Drupal root : D:\devdesktop\test
Drupal Settings File : sites/default/settings.php
Site path : sites/default
File directory path : sites/default/files
Temporary file directory path : c:\windows\temp
Sync config path : sync

[siolfyr]

I've dealt with this problem a lot (before it was fixed in the newer versions) and it seemed that if you import the config with the old version of drush, switching to the new version and importing the fields will still not create the tables. You should try importing them on a newly installed site, or from a backup of your site before you imported with the old drush. If you still need the current version of your site, try doing some config cleanup first. The new drush definitely works with this, as long as you don't have old config in the DB.