Magento 2 suddenly does not work with REDIS anymore after upgrade to (M2.1.6)

[commcad]

Preconditions

1. M2.1.4 > M2.1.6 - NGINX - phpREDIS

Steps to reproduce

1. Run $php bin/magento setup:static-content:deploy"
2. Getting:
   [Exception] Notice: unserialize(): Error at offset 0 of 1 bytes in /var/www/domain.com/vendor/magento/module-config/App/Config/Type/System.php on line 214

Expected result

1. Able to deploy without errors

Actual result

1. The newly updated System.php file (M2.1.6) "vendor/magento/module-config/App/Config/Type/System.php" was changed and breaking the connection with REDIS. It was however working fine with M2.1.4 and all earlier version of System.php
2. Run $php bin/magento setup:static-content:deploy"
3. Now getting:
   [Exception] Notice: unserialize(): Error at offset 0 of 1 bytes in /var/www/domain.com/vendor/magento/module-config/App/Config/Type/System.php on line 214

[AVoskoboinikov]

Hi @commcad.

Have you followed this steps while upgrading?

Please, pay attention for 5th step - clear var/ directories as well as run FLUSHALL command in redis cli.

[abhilashv1985]

I am using Redis version 3.0.6 and Magento 2.1.5. It is working smooth for me. Could you share the server confifuration ?

[mattias]

We just got this issue from a clean install of Magento 2.1.6 using composer installation.
We use CentOS 7, redis 3.2.8, php 7.0.18, nginx 1.12.0, mysql 5.6.36 and Magento 2.1.6.
Doing php bin/magento deploy:mode:set production, even after clearing all cache folders under the var folder, still gives this error.

We've set up countless of M2 sites prior to this version and have never had any issues.

Also worth noting is it is not only this file, sometimes it says the same about this:
Notice: unserialize(): Error at offset 0 of 1 bytes in /home/mediastrategi/public_html/vendor/magento/framework/Interception/PluginList/PluginList.php on line 272

It also only seems to happen after a setup:di:compile, if I run the command manually after it fails, I don't get this issue.

Now, because compile runs before static-content deploy and this sequence makes static-content deploy fail, I can't change from default mode to production mode.

Edit: I fixed the issue by first changing to developer mode then to production mode. Must be something with default mode -> production mode that doesn't work.

[bery]

When you run FLUSHALL on redis prior to deploying static content, all works fine but it is a workaround, not a permanent solution.

[KevinMace]

I can confirm that i'm receiving the exact same error during the deployment of static-content when upgrading to 2.1.5, even when following the documentation and flushing Redis.

[mattias]

Now we've upgraded a site from 2.1.4 to 2.1.6, cleaned and refreshed everything as well as flushing the redis cache etc. We were in production mode already but things were kind of broken, so we switched to developer mode then back to production and now it won't work since di:compile causes an issue when static-content:deploy runs right after as explained in an earlier post here.

static-content:deploy always works on a second try after it fails. But that's not how deploy:mode:set works when changing to production, it has to work the first time around.

This specific site is a clean Magento installation with sample-data (demo site). Right now it is unusable.

This bug must have been introduced with 2.1.6 since we haven't seen it before this version.

[wintermute-84]

Exactly the same problem. Introduced after 2.1.6 upgrade.
Getting

[Exception]                                                                                                                                                          
  Notice: unserialize(): Error at offset 0 of 1 bytes in /home/web/m2test/releases/20170505110400/vendor/magento/module-config/App/Config/Type/System.php on line 214

and sometimes [CredisException] read error on connection after that.

Unable to switch to production mode.
Tried to clean all possible caches, rarely it works, but most of the time it messes up with our deployment process.

[andersro93]

I can confirm that I get this issue to, and for our shop the best solution is to run the cache:flush command in prior to the setup:static-content:deploy command. Like this:
bin/magento cache:flush && bin/magento setup:static-content:deploy

[driskell]

Having same issue too with RedHat 7, PHP 7, Redis configured. Sometimes I can run cache:flush and it sometimes won't work (possibly due to activity on site) and can take another attempt. I have a replica environment without Redis configured and it works flawlessly. Seems to be introduced since upgrading to 2.1.4 I think.

It's not an upgrade problem either. Every time we attempt to deploy any change and we run setup:di:compile and then static-content:deploy, it currently ALWAYS fails. We then have to manually re-deploy the static content after a few cache flush.

In my case when I debugged the System.php file I found that it was breaking on Error at offset 0 of 1 bytes for a request for "default/web/secure/ssl_offloader", and when attempting to load the cache for "default/web" it was instead receiving "1" a few times, before eventually receiving the full cache entry. So it seems the cache is being poisoned.

[driskell]

Oh wow.

Does static-content deployment now fork?

I'm seeing this. LOAD/LOAD FAILURE is from System.php. The rest is from Cm_Redis_Cache_Backend::load().

LOAD: [system_defaultweb]
CMCACHE LOAD: 123_SYSTEM_DEFAULTWEB
CMCACHE LOAD: 123_SYSTEM_CACHE_EXISTS
CMCACHE DECODE: 1
LOAD FAILURE: [1]
CMCACHE DECODE: a:8

As you can see, there must be two processes running, and it seems the first process sends a hGet for 123_SYSTEM_DEFAULTWEB, then the other process, which is in System.php, sends a hGet for 123_SYSTEM_CACHE_EXISTS. Unfortunately, due to a race condition here because the connection is shared, the System.php actually receives the response for the request for 123_SYSTEM_DEFAULTWEB.

This also explains the many variants and random error messages. Running in a random loop over and over and aborting after a second I received various messages from System.php to Redis connection lost, to invalid data returned from Redis.

Seems this might be the cause... static-content:deploy could be forking and sharing the redis connection.

[driskell]

This also explains the reason cache:flush works. It forces both forks to populate from database. It also explains it not working 100% of the time since if one fork too slow it could start loading from Redis. Also if there's activity on the site it could cause cache to be populated again.

[driskell]

Workaround for now which is deterministically safe:

magento setup:static-content:deploy -j 1

The -j 1 sets parallel processing to just a single process, which effectively undoes the parallelism change that must have been introduced in Magento 2.1.4 or thereabouts.