Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[6.x] Serialization of models on PHP 7.4 with compatibility with PHP 7.3 and below #30605

Merged
merged 6 commits into from Nov 19, 2019

Conversation

@dkulyk
Copy link
Contributor

dkulyk commented Nov 15, 2019

This is rewrited #30604 with using __serialize/__unserialize

Tests are leaved from #30604

This implementation no needed of additional property and backward compatibility with older version of PHP(<7.4)

@dkulyk dkulyk changed the title Serialization of models with compatibility with PHP 7.3 and below Serialization of models on PHP 7.4 with compatibility with PHP 7.3 and below Nov 15, 2019
));
if (! array_key_exists($name, $values)) {
continue;
}

This comment has been minimized.

Copy link
@driesvints

driesvints Nov 15, 2019

Member

This part will unfortunately break jobs which are already queued. If any app is upgraded with these changes those jobs won't be able to unserialize anymore.

This comment has been minimized.

Copy link
@dkulyk

dkulyk Nov 15, 2019

Author Contributor

Why?
In previous implementation anything won't be able unserialize anymore.

This comment has been minimized.

Copy link
@dkulyk

dkulyk Nov 15, 2019

Author Contributor

PHP <7.4 does not use __serialize. If you will upgrade only Laravel php still used __slee/__wakeup.

This comment has been minimized.

Copy link
@driesvints

driesvints Nov 16, 2019

Member

Because any already queued job will still have their model identifiers set on the properties. If you upgrade to php 7.4 then your queued jobs will break while with the current implementation you can upgrade to php 7.4 smoothly.

We can do the unserialize thing in a year or two perhaps when we know most people have transitioned to php 7.4

This comment has been minimized.

Copy link
@dkulyk

dkulyk Nov 16, 2019

Author Contributor

... then php pass all properties to __unserialize as array. See #30604 (comment)

This comment has been minimized.

Copy link
@dkulyk

dkulyk Nov 16, 2019

Author Contributor

I've added serialization structure test.
In PHP 7.4 and PHP <7.3 serialized objects are identical.

This comment has been minimized.

Copy link
@driesvints

driesvints Nov 16, 2019

Member

Ah thanks for that. If @taylorotwell is okay with it then this PR is fine by me.

@driesvints driesvints changed the title Serialization of models on PHP 7.4 with compatibility with PHP 7.3 and below [6.x] Serialization of models on PHP 7.4 with compatibility with PHP 7.3 and below Nov 15, 2019
@driesvints

This comment has been minimized.

Copy link
Member

driesvints commented Nov 15, 2019

@dkulyk thanks for this! One concern atm with regards to BC for existing queued jobs.

dkulyk added 2 commits Nov 16, 2019
@dkulyk dkulyk force-pushed the dkulyk:php74_models_serializations branch from 26dd8d9 to 01f0745 Nov 16, 2019
CS
@dkulyk dkulyk force-pushed the dkulyk:php74_models_serializations branch from 01f0745 to 3ef985f Nov 16, 2019
@taylorotwell taylorotwell reopened this Nov 18, 2019
if ($property->isPrivate()) {
$name = "\0{$class}\0{$name}";
} elseif ($property->isProtected()) {
$name = "\0*\0{$name}";

This comment has been minimized.

Copy link
@taylorotwell

taylorotwell Nov 18, 2019

Member

What is this stuff?

This comment has been minimized.

Copy link
@dkulyk

dkulyk Nov 18, 2019

Author Contributor

PHP adds \0*\0 for protected and \0className\0 for private properties in serialization.
serialization_objects_test, zend_mangle_property_name and usage 1, 2

This comment has been minimized.

Copy link
@netpok

netpok Nov 18, 2019

Contributor

It's needed for backward compatibility with php7.3 workers.

A php7.3 serialize command will produce this name in the serialized array so by naming it this way a 7.3 worker can work together with a 7.4 worker (as long as it's not using typed properties).

@netpok

This comment has been minimized.

Copy link
Contributor

netpok commented Nov 18, 2019

One thing I noticed: the parent classes private variable are lost this way:

take the following code (which is basically the same without the model wrapping):

use ReflectionClass;

class A extends C
{
    use B;
    private $a = 1;
    protected $b = 2;
    public $c = 3;

    public function __serialize()
    {
        $values = [];
        $properties = (new ReflectionClass($this))->getProperties();
        $class = get_class($this);
        foreach ($properties as $property) {
            if ($property->isStatic()) {
                continue;
            }
            $name = $property->getName();
            if ($property->isPrivate()) {
                $name = "\0{$class}\0{$name}";
            } elseif ($property->isProtected()) {
                $name = "\0*\0{$name}";
            }
            $property->setAccessible(true);
            $values[$name] = $property->getValue($this);
        }
        return $values;
    }

}

trait B{
    private $d = 4;
}

class C{
    private $e = 5;
}

calling serialize(new A) with the __serialize method defined will result in:

"O:5:"App\A":4:{s:8:"\0App\A\0a";i:1;s:4:"\0*\0b";i:2;s:1:"c";i:3;s:8:"\0App\A\0d";i:4;}"

calling serialize(new A) without the __serialize method defined will result in:

"O:5:"App\A":5:{s:8:"\0App\A\0a";i:1;s:4:"\0*\0b";i:2;s:1:"c";i:3;s:8:"\0App\C\0e";i:5;s:8:"\0App\A\0d";i:4;}"

Meaning the C::e variable were lost in the serialization process.


A possible solution would be to travel up to the root class and get all the private properties:

    public function __serialize()
    {
        $values = [];
        $reflectionClass = new ReflectionClass($this);

        $properties = ($reflectionClass)->getProperties();

        $class = $reflectionClass->getName;
        foreach ($properties as $property) {
            if ($property->isStatic()) {
                continue;
            }
            $name = $property->getName();
            if ($property->isPrivate()) {
                $name = "\0{$class}\0{$name}";
            } elseif ($property->isProtected()) {
                $name = "\0*\0{$name}";
            }
            $property->setAccessible(true);
            $values[$name] = $property->getValue($this);
        }

        while ($reflectionClass = $reflectionClass->getParentClass()) {
            $class = $reflectionClass->getName;

            foreach ($reflectionClass->getProperties(ReflectionProperty::IS_PRIVATE) as $property) {
                $name = $property->getName();
                $name = "\0{$class}\0{$name}";
                $property->setAccessible(true);
                $values[$name] = $property->getValue($this);
            }
        }

        return $values;
    }
dkulyk added 2 commits Nov 18, 2019
CS
@dkulyk

This comment has been minimized.

Copy link
Contributor Author

dkulyk commented Nov 18, 2019

One thing I noticed: the parent classes private variable are lost this way:
I've added test for this. for 7.3 also private fields not present for parent class

@netpok

This comment has been minimized.

Copy link
Contributor

netpok commented Nov 18, 2019

@dkulyk I sent a possible solution, of course the serialization logic need to be changed to the correct one


Also some performance measurements should be done, maybe somehow cache the serializeable property names if needed.

@dkulyk

This comment has been minimized.

Copy link
Contributor Author

dkulyk commented Nov 18, 2019

__sleep

It is not possible for __sleep() to return names of private properties in parent classes. Doing this will result in an E_NOTICE level error.

But it can be implemented in __serialize

@netpok

This comment has been minimized.

Copy link
Contributor

netpok commented Nov 18, 2019

I only tested the latter so there is a chance the other one never worked.

Tested with current implementation, it does indeed remove parent private variables, maybe it would be preferred to keep the exact working (as it can be implemented in a way more optimized fashion).

@netpok

This comment has been minimized.

Copy link
Contributor

netpok commented Nov 18, 2019

Probably the best way would be to drop private variable support altogether in v7.x, that would allow to access parameters with $this->{$name} meaning the reflection could be "booted" once per class per session.

On first run class public and protected properties are cached, on each subsequent call it is only pulled form the cache.

Also private variables could be used after this to handle internal non-serializable stuff.

@driesvints

This comment has been minimized.

Copy link
Member

driesvints commented Nov 18, 2019

@netpok excellent point. Maybe it's indeed better to only serialize protected and public and leave private alone?

@dkulyk

This comment has been minimized.

Copy link
Contributor Author

dkulyk commented Nov 19, 2019

This PR implements same behavior like for PHP 7.3. If @taylorotwell will decided what need remove private properties or add from parent classes - I'll change it.

@driesvints

This comment has been minimized.

Copy link
Member

driesvints commented Nov 19, 2019

@dkulyk that's true 👍

@taylorotwell taylorotwell merged commit da12c80 into laravel:6.x Nov 19, 2019
2 checks passed
2 checks passed
continuous-integration/styleci/pr The analysis has passed
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@dkulyk dkulyk deleted the dkulyk:php74_models_serializations branch Nov 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.