Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add SplFixedArray::__set_state #6261

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 3 additions & 0 deletions NEWS
Expand Up @@ -11,5 +11,8 @@ PHP NEWS
- PSpell:
. Convert resource<pspell> to object \PSpell. (Sara)
. Convert resource<pspell config> to object \PSPellConfig. (Sara)
- SPL:
. Add SplFixedArray::__set_state (FR #75268) (levim). Note this converts
SplFixedArray from get_properties to get_properties_for.

<<< NOTE: Insert NEWS from last stable release here prior to actual release! >>>
133 changes: 121 additions & 12 deletions ext/spl/spl_fixedarray.c
Expand Up @@ -195,23 +195,18 @@ static HashTable* spl_fixedarray_object_get_gc(zend_object *obj, zval **table, i
return ht;
}

static HashTable* spl_fixedarray_object_get_properties(zend_object *obj)
static HashTable* spl_fixedarray_get_properties_for(zend_object *obj, zend_prop_purpose purpose)
{
spl_fixedarray_object *intern = spl_fixed_array_from_obj(obj);
HashTable *ht = zend_std_get_properties(obj);

if (!spl_fixedarray_empty(&intern->array)) {
zend_long j = zend_hash_num_elements(ht);
/* Keep the values and properties separate*/
HashTable *ht = zend_array_dup(zend_std_get_properties(obj));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#8046

In order for a native datastructure to correctly implement
*get_properties_for for var_export's cycle detection,
it would need to return the exact same array every time prior to this PR. (#8046)

I was working with get_properties_for again after the last review, I'm pretty sure this will cause new problems with var_export on get_properties_for if #8046 isn't merged, for

$arr = new SplFixedArray(1);
$arr[0] = $arr;
var_export($arr);  // now calls get_properties_for with purpose *VAR_EXPORT


if (!spl_fixedarray_empty(&intern->array)) {
for (zend_long i = 0; i < intern->array.size; i++) {
zend_hash_index_update(ht, i, &intern->array.elements[i]);
zend_hash_index_add_new(ht, i, &intern->array.elements[i]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks correct - If I remember correctly, everything in zend_std_get_properties is a string starting in php 7.x(I forget the exact minor version), and even 123 gets coerced to '123' when added to the properties table.

php > $x = new SplFixedArray(1); $x[0] = 'field'; $x->{'0'} = 'prop'; var_dump($x);
object(SplFixedArray)#1 (2) {
  ["0"]=>
  string(4) "prop"
  [0]=>
  string(5) "field"
}
php > echo serialize($x);
O:13:"SplFixedArray":2:{s:1:"0";s:4:"prop";i:0;s:5:"field";}

Z_TRY_ADDREF(intern->array.elements[i]);
}
if (j > intern->array.size) {
for (zend_long i = intern->array.size; i < j; ++i) {
zend_hash_index_del(ht, i);
}
}
}

return ht;
Expand Down Expand Up @@ -628,7 +623,6 @@ PHP_METHOD(SplFixedArray, fromArray)
} else if (num > 0 && !save_indexes) {
zval *element;
zend_long i = 0;

spl_fixedarray_init(&array, num);

ZEND_HASH_FOREACH_VAL(Z_ARRVAL_P(data), element) {
Expand Down Expand Up @@ -754,6 +748,120 @@ PHP_METHOD(SplFixedArray, getIterator)
zend_create_internal_iterator_zval(return_value, ZEND_THIS);
}

/* Assumes the object has been created and the spl_fixedarray_object's
* array member is uninitialized or zero-initialized.
* The state can have string keys, but they must come prior to integers:
*
* SplFixedArray::__set_state(array(
* 'property' => 'value',
* 0 => 1,
* 1 => 2,
* 2 => 3,
* ))
*/
static zend_result spl_fixedarray_import(zend_object *object, HashTable *ht)
{
spl_fixedarray_object *intern = spl_fixed_array_from_obj(object);
spl_fixedarray *fixed = &intern->array;

spl_fixedarray_init(fixed, 0);

/* For performance we do a single pass over the input array and the
* spl_fixedarray elems. This complicates the implementation, but part
* of the reason for choosing SplFixedArray is for peformance, and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* of the reason for choosing SplFixedArray is for peformance, and
* of the reason for choosing SplFixedArray is for performance, and

* although that is primarily for memory we should still care about CPU.
*
* We need to track the number of string keys, which must come before
* all the integer keys. This way the total size of the input minus the
* number of string keys equals the number of integer keys, so we can
* allocate the spl_fixedarray.elements and do this in a single pass.
*/

// The total size of the input array
const zend_long nht = zend_hash_num_elements(ht);
zend_long nstrings = 0; // The number of string keys
zend_long nints; // will be nht - nstrings

zend_string *str_idx = NULL; // current string key of input array
zend_ulong num_idx = 0; // current int key (valid iff str_idx == NULL)
zval *val = NULL; // current value of input array
zend_long max_idx = -1; // the largest index found so far
zval *begin = NULL; // pointer to beginning of fixedarray's storage
zval *end = NULL; // points one element passed the last element
const char *ex_msg = NULL; // message for the value exception

ZEND_HASH_FOREACH_KEY_VAL(ht, num_idx, str_idx, val) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
ZEND_HASH_FOREACH_KEY_VAL(ht, num_idx, str_idx, val) {
ZEND_HASH_FOREACH_KEY_VAL(ht, num_idx, str_idx, val) {
ZVAL_DEREF(val);

This should either dereference references or throw an exception, e.g. for $arr = [&$val]; SplFixedArray::__set_state($arr); there shouldn't be references in the final result.

if (str_idx != NULL) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would UNEXPECTED make sense - common use cases wouldn't add dynamic properties

if (UNEXPECTED(max_idx >= 0)) {
ex_msg = "must have all its string keys come before all integer keys";
goto value_error;
}

++nstrings;
object->handlers->write_property(object, str_idx, val, NULL);

} else if (UNEXPECTED((zend_long)num_idx != ++max_idx)) {
ex_msg = "did not have integer keys that start at 0 and increment sequentially";
goto value_error;

} else {
if (UNEXPECTED(max_idx == 0)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: UNEXPECTED may be a bit worse for performance - it's guaranteed to happen exactly once in non-empty SplFixedArrays, and the typical fixed array is small. It moves the code to a cold region far away from the fast code.
(i.e. arrays of size 1 are more common than size 2, and so on)

The UNEXPECTED macro means you think there's a 90%+ chance the branch won't be taken in gcc by default # define UNEXPECTED(condition) __builtin_expect(!!(condition), 0) - newer gcc versions support __builtin_expect_with_probability for specifying a higher probability, but this is off topic

https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html

For the purposes of branch prediction optimizations, the probability that a __builtin_expect expression is true is controlled by GCC’s builtin-expect-probability parameter, which defaults to 90%.

nints = nht - nstrings;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to check if the size changed to be safe? If there's UNEXPECTEDly a non-empty array when this line is changed, then either

  1. Throw
  2. Clear the array (e.g. call destructors) and (optionally) check for an exception - it's probably safe not to check for an exception from clearing or write_property but I'm not 100% sure

For example, really ill-formed code might do the following - this may be worth adding as a unit test (does it warn about a leak with --enable-debug)

class MyFixedArray extends SplFixedArray {
    public function __set($name, $value) { $this->setSize(1); $this[0] = [$name, $value]; }
}
// write_property calls __set?
MyFixedArray::__set_state(['key' => 'value', 0 => 'other value']);

fixed->size = nints;
fixed->elements =
safe_emalloc(nints, sizeof(zval), 0);
begin = fixed->elements;
end = fixed->elements + nints;
}

ZEND_ASSERT(num_idx == max_idx);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is redundant from the previous condition

ZEND_ASSERT(begin != end);

ZVAL_COPY(begin++, val);
}
} ZEND_HASH_FOREACH_END();

ZEND_ASSERT(begin == end);
return SUCCESS;

value_error:
spl_fixedarray_dtor_range(fixed, 0, max_idx);
if (fixed->elements) {
efree(fixed->elements);
}

/* Zero-initialize so the object release is valid. */
spl_fixedarray_init(fixed, 0);
zend_object_release(object);

zend_argument_value_error(1, ex_msg);
return FAILURE;
}

PHP_METHOD(SplFixedArray, __set_state)
{
zval *state = NULL;

ZEND_PARSE_PARAMETERS_START(1, 1)
Z_PARAM_ARRAY(state);
ZEND_PARSE_PARAMETERS_END();

HashTable *ht = Z_ARRVAL_P(state);
zend_class_entry *called_scope = zend_get_called_scope(execute_data);

zend_result result = object_init_ex(return_value, called_scope);
if (UNEXPECTED(result != SUCCESS)) {
ZVAL_NULL(return_value);
RETURN_THROWS();
}

zend_object *object = Z_OBJ_P(return_value);
if (UNEXPECTED(spl_fixedarray_import(object, ht) != SUCCESS)) {
ZVAL_NULL(return_value);
RETURN_THROWS();
}
}

static void spl_fixedarray_it_dtor(zend_object_iterator *iter)
{
zval_ptr_dtor(&iter->data);
Expand Down Expand Up @@ -844,7 +952,8 @@ PHP_MINIT_FUNCTION(spl_fixedarray)
spl_handler_SplFixedArray.unset_dimension = spl_fixedarray_object_unset_dimension;
spl_handler_SplFixedArray.has_dimension = spl_fixedarray_object_has_dimension;
spl_handler_SplFixedArray.count_elements = spl_fixedarray_object_count_elements;
spl_handler_SplFixedArray.get_properties = spl_fixedarray_object_get_properties;
spl_handler_SplFixedArray.get_properties_for
= spl_fixedarray_get_properties_for;
spl_handler_SplFixedArray.get_gc = spl_fixedarray_object_get_gc;
spl_handler_SplFixedArray.dtor_obj = zend_objects_destroy_object;
spl_handler_SplFixedArray.free_obj = spl_fixedarray_object_free_storage;
Expand Down
3 changes: 3 additions & 0 deletions ext/spl/spl_fixedarray.stub.php
Expand Up @@ -49,4 +49,7 @@ public function offsetSet($index, mixed $value) {}
public function offsetUnset($index) {}

public function getIterator(): Iterator {}

/** @return SplFixedArray */
public static function __set_state(array $state) {}
}
8 changes: 7 additions & 1 deletion ext/spl/spl_fixedarray_arginfo.h
@@ -1,5 +1,5 @@
/* This is a generated file, edit the .stub.php file instead.
* Stub hash: 831fe70055eb62135ae49321e5e5f3fe08c3d95f */
* Stub hash: 54028c566f2d868da1f91e5f4f627fbe6e823bb5 */

ZEND_BEGIN_ARG_INFO_EX(arginfo_class_SplFixedArray___construct, 0, 0, 0)
ZEND_ARG_TYPE_INFO_WITH_DEFAULT_VALUE(0, size, IS_LONG, 0, "0")
Expand Down Expand Up @@ -39,6 +39,10 @@ ZEND_END_ARG_INFO()
ZEND_BEGIN_ARG_WITH_RETURN_OBJ_INFO_EX(arginfo_class_SplFixedArray_getIterator, 0, 0, Iterator, 0)
ZEND_END_ARG_INFO()

ZEND_BEGIN_ARG_INFO_EX(arginfo_class_SplFixedArray___set_state, 0, 0, 1)
ZEND_ARG_TYPE_INFO(0, state, IS_ARRAY, 0)
ZEND_END_ARG_INFO()


ZEND_METHOD(SplFixedArray, __construct);
ZEND_METHOD(SplFixedArray, __wakeup);
Expand All @@ -52,6 +56,7 @@ ZEND_METHOD(SplFixedArray, offsetGet);
ZEND_METHOD(SplFixedArray, offsetSet);
ZEND_METHOD(SplFixedArray, offsetUnset);
ZEND_METHOD(SplFixedArray, getIterator);
ZEND_METHOD(SplFixedArray, __set_state);


static const zend_function_entry class_SplFixedArray_methods[] = {
Expand All @@ -67,5 +72,6 @@ static const zend_function_entry class_SplFixedArray_methods[] = {
ZEND_ME(SplFixedArray, offsetSet, arginfo_class_SplFixedArray_offsetSet, ZEND_ACC_PUBLIC)
ZEND_ME(SplFixedArray, offsetUnset, arginfo_class_SplFixedArray_offsetUnset, ZEND_ACC_PUBLIC)
ZEND_ME(SplFixedArray, getIterator, arginfo_class_SplFixedArray_getIterator, ZEND_ACC_PUBLIC)
ZEND_ME(SplFixedArray, __set_state, arginfo_class_SplFixedArray___set_state, ZEND_ACC_PUBLIC|ZEND_ACC_STATIC)
ZEND_FE_END
};
@@ -1,5 +1,7 @@
--TEST--
Objects with overloaded get_properties are incompatible with ArrayObject
--XFAIL--
SplFixedArray has migrated to get_properties_for; find new test case
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess you could add a stub class to ext/zend_test and move this test there for unit testing?

--FILE--
<?php

Expand Down
60 changes: 60 additions & 0 deletions ext/spl/tests/spl_fixedarray___set_state.phpt
@@ -0,0 +1,60 @@
--TEST--
SplFixedArray: __set_state
--FILE--
<?php

$fixed = SplFixedArray::__set_state([1, 2, 3]);
var_export($fixed);
echo "\n---\n";

// dynamic properties
$fixed->undefined = 'undef';
var_export($fixed);
echo "\n---\n";

// inheritance
class Vec
extends SplFixedArray
{
public $type;
}
$vec = new Vec(3);
$vec[0] = 1;
$vec[1] = 2;
$vec[2] = 3;
$vec->type = 'int';
var_export($vec);
echo "\n---\n";

$vec->undefined = 'undef';
var_export($vec);

?>
--EXPECT--
SplFixedArray::__set_state(array(
0 => 1,
1 => 2,
2 => 3,
))
---
SplFixedArray::__set_state(array(
'undefined' => 'undef',
0 => 1,
1 => 2,
2 => 3,
))
---
Vec::__set_state(array(
'type' => 'int',
0 => 1,
1 => 2,
2 => 3,
))
---
Vec::__set_state(array(
'type' => 'int',
'undefined' => 'undef',
0 => 1,
1 => 2,
2 => 3,
))
36 changes: 36 additions & 0 deletions ext/spl/tests/spl_fixedarray___set_state_errors.phpt
@@ -0,0 +1,36 @@
--TEST--
SplFixedArray: __set_state
--FILE--
<?php

try {
$fixed = SplFixedArray::__set_state([1 => 2]);
} catch (ValueError $error) {
var_dump($error->getMessage());
}

try {
$fixed = SplFixedArray::__set_state([0 => 1, 1 => 2, 3 => 4]);
} catch (ValueError $error) {
var_dump($error->getMessage());
}

try {
$fixed = SplFixedArray::__set_state([-1 => 2]);
} catch (ValueError $error) {
var_dump($error->getMessage());
}

echo "-----\n";

try {
$fixed = SplFixedArray::__set_state([0 => 1, 'type' => 'undefined']);
} catch (ValueError $error) {
var_dump($error->getMessage());
}
--EXPECTF--
string(%d) "SplFixedArray::__set_state(): Argument #1 ($state) did not have integer keys that start at 0 and increment sequentially"
string(%d) "SplFixedArray::__set_state(): Argument #1 ($state) did not have integer keys that start at 0 and increment sequentially"
string(%d) "SplFixedArray::__set_state(): Argument #1 ($state) did not have integer keys that start at 0 and increment sequentially"
-----
string(%d) "SplFixedArray::__set_state(): Argument #1 ($state) must have all its string keys come before all integer keys"
49 changes: 49 additions & 0 deletions ext/spl/tests/spl_fixedarray_export_import.phpt
@@ -0,0 +1,49 @@
--TEST--
SplFixedArray: __set_state export and import
--FILE--
<?php

function compare_export($var, string $case) {
$exported = var_export($var, true);

$imported = eval("return {$exported};");
$reexported = var_export($imported, true);

if ($exported == $reexported) {
echo "Pass {$case}.\n";
} else {
echo "Fail {$case}.\nExpected:\n{$exported}\nActual:\n{$reexported}\n";
}
}

// simple
$fixed = SplFixedArray::__set_state([1, 2, 3]);
compare_export($fixed, 'simple');

// dynamic properties
$fixed->undefined = 'undef';
compare_export($fixed, 'dynamic properties');

// inheritance
class Vec
extends SplFixedArray
{
public $type;
}
$vec = new Vec(3);
$vec[0] = 1;
$vec[1] = 2;
$vec[2] = 3;
$vec->type = 'int';

compare_export($vec, 'inheritance');

// dynamic properties and inheritance
$vec->undefined = 'undef';
compare_export($vec, 'dynamic properties and inheritance');

--EXPECT--
Pass simple.
Pass dynamic properties.
Pass inheritance.
Pass dynamic properties and inheritance.