Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Zend\Db\Resultset fix buffering #5308

Closed
wants to merge 3 commits into from

5 participants

Yaroslav Sych Abdul Malik Ikhsan Ralph Schindler Matthew Weier O'Phinney Marco Pivetta
Yaroslav Sych

fix buffering
optimize some methods

library/Zend/Db/ResultSet/HydratingResultSet.php
@@ -56,6 +58,16 @@ public function setObjectPrototype($objectPrototype)
}
/**
+ * Get the row object prototype
+ *
+ * @return stdObject

stdClass maybe ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Yaroslav Sych

My local tests is ok - where i wrong?

Abdul Malik Ikhsan

it seems the error is on ZendTest\Http\ClientTest::testUsageOfCustomResponseInvalidCode which not related with your commits.

Ralph Schindler
Collaborator

Can you expand on the problems you're trying to solve in these commits?

Yaroslav Sych

ResultSet and HydratingResultSet not use the buffer for converted values (see current() method) and convert current data on each current() call.
Parameters for ResultSet::__construct is excess, because you can set only $arrayObjectPrototype and $returnType can be calculated
Сode simplification.

Yaroslav Sych

Sorry - only ResultSet not use the buffer for converted values (see current() method) and convert current data on each current() call.
HydratingResultSet (in current()) using code which already exist in AbstractResultSet

turrsis added some commits
Yaroslav Sych turrsis fix buffering
optimize some methods
c3f3ac8
Yaroslav Sych turrsis added twice foreach tests
fixes for added tests
8612b40
Ralph Schindler
Collaborator

Hey @turris,

I can't seem to duplicate your problem. Using my repo at Zend_Db-Example (https://github.com/ralphschindler/Zend_Db-Examples):

I've created the following script and witness that the buffered values are in fact utilized:

<?php

/** @var $adapter Zend\Db\Adapter\Adapter */
$adapter = include ((file_exists('bootstrap.php')) ? 'bootstrap.php' : 'bootstrap.dist.php');
refresh_data($adapter);

use Zend\Db\ResultSet\HydratingResultSet;
use Zend\Db\ResultSet\ResultSet;

$stmt = $adapter->query('SELECT * FROM album');
$result = $stmt->execute();

if (1) {
    class Foo extends ArrayObject { public function __clone() { echo 'cloning' . PHP_EOL; } }
    $hrs = new HydratingResultSet();
    $hrs->setObjectPrototype(new Foo);
    $hrs->initialize($result);
    $hrs->buffer();
    foreach ($hrs as $row) {
        var_dump($row['title']);
    }
    foreach ($hrs as $row) {
        var_dump($row['title']);
    }
} else {
    $rs = new ResultSet;
    $rs->initialize($result);
    $rs->buffer();
    foreach ($rs as $row) {
        var_dump($row['title']);
    }
    foreach ($rs as $row) {
        var_dump($row['title']);
    }
}

If you still find that not to be the case, how can I setup a reproduction of your problem?

Yaroslav Sych

@ralphschindler If merge last commit to develop, only HydratingResultSetTest::testSameCurrentForIteratorDataAndBuffering() is ok

Ralph Schindler
Collaborator

So what you're saying is concurrent calls to current() without moving the internal pointer do not use the buffer. Ok, now I see what you're getting at. I'll have a look.

Yaroslav Sych

@ralphschindler you're right, sorry - this is my inaccurate description.

Matthew Weier O'Phinney

@ralphschindler Can you update and specify a target milestone, please?

Ralph Schindler ralphschindler added this to the 2.2.6 milestone
Matthew Weier O'Phinney weierophinney modified the milestone: 2.3.0, 2.2.6
Ralph Schindler
Collaborator

Hey @turrsis, while I admit there is an issue with iteration and buffering, there is also too much unnecessary refactoring going on in this PR to take as is. I will work backwards from your unit tests and produce an alternative PR.

Ralph Schindler
Collaborator

For the record, this would be the unit test to prove this is an issue:

namespace ZendTest\Db\ResultSet;

class AbstractResultSetIntegrationTest extends \PHPUnit_Framework_TestCase
{
    protected $resultSet;
    protected function setUp() {
        $this->resultSet = $this->getMockForAbstractClass('Zend\Db\ResultSet\AbstractResultSet');
    }

    public function testCurrentCallsDataSourceOnceWithoutBuffer()
    {
        $result = $this->getMock('Zend\Db\Adapter\Driver\ResultInterface');
        $this->resultSet->initialize($result);
        $result->expects($this->once())->method('current')->will($this->returnValue(array('foo' => 'bar')));
        $value1 = $this->resultSet->current();
        $value2 = $this->resultSet->current();
        $this->assertEquals($value1, $value2);
    }
}

That said, after thinking about this issue, I don't think this is a valid use case. I would expect that at the abstract level, without any buffering, that if I called current() twice, it should do just that with respect to the DataSource.

Marco Pivetta Ocramius commented on the diff
library/Zend/Db/ResultSet/AbstractResultSet.php
@@ -73,9 +81,7 @@ public function initialize($dataSource)
$this->dataSource->rewind();
}
return $this;
- }
-
- if (is_array($dataSource)) {
+ } elseif (is_array($dataSource)) {
Marco Pivetta Collaborator

Any reason for this to be turned into elseif?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Marco Pivetta Ocramius commented on the diff
library/Zend/Db/ResultSet/AbstractResultSet.php
@@ -49,6 +48,11 @@
protected $position = 0;
/**
+ * @var mixed
+ */
+ protected $current = null;
Marco Pivetta Collaborator

= null is not needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Marco Pivetta
Collaborator

@turrsis what happens if the data source is mutable? Shouldn't the value for current() change here?

Ralph Schindler ralphschindler referenced this pull request from a commit
Ralph Schindler ralphschindler Closes #5308 as Not An Issue
Merge branch 'hotfix/db-resultset-current-with-buffer-alt' into develop

* hotfix/db-resultset-current-with-buffer-alt:
  Added unit tests for Zend\Db\ResultSet current() handling
f964394
Ralph Schindler
Collaborator

I've committed a few unit tests that describe the current behavior as the expected intended behavior.

Yaroslav Sych turrsis deleted the branch
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 15, 2013
  1. Yaroslav Sych

    fix buffering

    turrsis authored
    optimize some methods
  2. Yaroslav Sych

    added twice foreach tests

    turrsis authored
    fixes for added tests
Commits on Nov 19, 2013
  1. Yaroslav Sych

    added test for Iterator data

    turrsis authored
This page is out of date. Refresh to see the latest.
119 library/Zend/Db/ResultSet/AbstractResultSet.php
View
@@ -10,7 +10,6 @@
namespace Zend\Db\ResultSet;
use ArrayIterator;
-use ArrayObject;
use Countable;
use Iterator;
use IteratorAggregate;
@@ -49,6 +48,11 @@
protected $position = 0;
/**
+ * @var mixed
+ */
+ protected $current = null;
Marco Pivetta Collaborator

= null is not needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+ /**
* Set the data source for the result set
*
* @param Iterator|IteratorAggregate|ResultInterface $dataSource
@@ -62,6 +66,10 @@ public function initialize($dataSource)
$this->buffer = array();
}
+ $this->count = null;
+ $this->fieldCount = null;
+ $this->current = null;
+
if ($dataSource instanceof ResultInterface) {
$this->count = $dataSource->count();
$this->fieldCount = $dataSource->getFieldCount();
@@ -73,9 +81,7 @@ public function initialize($dataSource)
$this->dataSource->rewind();
}
return $this;
- }
-
- if (is_array($dataSource)) {
+ } elseif (is_array($dataSource)) {
Marco Pivetta Collaborator

Any reason for this to be turned into elseif?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
// its safe to get numbers from an array
$first = current($dataSource);
reset($dataSource);
@@ -91,10 +97,26 @@ public function initialize($dataSource)
throw new Exception\InvalidArgumentException('DataSource provided is not an array, nor does it implement Iterator or IteratorAggregate');
}
- if ($this->count == null && $this->dataSource instanceof Countable) {
+ if ($this->count === null && $this->dataSource instanceof Countable) {
$this->count = $this->dataSource->count();
}
+ if ($this->fieldCount === null) {
+ if ($this->count == 0) {
+ $this->fieldCount = 0;
+ } else {
+ $dataSource->rewind();
+ $row = $dataSource->current();
+ if ($row instanceof Countable) {
+ $this->fieldCount = $row->count();
+ } else {
+ $this->fieldCount = count((array)$row);
+ }
+ }
+ } else {
+ $this->fieldCount = 0;
+ }
+
return $this;
}
@@ -113,10 +135,16 @@ public function buffer()
public function isBuffered()
{
- if ($this->buffer === -1 || is_array($this->buffer)) {
- return true;
+ return ($this->buffer === -1 || is_array($this->buffer));
+ }
+
+ protected function checkBuffered()
+ {
+ if ($this->buffer === null) {
+ // implicitly disable buffering from here on
+ $this->buffer = -2;
}
- return false;
+ return $this;
}
/**
@@ -136,29 +164,6 @@ public function getDataSource()
*/
public function getFieldCount()
{
- if (null !== $this->fieldCount) {
- return $this->fieldCount;
- }
-
- $dataSource = $this->getDataSource();
- if (null === $dataSource) {
- return 0;
- }
-
- $dataSource->rewind();
- if (!$dataSource->valid()) {
- $this->fieldCount = 0;
- return 0;
- }
-
- $row = $dataSource->current();
- if (is_object($row) && $row instanceof Countable) {
- $this->fieldCount = $row->count();
- return $this->fieldCount;
- }
-
- $row = (array) $row;
- $this->fieldCount = count($row);
return $this->fieldCount;
}
@@ -169,11 +174,10 @@ public function getFieldCount()
*/
public function next()
{
- if ($this->buffer === null) {
- $this->buffer = -2; // implicitly disable buffering from here on
- }
+ $this->checkBuffered();
$this->dataSource->next();
$this->position++;
+ $this->current = null;
}
/**
@@ -193,14 +197,36 @@ public function key()
*/
public function current()
{
- if ($this->buffer === null) {
- $this->buffer = -2; // implicitly disable buffering from here on
- } elseif (is_array($this->buffer) && isset($this->buffer[$this->position])) {
- return $this->buffer[$this->position];
+ if ($this->current !== null) {
+ return $this->current;
}
- $data = $this->dataSource->current();
+ $this->checkBuffered();
if (is_array($this->buffer)) {
- $this->buffer[$this->position] = $data;
+ if (!isset($this->buffer[$this->position])) {
+ return $this->current = $this->buffer[$this->position] = $this->hydrateCurrent();
+ }
+ return $this->current = $this->buffer[$this->position];
+ }
+ return $this->current = $this->hydrateCurrent();
+ }
+
+ protected function hydrateCurrent()
+ {
+ return $this->dataSource->current();
+ }
+
+ protected function extract($data)
+ {
+ if (is_array($data)) {
+ return $data;
+ } elseif (method_exists($data, 'toArray')) {
+ return $data->toArray();
+ } elseif (method_exists($data, 'getArrayCopy')) {
+ return $data->getArrayCopy();
+ } else {
+ throw new Exception\RuntimeException(
+ 'Rows as part of this DataSource, with type ' . gettype($data) . ' cannot be cast to an array'
+ );
}
return $data;
}
@@ -238,6 +264,7 @@ public function rewind()
}
}
$this->position = 0;
+ $this->current = null;
}
/**
@@ -264,17 +291,7 @@ public function toArray()
{
$return = array();
foreach ($this as $row) {
- if (is_array($row)) {
- $return[] = $row;
- } elseif (method_exists($row, 'toArray')) {
- $return[] = $row->toArray();
- } elseif (method_exists($row, 'getArrayCopy')) {
- $return[] = $row->getArrayCopy();
- } else {
- throw new Exception\RuntimeException(
- 'Rows as part of this DataSource, with type ' . gettype($row) . ' cannot be cast to an array'
- );
- }
+ $return[] = $this->extract($row);
}
return $return;
}
54 library/Zend/Db/ResultSet/HydratingResultSet.php
View
@@ -34,7 +34,7 @@ class HydratingResultSet extends AbstractResultSet
public function __construct(HydratorInterface $hydrator = null, $objectPrototype = null)
{
$this->setHydrator(($hydrator) ?: new ArraySerializable);
- $this->setObjectPrototype(($objectPrototype) ?: new ArrayObject);
+ $this->setObjectPrototype($objectPrototype);
}
/**
@@ -46,7 +46,9 @@ public function __construct(HydratorInterface $hydrator = null, $objectPrototype
*/
public function setObjectPrototype($objectPrototype)
{
- if (!is_object($objectPrototype)) {
+ if (!$objectPrototype) {
+ $objectPrototype = new ArrayObject;
+ } elseif (!is_object($objectPrototype)) {
throw new Exception\InvalidArgumentException(
'An object must be set as the object prototype, a ' . gettype($objectPrototype) . ' was provided.'
);
@@ -56,6 +58,16 @@ public function setObjectPrototype($objectPrototype)
}
/**
+ * Get the row object prototype
+ *
+ * @return stdClass
+ */
+ public function getObjectPrototype()
+ {
+ return $this->objectPrototype;
+ }
+
+ /**
* Set the hydrator to use for each row object
*
* @param HydratorInterface $hydrator
@@ -77,40 +89,16 @@ public function getHydrator()
return $this->hydrator;
}
- /**
- * Iterator: get current item
- *
- * @return object
- */
- public function current()
+ protected function hydrateCurrent()
{
- if ($this->buffer === null) {
- $this->buffer = -2; // implicitly disable buffering from here on
- } elseif (is_array($this->buffer) && isset($this->buffer[$this->position])) {
- return $this->buffer[$this->position];
- }
- $data = $this->dataSource->current();
- $object = is_array($data) ? $this->hydrator->hydrate($data, clone $this->objectPrototype) : false;
-
- if (is_array($this->buffer)) {
- $this->buffer[$this->position] = $object;
- }
-
- return $object;
+ $current = parent::hydrateCurrent();
+ return is_array($current)
+ ? $this->hydrator->hydrate($current, clone $this->getObjectPrototype())
+ : false;
}
- /**
- * Cast result set to array of arrays
- *
- * @return array
- * @throws Exception\RuntimeException if any row is not castable to an array
- */
- public function toArray()
+ protected function extract($data)
{
- $return = array();
- foreach ($this as $row) {
- $return[] = $this->getHydrator()->extract($row);
- }
- return $return;
+ return $this->getHydrator()->extract($data);
}
}
66 library/Zend/Db/ResultSet/ResultSet.php
View
@@ -14,17 +14,7 @@
class ResultSet extends AbstractResultSet
{
const TYPE_ARRAYOBJECT = 'arrayobject';
- const TYPE_ARRAY = 'array';
-
- /**
- * Allowed return types
- *
- * @var array
- */
- protected $allowedReturnTypes = array(
- self::TYPE_ARRAYOBJECT,
- self::TYPE_ARRAY,
- );
+ const TYPE_ARRAY = 'array';
/**
* @var ArrayObject
@@ -38,17 +28,21 @@ class ResultSet extends AbstractResultSet
*/
protected $returnType = self::TYPE_ARRAYOBJECT;
+ protected $defaultReturnType = self::TYPE_ARRAYOBJECT;
+
/**
* Constructor
*
* @param string $returnType
- * @param null|ArrayObject $arrayObjectPrototype
+ * @param null|ArrayObject $arrayObjectPrototype - this parameter id deprecated
*/
- public function __construct($returnType = self::TYPE_ARRAYOBJECT, $arrayObjectPrototype = null)
+ public function __construct($returnType = self::TYPE_ARRAYOBJECT)
{
- $this->returnType = (in_array($returnType, array(self::TYPE_ARRAY, self::TYPE_ARRAYOBJECT))) ? $returnType : self::TYPE_ARRAYOBJECT;
- if ($this->returnType === self::TYPE_ARRAYOBJECT) {
- $this->setArrayObjectPrototype(($arrayObjectPrototype) ?: new ArrayObject(array(), ArrayObject::ARRAY_AS_PROPS));
+ if (func_num_args() == 2 && func_get_arg(1) != null) {
+ //backward compatibility
+ $this->setArrayObjectPrototype(func_get_arg(1));
+ } else {
+ $this->setArrayObjectPrototype($returnType);
}
}
@@ -61,13 +55,23 @@ public function __construct($returnType = self::TYPE_ARRAYOBJECT, $arrayObjectPr
*/
public function setArrayObjectPrototype($arrayObjectPrototype)
{
- if (!is_object($arrayObjectPrototype)
- || (!$arrayObjectPrototype instanceof ArrayObject && !method_exists($arrayObjectPrototype, 'exchangeArray'))
-
- ) {
- throw new Exception\InvalidArgumentException('Object must be of type ArrayObject, or at least implement exchangeArray');
+ if (in_array($arrayObjectPrototype, array(null, self::TYPE_ARRAY, self::TYPE_ARRAYOBJECT), true)) {
+ $this->returnType = $arrayObjectPrototype ?:$this->defaultReturnType;
+ if ($this->returnType == self::TYPE_ARRAYOBJECT) {
+ $this->arrayObjectPrototype = new ArrayObject(array(), ArrayObject::ARRAY_AS_PROPS);
+ } else {
+ $this->arrayObjectPrototype = null;
+ }
+ } elseif (is_object($arrayObjectPrototype) && method_exists($arrayObjectPrototype, 'exchangeArray')) {
+ $this->returnType = self::TYPE_ARRAYOBJECT;
+ $this->arrayObjectPrototype = $arrayObjectPrototype;
+ } else {
+ throw new Exception\InvalidArgumentException(sprintf(
+ 'Object must be of type ArrayObject, or at least implement exchangeArray, or must be %s or $s',
+ self::TYPE_ARRAYOBJECT,
+ self::TYPE_ARRAY
+ ));
}
- $this->arrayObjectPrototype = $arrayObjectPrototype;
return $this;
}
@@ -91,22 +95,14 @@ public function getReturnType()
return $this->returnType;
}
- /**
- * @return array|\ArrayObject|null
- */
- public function current()
+ protected function hydrateCurrent()
{
- $data = parent::current();
-
- if ($this->returnType === self::TYPE_ARRAYOBJECT && is_array($data)) {
- /** @var $ao ArrayObject */
+ $current = parent::hydrateCurrent();
+ if (is_object($this->arrayObjectPrototype) && is_array($current)) {
$ao = clone $this->arrayObjectPrototype;
- if ($ao instanceof ArrayObject || method_exists($ao, 'exchangeArray')) {
- $ao->exchangeArray($data);
- }
+ $ao->exchangeArray($current);
return $ao;
}
-
- return $data;
+ return $current;
}
}
39 tests/ZendTest/Db/ResultSet/AbstractResultSetTest.php
View
@@ -67,6 +67,44 @@ public function testBuffer()
$resultSet->buffer();
}
+ public function testBuffer_ForEachTwice_WithArray()
+ {
+ $resultSet = $this->getMockForAbstractClass('Zend\Db\ResultSet\AbstractResultSet');
+ $resultSet->initialize(array(
+ array('id' => 1, 'name' => 'one'),
+ array('id' => 2, 'name' => 'two'),
+ array('id' => 3, 'name' => 'three'),
+ ));
+ $resultSet->buffer();
+
+ foreach($resultSet as $key=>$value) {
+ $this->assertEquals($value['id'], $key + 1);
+ }
+
+ foreach($resultSet as $key=>$value) {
+ $this->assertEquals($value['id'], $key + 1);
+ }
+ }
+
+ public function testBuffer_ForEachTwice_WithArrayIterator()
+ {
+ $resultSet = $this->getMockForAbstractClass('Zend\Db\ResultSet\AbstractResultSet');
+ $resultSet->initialize(new \ArrayIterator(array(
+ array('id' => 1, 'name' => 'one'),
+ array('id' => 2, 'name' => 'two'),
+ array('id' => 3, 'name' => 'three'),
+ )));
+ $resultSet->buffer();
+
+ foreach($resultSet as $key=>$value) {
+ $this->assertEquals($value['id'], $key + 1);
+ }
+
+ foreach($resultSet as $key=>$value) {
+ $this->assertEquals($value['id'], $key + 1);
+ }
+ }
+
/**
* @covers Zend\Db\ResultSet\AbstractResultSet::isBuffered
*/
@@ -179,6 +217,7 @@ public function testRewind()
array('id' => 3, 'name' => 'three'),
)));
$this->assertNull($resultSet->rewind());
+ $this->assertEquals(array('id' => 1, 'name' => 'one'), $resultSet->current());
}
/**
27 tests/ZendTest/Db/ResultSet/HydratingResultSetTest.php
View
@@ -67,4 +67,31 @@ public function testToArray()
$obj = $hydratingRs->toArray();
$this->assertInternalType('array', $obj);
}
+
+ public function testSameCurrentForIteratorData()
+ {
+ $hydratingRs = new HydratingResultSet;
+ $hydratingRs->initialize(new \ArrayIterator(array(
+ array('q1'),
+ )));
+ $hydratingRs->rewind();
+ $this->assertSame(
+ $hydratingRs->current(),
+ $hydratingRs->current()
+ );
+ }
+
+ public function testSameCurrentForIteratorDataAndBuffering()
+ {
+ $hydratingRs = new HydratingResultSet;
+ $hydratingRs->initialize(new \ArrayIterator(array(
+ array('q1'),
+ )));
+ $hydratingRs->rewind();
+ $hydratingRs->buffer();
+ $this->assertSame(
+ $hydratingRs->current(),
+ $hydratingRs->current()
+ );
+ }
}
41 tests/ZendTest/Db/ResultSet/ResultSetIntegrationTest.php
View
@@ -16,7 +16,7 @@
use stdClass;
use Zend\Db\ResultSet\ResultSet;
-class ResultSetTest extends TestCase
+class ResultSetIntegrationTest extends TestCase
{
/**
* @var ResultSet
@@ -29,7 +29,6 @@ class ResultSetTest extends TestCase
*/
protected function setUp()
{
-
$this->resultSet = new ResultSet;
}
@@ -41,18 +40,25 @@ public function testRowObjectPrototypeIsPopulatedByRowObjectByDefault()
public function testRowObjectPrototypeIsMutable()
{
- $row = new \ArrayObject();
+ $row = new ArrayObject();
$this->resultSet->setArrayObjectPrototype($row);
$this->assertSame($row, $this->resultSet->getArrayObjectPrototype());
}
- public function testRowObjectPrototypeMayBePassedToConstructor()
+ public function testRowObjectPrototypeMayBePassedToConstructorWithDeprecatedArgument()
{
- $row = new \ArrayObject();
+ $row = new ArrayObject();
$resultSet = new ResultSet(ResultSet::TYPE_ARRAYOBJECT, $row);
$this->assertSame($row, $resultSet->getArrayObjectPrototype());
}
+ public function testRowObjectPrototypeMayBePassedToConstructor()
+ {
+ $row = new ArrayObject();
+ $resultSet = new ResultSet($row);
+ $this->assertSame($row, $resultSet->getArrayObjectPrototype());
+ }
+
public function testReturnTypeIsObjectByDefault()
{
$this->assertEquals(ResultSet::TYPE_ARRAYOBJECT, $this->resultSet->getReturnType());
@@ -208,4 +214,29 @@ public function testCallingBufferAfterIterationThrowsException()
$this->resultSet->buffer();
}
+ public function testSameCurrentForIteratorData()
+ {
+ $this->resultSet->initialize(new \ArrayIterator(array(
+ array('q1'),
+ )));
+ $this->resultSet->rewind();
+ $this->assertSame(
+ $this->resultSet->current(),
+ $this->resultSet->current()
+ );
+ }
+
+ public function testSameCurrentForIteratorDataAndBuffering()
+ {
+ $this->resultSet->initialize(new \ArrayIterator(array(
+ array('q1'),
+ )));
+ $this->resultSet->rewind();
+ $this->resultSet->buffer();
+ $this->assertSame(
+ $this->resultSet->current(),
+ $this->resultSet->current()
+ );
+ }
+
}
Something went wrong with that request. Please try again.