Skip to content

Loading…

Oci8 Driver generating "Fetch out of sequence warning" #3958

Closed
wants to merge 1 commit into from

5 participants

@philipobenito

Zend\Db\Adapter\Driver\Oci8\Result line 151 is generating a oci_fetch_assoc(): ORA-01002: fetch out of sequence warning for any larger queries. Limiting the results to less than 50 stops the warning appearing but anything larger it appears.

Suppressing warnings with @oci_fetch_assoc get's rid of the warning and returns results as expected but obviously this isn't fixing the issue.

The warning is usually generated when a fetch is performed after a commit.

@hamilok

@oci_fetch_assoc classic workaround

@bartmcleod
@philipobenito
@bartmcleod
@philipobenito
@hamilok

A collegue did a similar query with traditional php oci, he did not get the warning, do you know what might be the difference?

it exceeded the limit prefetch value?

@philipobenito

I don't think that's the issue as running the query in procedural PHP would have the same prefetch limit and that doesn't throw any warnings.

@hamilok

works for me

@philipobenito
@ralphschindler
Zend Framework member

Is there a way someone could help me reproduce this? Perhaps with adding some large amount of data to the schema in https://github.com/ralphschindler/Zend_Db-Examples, then a script to reproduce the error that I can run?

@philipobenito
@ralphschindler
Zend Framework member

That's fine - whenever you can. Thanks, @happyaccidents

@cjbj

Turning on oci_internal_debug(1) in the .php code may help to identify what OCI8 calls are being used. If you can recompile PHP OCI8, the patch below will print out SQL statements and whether the execute mode is autocommit or not:

diff --git a/ext/oci8/oci8_statement.c b/ext/oci8/oci8_statement.c
index 89facb0..ef5dc67 100644
--- a/ext/oci8/oci8_statement.c
+++ b/ext/oci8/oci8_statement.c
@@ -52,6 +52,11 @@ php_oci_statement *php_oci_statement_create (php_oci_connection *connection, cha
    if (!query_len) {
        /* do not allocate stmt handle for refcursors, we'll get it from OCIStmtPrepare2() */
        PHP_OCI_CALL(OCIHandleAlloc, (connection->env, (dvoid **)&(statement->stmt), OCI_HTYPE_STMT, 0, NULL));
+   } else {
+       if (OCI_G(debug_mode)) {
+           php_printf ("OCI8 DEBUG SQL: \"%s\" at (%s:%d) \n", query, __FILE__, __LINE__);
+       }
+
    }
            
    PHP_OCI_CALL(OCIHandleAlloc, (connection->env, (dvoid **)&(statement->err), OCI_HTYPE_ERROR, 0, NULL));
@@ -421,6 +426,9 @@ int php_oci_statement_execute(php_oci_statement *statement, ub4 mode TSRMLS_DC)
        case OCI_DESCRIBE_ONLY:
        case OCI_DEFAULT:
            /* only these are allowed */
+           if (OCI_G(debug_mode)) {
+               php_printf ("OCI8 DEBUG: Execute mode 0x%x at (%s:%d) \n", mode, __FILE__, __LINE__);
+           }
            break;
        default:
            php_error_docref(NULL TSRMLS_CC, E_WARNING, "Invalid execute mode given: %d", mode);

@bartmcleod

@ralphschindler Were you able to reproduce this?

@ralphschindler
Zend Framework member

I've not been given a reproduction case yet. ping @happyaccidents

@hamilok

This error may occur if you perform a FETCH on an active cursor after all records have been fetched. The option to resolve this error are:

Zend/Db/Adapter/Driver/Oci8/Result.php ( Updated)

    protected $currentData = null;

    public function rewind()
    {
        if ($this->position > 0) {
            throw new Exception\RuntimeException(
                'Oci8 results cannot be rewound for multiple iterations');
        }
    }

    public function next()
    {
        ++$this->position;
    }

    public function valid()
    {
        if ($this->currentData === false) {
            return false;
        }

        $this->currentData = oci_fetch_assoc($this->resource);

        return ($this->currentData !== false);
    }

    public function current()
    {
        return $this->currentData;
    }

    public function key()
    {
        return $this->position;
    }
@philipobenito
@philipobenito

This issue is being caused by 1 too many calls being sent to Zend\Db\Adapter\Driver\Oci8\Result::loadData but this only happens when the data set is above a certain size. A temporary fix is here 5684b17918b167317cb54fcd317e3ae92d3a11ca but I am trying to trace where it is decided whether to make a call or stop. @ralphschindler could you shed any light on this? Still working on a large schema for you but it's taking longer than I planned.

@hamilok

You can reproduce this situation with a small set of data ( such as 5 records ) and the set option oci8.default_prefetch=3

The current implementation is too many calls to loadData function. This makes it difficult to read and understand the code.

Sorry for my english

@ralphschindler
Zend Framework member

@hamilok or @cjbj do either of you have documentation on the default_prefetch setting? I don't get much from the php.net/manual.

@cjbj changing the prefetch, does that affect how one might code? It seems like it should be a driver level optimization only?

@philipobenito

@ralphschindler The prefetch limit shouldn't have to be set by the driver though, it's still getting two false returns from oci_fetch_assoc(), it's just that if we're within the prefetch limit then it doesn't care, if we've exceeded prefetch it complains.

@ralphschindler
Zend Framework member

The only problem I see with @hamilok's patch is that it fails a common use case, which is getting a row from the result without iteration (consider this use case):

$result = $table->select(['id' => 5]);
if ($result) {
    $row = $result->current();
}

. In the current Result, the current looks like this:

    public function current()
    {
        if ($this->currentComplete == false) {
            if ($this->loadData() === false) {
                return false;
            }
        }

        return $this->currentData;
    }

The two primary use cases being: a) just call current to get the current data, or b) iterate using the iterate workflow to then call current to get a row.

@philipobenito

I've temporarily changed the check in loadData() to check if the previous call was false before calling oci_fetch_assoc(), this way, a single row should be fine as $currentData starts as null. My thoughts were that this check should be performed elsewhere though.

5684b17

@hamilok

@ralphschindler Excuse me, please. In my amendment is not considered a direct function call "current" on the resulting data set.

    public function current()
    {
+        if ($this->currentData === null) {
+            $this->valid();
+        }
        return $this->currentData;
    }

either the solution proposed @happyaccidents

either use an explicit calls

$result = $table->select(['id' => 5]);
if ($result->valid()) {
    $row = $result->current();
}
@ralphschindler ralphschindler Zend\Db
Oci Result fixed logic error resulting in sequence warning
b0fc3d5
@ralphschindler
Zend Framework member

Can you guys please test the patch in this PR? I believe it was a logic error. This works for me.

@cjbj

To expand on @happyaccidents comment about being within the prefetch
limit it doesn't care, I can explain the underlying effect of
oci8.default_prefetch in reproducing the bug.

Prefetching reduces network traffic and DB interrupts etc by fetching
multiple rows of data and buffering them in the Oracle libraries used
by PHP OCI8. See the Underground PHP and Oracle Manual for a picture
(p. 174) of how prefetching works.

The design premise is that when oci_fetch_assoc() returns false, then
there is no more data. The application should never call
oci_fetch_assoc() again after this so it will never see an ORA error.
Due to the bug in ZF2 it is calling oci_fetch_assoc() too many times
and it is seeing the ORA-01002. (This error comes from the DB side of
the network)

The reason the error doesn't always occur is an optimization in the
Oracle libraries. When all rows can be fetched in one round trip to
the database (i.e. when the oci8.default_prefetch value is greater
than the number of rows in the queried table) then the Oracle client
libraries know that there is no more data. So no more networks trips
are initiated. Since the DB server is not invoked then the DB will
never generate the ORA-01002 error. So repeated calls to
oci_fetch_assoc() will simply return false to indicate there is no
more data.

For cases where oci8.default_prefetch is less than the number of rows
in the query result set, the ORA-01002 is generated by the DB when
oci_fetch_assoc() initiates some subsequent network trip to the DB.

@philipobenito

@cjbj :+1: well explained

@philipobenito

@ralphschindler yes, the fix works fine

@weierophinney weierophinney added a commit that closed this pull request
@weierophinney weierophinney Merge branch 'hotfix/3958'
Close #3958
e3b6b6d
@weierophinney weierophinney added a commit that referenced this pull request
@weierophinney weierophinney Merge branch 'hotfix/3958' into develop
Forward port #3958
aa01d5a
@bartmcleod

Great this is fixed, works for me too.

@ghost Unknown pushed a commit that referenced this pull request
@weierophinney weierophinney Merge branch 'hotfix/3958'
Close #3958
d967176
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 9, 2013
  1. @ralphschindler

    Zend\Db

    ralphschindler committed
    Oci Result fixed logic error resulting in sequence warning
This page is out of date. Refresh to see the latest.
Showing with 2 additions and 5 deletions.
  1. +2 −5 library/Zend/Db/Adapter/Driver/Oci8/Result.php
View
7 library/Zend/Db/Adapter/Driver/Oci8/Result.php
@@ -145,14 +145,11 @@ public function current()
*/
protected function loadData()
{
- $this->currentComplete = false;
- $this->currentData = null;
-
+ $this->currentComplete = true;
$this->currentData = oci_fetch_assoc($this->resource);
if ($this->currentData !== false) {
$this->position++;
- $this->currentComplete = true;
return true;
}
return false;
@@ -192,7 +189,7 @@ public function rewind()
public function valid()
{
if ($this->currentComplete) {
- return true;
+ return ($this->currentData !== false);
}
return $this->loadData();
Something went wrong with that request. Please try again.