$options['limit'] breaking find() query.

[Zelfapp]

This one has me baffled. We've spent about 6 hours today trying to understand what we're doing wrong when using limit.

The following code works and returns 4 documents. For this particularly find there are only 4 documents that would ever match:

$filter = [
    'cancel_reason' => null,
    'financial_status' => ['$ne' => 'pending'],
    'tags' => new MongoDB\BSON\Regex("kiosk", 'i'),
    'created_at' => ['$gte' => '2016-03-01T03:00:00-04:00', '$lte' => '2016-04-01T02:59:59-04:00'],
    '$or' => [
        [
        'tags' => new MongoDB\BSON\Regex("Fort Belvoir VA", 'i')
        ]
    ]
];

$options['skip'] = 0;
$options['limit'] = 4;

$cursor = $collection->find($filter, $options);

If I change limit to 3 like below, no documents are returned the script hangs and then dies with zero php errors and no exceptions being thrown:

$options['limit'] = 3;

[jmikola]

@Zelfapp: Can you reproduce this using the extension directly (e.g. Query::__construct()). Also, if you could share the relevant phpinfo() for the extension and some sample data for the collection (perhaps just five documents, including the four that would match and one additional document), we can investigate on our end.

[Zelfapp]

Here's 5 documents for testing. 4 will match query. My environment is below. I tested using Query::__construct() and received the same results.

{
    "created_at" : "2016-03-21T11:18:57-07:00",
    "financial_status" : "paid",
    "cancel_reason" : null,
    "tags" : "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking"
}

{
    "created_at" : "2016-03-19T13:22:10-07:00",
    "financial_status" : "paid",
    "cancel_reason" : null,
    "tags" : "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking"
}

{
    "created_at" : "2016-03-19T07:55:44-07:00",
    "financial_status" : "paid",
    "cancel_reason" : null,
    "tags" : "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking"
}

{
    "created_at" : "2016-03-04T13:08:53-08:00",
    "financial_status" : "paid",
    "cancel_reason" : null,
    "tags" : "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking"
}

{
    "created_at" : "2016-03-01T07:35:50-08:00",
    "financial_status" : "paid",
    "cancel_reason" : null,
    "tags" : "1188, Fort Belvoir  VA, kiosk, Processing-Picking"
}

Environment
Windows 10
Wamp
PHP 5.5.12
Mongodb driver 1.1.5
mongodb php library 1.0.2. Installed using composer.
Mongo instance version 2.4.6

[jmikola]

I cannot reproduce this with the following extension test case:

--TEST--
PHPLIB-153: $options['limit'] breaking find() query
--SKIPIF--
<?php require __DIR__ . "/../utils/basic-skipif.inc"; ?>
--FILE--
<?php
require_once __DIR__ . "/../utils/basic.inc";

$bulk = new MongoDB\Driver\BulkWrite;
$bulk->insert([
    "created_at" => "2016-03-21T11:18:57-07:00",
    "financial_status" => "paid",
    "cancel_reason" => null,
    "tags" => "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking",
]);
$bulk->insert([
    "created_at" => "2016-03-19T13:22:10-07:00",
    "financial_status" => "paid",
    "cancel_reason" => null,
    "tags" => "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking",
]);
$bulk->insert([
    "created_at" => "2016-03-19T07:55:44-07:00",
    "financial_status" => "paid",
    "cancel_reason" => null,
    "tags" => "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking",
]);
$bulk->insert([
    "created_at" => "2016-03-04T13:08:53-08:00",
    "financial_status" => "paid",
    "cancel_reason" => null,
    "tags" => "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking",
]);
$bulk->insert([
    "created_at" => "2016-03-01T07:35:50-08:00",
    "financial_status" => "paid",
    "cancel_reason" => null,
    "tags" => "1188, Fort Belvoir  VA, kiosk, Processing-Picking",
]);

$manager = new MongoDB\Driver\Manager(STANDALONE);
$manager->executeBulkWrite(NS, $bulk);

$query = new MongoDB\Driver\Query(
    [
        'cancel_reason' => null,
        'financial_status' => ['$ne' => 'pending'],
        'tags' => new MongoDB\BSON\Regex("kiosk", 'i'),
        'created_at' => ['$gte' => '2016-03-01T03:00:00-04:00', '$lte' => '2016-04-01T02:59:59-04:00'],
        '$or' => [
            ['tags' => new MongoDB\BSON\Regex("Fort Belvoir VA", 'i')],
        ],
    ],
    [
        'limit' => 3,
        'skip' => 0,
        'projection' => ['_id' => 0],
    ]
);

$cursor = $manager->executeQuery(NS, $query);

var_dump($cursor->toArray());

?>
===DONE===
<?php exit(0); ?>
--EXPECTF--
array(3) {
  [0]=>
  object(stdClass)#%d (%d) {
    ["created_at"]=>
    string(25) "2016-03-21T11:18:57-07:00"
    ["financial_status"]=>
    string(4) "paid"
    ["cancel_reason"]=>
    NULL
    ["tags"]=>
    string(66) "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking"
  }
  [1]=>
  object(stdClass)#%d (%d) {
    ["created_at"]=>
    string(25) "2016-03-19T13:22:10-07:00"
    ["financial_status"]=>
    string(4) "paid"
    ["cancel_reason"]=>
    NULL
    ["tags"]=>
    string(66) "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking"
  }
  [2]=>
  object(stdClass)#%d (%d) {
    ["created_at"]=>
    string(25) "2016-03-19T07:55:44-07:00"
    ["financial_status"]=>
    string(4) "paid"
    ["cancel_reason"]=>
    NULL
    ["tags"]=>
    string(66) "1188, FORT BELVOIR VA, kiosk, Processing-Kiosk, Processing-Picking"
  }
}
===DONE===

Note: I excluded the "_id" field in the projection as 2.4 allows it to be appended at the end of the document, while 2.6+ ensures it is serialized first. That said, I ran this test on minor versions of the database through 3.2 (with and without the "_id" field excluded) without issue -- I merely had to adjust the expected output when testing against 2.4. None of the versions resulted in a hang.

The only thing I can suggest is collecting output with the mongodb.debug INI setting enabled. You can use "stderr" if you're running this from the CLI or specify a writeable directory if it's being invoked from a web SAPI.

[Zelfapp]

I've attached the mongodb.debug error log PHO45F6.txt. In my CLI you can see it ends in "segmentation fault".

PHO45F6.txt

[jmikola]

Based on that trace, you're using version 1.1.4 of the library with bundled versions of libmongoc and libbson. 1.1.5 is the latest stable release and includes an important fix for Linux/OSX users. Please upgrade and see if that resolves the segfault.

If not, please follow-up with a GDB backtrace from 1.1.5 so we can diagnose this further.

[Zelfapp]

This bug or missing dll functionality has pretty much stopped our mongodb upgrade in it's tracks. We've tried and tried to get gdb running on windows. The best we can do it seems is strace.

Attached strace.txt:
strace.txt

I have the latest windows dll installed from: https://pecl.php.net/package/mongodb/1.1.5/windows. It is running libbson version 1.13, libmongoc version 1.12 as you can see in screenshot below.

Also, we tested against Mongo instance mongod version: 2.6.12 and mongod version: 3.0.11 and received the same segfault results. Loving windows right now.

[Zelfapp]

@jmikola You might not have seen my last answer. We're a bit stuck. Any more ideas for us?

[jmikola]

Unfortunately, that strace output isn't helpful at pinpointing the cause of the segfault, which is why we'd need a proper backtrace to get to the heart of it. Based on the mongodb.debug output you attached earlier, the request looks successful. We see connections initialized, some documents returned via the query, and then resources cleaned up and the socket left persisted for the subsequent request(s).

[Zelfapp]

We've tested on several different Windows 10 PC's at our workplace thinking it might be one PC, but all PC's have this seg fault failure. Now what?? We spent about 5 hours trying to get gdb running on blasted windows, but we could not get it to interrupt. There's a lot of other people complaining about that with no solutions that worked for us. No windows environments you guys could test on at Mongo? We're dying for this upgrade or we need to figure a different data store I guess. Would love to provide any more info I can, but getting gdb running on windows right now seems impossible to me having never used it before.

[jmikola]

@bjori pointed me to https://bugs.php.net/bugs-generating-backtrace-win32.php, which has instructions for generating backtraces on Windows systems (with and without compilers). Is that more helpful?

I extrapolated the test script above into a stand-alone PHP file: https://gist.github.com/jmikola/240af945eb396c8da46d2224571e7d98. You'll need to edit the defined constants atop the file to customize the server, database, and collection.

I'm currently traveling and won't have access to our Windows build machine (Windows 7) until Monday. Once I'm back, I can test the above script as well. We currently do not have a Windows 10 environment for testing the driver.

[Zelfapp]

The segmentation fault as it turns out IS NOT triggered when we run your simple script as a standalone with options limit set to 3.

When we try to run your script in our Codeigniter 3.0.6 framework (https://github.com/Zelfapp/ci_mongo/blob/master/application/controllers/Mongodriver.php) it is triggered every time. (You can get repo here for testing inside/outside framework: https://github.com/Zelfapp/ci_mongo)

You can see here that a CI developer bcit-ci/CodeIgniter#4574 suggest the issue is not CI at all, and points back to more memory being used as the issue, and that this would happen on any framework that is using x amount of memory to trigger the issue basically, which sounds very plausible.

I don't think the dev performed any test using CI, so I'm back to square one that this is an issue with the library, driver, or windows, etc.

Thanks for your help. I'm still trying to get the debug trace working as well when running inside CI framework.

[pix0id]

Hello,

Using the repo Zelfapp posted, I performed the same test on a mac, and am getting the same results.

[jmikola]

@pix0id: If you're able to reproduce the segfault on OSX, can you supply a backtrace?

@Zelfapp: Thanks for the application repo. I'm installing the PHP 5.5 version of WAMP server on Windows 7 to test this now.

[jmikola]

@Zelfapp: I believe I'm able to reproduce the same error with your application repository. I installed WAMP with PHP 5.5, MongoDB 2.4.14. and the same extension and library versions you cited above.

While I don't see any evidence of a segfault in PHP logs, the Apache error log does report httpd.exe being restarted with a new worker pool. There is also evidence of a crash in Windows' event logs (access through the "Computer Management" app in Control Panel).

Through some trial and error, this appears to be related to the Cursor object being garbage collected. I was able to avoid the segfault by manually unsetting the $cursor object in your controller function after iteration (i.e. $cursor->toArray()). If that works for you, I would suggest it until we can get around to properly diagnosing and fixing this issue.

[Zelfapp]

unset($cursor); does the trick.

Really appreciate you digging into this and identifying a workaround.

[jmikola]

Thanks for your patience in diagnosing this!

I was able to generate a backtrace using PHP's built-in web server on Linux. Please follow PHPC-671 for further updates. I'll swing back to this thread once a fix is on the way.

[jmikola]

This has been fixed in ext-mongodb 1.1.7.