Aggregation Framework isDead true

[Roycohen]

Hi Guys,

I had an aggregation pipeline which worked in the old Mongo driver.
I saw that there is no change between the 2 drivers, however the pipeline is not returning data.

The 'match' stage includes MongoRegex which I've changed to 'MongoDB\BSON\Regex' and MongoDate in unix timestamp which I've change to MongoDB\BSON\UTCDateTime in milliseconds.

2 question:

1. Does MongoDB\BSON\UTCDateTime needs to get time in milliseconds or can I send it php unix timestamp which is in seconds? I think it is currently only working with milliseconds.
2. The most important one: Why does it my pipeline is not working. The same pipeline works in the old driver. BTW, When the result is returned I always see isDead=true.

I really need your help guys.

[lindelius]

The constructor for MongoDB\BSON\UTCDateTime does indeed require the time to be in milliseconds, and not in seconds. If you only have the time in seconds (UNIX timestamp), you can just multiply it by 1000 when you create the UTCDateTime object.

$date = new UTCDateTime($timestamp * 1000);

Could you please post the actual code that you use when calling the aggregate() function, including the entire pipeline.

[Roycohen]

aggregation-pipeline.txt
Uploaded.
I've attached pipeline, result and options.

Please let me know what can I do.
I must understand what is not working
aggregation-pipeline.txt

One of the file has the result as well.

[moderndeveloperllc]

Also note that the forthcoming 1.2 version of the driver will allow you to set a DateTimeInterface object (DateTime or DateTimeImmutable) as the instantiation parameter for MongoDB\BSON\UTCDateTime. It does the math on the DateTime object for you.

use MongoDB\BSON\UTCDateTime;

$dateTime = (new DateTime())->setTimestamp($timestamp);
$mongoDate = new UTCDateTime($dateTime);

[Roycohen]

Thanks!!
multiplying it by 1000 :).
I'm doing something similar in my app. I have a class which gets anything and returns a MongoDB\BSON\UTCDateTime in the right format.

Can you tell me why my pipeline is not working and why the result object is dead.

[moderndeveloperllc]

@Roycohen When you iterate over the cursor, you get no results? That's one big change between drivers - you now get an iterable Cursor object as opposed to an array of results. Per the docs:

A cursor being "dead" does not mean that it cannot be iterated further. The database sends responses in batches of documents (up to 4MB of documents per response). The client's cursor may still have results to be iterated even if the server has no more batches to send.

From a pure performance point of view, I would question the need to search for the entire ID using regex expressions. But I'm guessing that IDs aren't the only thing your code can put into this pipeline.

[jmikola]

@Roycohen: Both of the text files you attached have extra formatting. Can you either re-attach these as plain text or simply embed the content as a fenced code block within your comment?

Since it hasn't been discussed above, I should remind you that MongoRegex::__construct() and MongoDB\BSON\Regex::__construct() take their arguments in different forms.

[moderndeveloperllc]

@jmikola Just an FYI if you want to see them now, make the extension .rtf and they open nicely. The second is all you need I think as the returned cursor dump includes the command from the first file.

[Roycohen]

Hi Guys,

I know Regex is an ass, and I should avoid it. Thanks.
The Regex is built in the new format. Otherwise you'll get an exception.
MongoDB\BSON\Regex("/$id/", "i")

@jmikola, whats @moderndeveloperllc suggest indeed makes the file readable. If it still doesn't work for you, just let me know.

@moderndeveloperllc, I know it is a cursor. Before I've user aggregateCursor so it is the same.
I'm iterating over this object but never have results.
I'm using read preference.

$readPreference = new \MongoDB\Driver\ReadPreference(\MongoDB\Driver\ReadPreference::RP_SECONDARY_PREFERRED);
try {
    $result = $this->_statsMongo->{$aggrCollection}->aggregate(
        $pipeline,
        array(
            'allowDiskUse' => true,
            'batchSize' => 100,
            'explain' => true,
            'readPreference' => $readPreference
        )
    );
}

Really need help in this. Something is not working there.

[jmikola]

MongoDB\BSON\Regex("/$id/", "i")

MongoDB\BSON\Regex::__construct() does not expect the pattern to be wrapped with delimiters. MongoRegex used delimiters because it took the pattern and flags in a single string. I expect that's your problem.

[Roycohen]

Thanks..
Funny I've tried to remove it and it didn't worked.
Now when I ran it, it is working.

Any how, I'll keep you updated with more things like this.
Now I have a strange phenomena which I don't know whether it is related to this library.

[Roycohen]

Now I have a very strange behavior. Don't know if it is because PHP7 or because pf the driver but I suspect 99.9% it is something in the driver.
I'm iterating over the result cursor from the aggregation pipeline and I have the following structure in each returned doc:
array with 2 fields:

_id => array(
'date' => MongoDB\BSON\UTCDateTime
)
'value' => array(
'x' => 1,
'y' => 2,
...
)

And I want to get the date as timestamp so I'm using this function:

static public function getTimestampFromMongoDate($mongoDate, $usec = false)
    {
//        var_dump($mongoDate);
        if (is_object($mongoDate) && is_a($mongoDate, 'MongoDate')) {
            return $mongoDate->sec;
        } else if (is_object($mongoDate)
            && is_a($mongoDate, 'MongoDB\BSON\UTCDateTime')
            && isset($mongoDate->milliseconds)
        ) {
            if ($usec) {
                return $mongoDate->milliseconds;
            } else {
                return intval($mongoDate->milliseconds/1000);
            }
        } else if (is_numeric($mongoDate)) {
            return intval($mongoDate);
        } else {
            return null;
        }
    }

Now the phenomena...
When I'm stopping in xDebug on the place where I'm using the above function the timestamp is returned good.
When I'm running it without debug the timestamp is 0 or empty.
If I'm adding a var_dump to the beginning of the 'getTimestampFromMongoDate' function, it is working as expected without debug... But obviously I can't do that :)

I can show you this phenomena in Remote Access if you want to see this in action.
I know it might not make sense but it is happening to me, and really appreciate your help on this guys.

[lindelius]

MongoDB\BSON\UTCDateTime doesn't have any public properties, so isset($mongoDate->milliseconds) is always going to return false.

When you pass a MongoDB\BSON\UTCDateTime object to this function, the returned value will always be null. You'll want to rewrite your function to something like this:

static public function getTimestampFromMongoDate($mongoDate, $usec = false)
{
    if ($mongoDate instanceof MongoDate) {
        if ($usec) {
            return intval($mongoDate->sec . $mongoDate->usec);
        } else {
            return $mongoDate->sec;
        }
    }

    if ($mongoDate instanceof MongoDB\BSON\UTCDateTime) {
        if ($usec) {
            return intval((string) $mongoDate);
        } else {
            return (intval((string) $mongoDate) / 1000);
        }
    }

    return null;
}

[Roycohen]

OK Great, thanks, it seems to work.
There is only one question that I would love to get an answer.
Why when I stop in debug or var_dump the object before using it as I used it before, it worked?

[lindelius]

I don't know what you mean by "worked as I used it before", but if you var_dump() an instance of MongoDB\BSON\UTCDateTime it outputs something like this

object(MongoDB\BSON\UTCDateTime)#2 (1) {
    ["milliseconds"]=>
    string(13) "1470580534000"
}

The milliseconds property that is displayed is an internal property, so even though it's there when you dump it out, you can't access it. I don't know if this answers the question as I don't know how you used it before, or how it "worked" when you debugged it, but If it doesn't, please reply with an example and an explanation of what seems to work when you are debugging it, and I'll have a look.

[Roycohen]

Thanks @lindelius.
First of all, your solution to cast the 'MongoDB\BSON\UTCDateTime' is working.
Now for my question. I don't know if it is related to the Library, but I would like to hear your thoughts.

My 'getTimestampFromMongoDate' function that posted above didn't worked if I used it as is.
But if you would take a look in the function, in the beginning of it I commented out var_dump($mongoDate);. If I'm un-commenting this line and execute my function then the function returns the right result and the result isn't NULL. if I'm commenting the line back then the result is as you said, NULL.

So my Question is, how can you explain this behavior?
Hope you understand what I meant.