DateTime Precision is lost when adding date to model factories

[jamiehd]

• Laravel Version: v5.6.17
• PHP Version: 7.1.17

Description:

When using a model factory with a fixed date the microseconds are lost. This means when asserting in a unit test dates do not match when they should.

Steps To Reproduce:

public function some_test() {
        Carbon::setTestNow(new Carbon());
        $t = factory(Model::class)->make(['date_field' => Carbon::now()->addWeeks(2)]);
        $this->assertEquals($t->date_field, Carbon::now()->addWeeks(2));
}

Failed asserting that two DateTime objects are equal.
--- Expected
+++ Actual
@@ @@
-2018-05-28T21:53:00.000000+0000
+2018-05-28T21:53:00.613965+0000

[laurencei]

When using a model factory with a fixed date the microseconds are lost.

This is expected behavior though, isnt it? A database timestamp does not have microseconds?

This means when asserting in a unit test dates do not match when they should.

Your assertEquals() would still fail - because 0.00003 microseconds has past between the two line items, so they would never be identical in that way?

[sisve]

@laurencei We support specifying fractional seconds in datetime fields since #18847. The SQL standard uses 6 digits for this, per default, while mysql uses 0.

Note that the test calls Carbon::setTestNow(...). This means that further calls to Carbon::now() in the test will return a specific instance, and there will be no microsecond difference in the values.

[laurencei]

@sisve - yep, your right.

The issue seems to be someone due to the Carbon change in v1.22.

v1.21 works correctly - I can make the test pass. Switching to v1.22 causing the situation above.

I found this: briannesbitt/Carbon#1234

I had a quick look and I cant work out why it's doing it though - as both use Carbon under the hood. I'll have a play later on if I get time.

[sisve]

Carbon 1.22 was severely broken in many ways. Stay away from it. briannesbitt/Carbon#863

[laurencei]

@sisve - yes, but it's not a break - it's actually an improvement in percission due to a PHP7 bug.

The issue flows all the way forward to the latest version, and was an intended change.

The strange thing is why it's dropping percission when going through the factory - I dont see an obvious reason why - so I'm going to dig in further.

[staudenmeir]

The microseconds are removed by HasAttributes::setAttribute(), which uses the database connection's default date format. With the exception of SQL Server, this is always Y-m-d H:i:s.

You can specify a custom format in your model:

protected $dateFormat = 'Y-m-d H:i:s.u';

[tillkruss]

Is this a Carbon or Laravel issue?

[laurencei]

@tillkruss - it's an intendend change in Carbon behaviour, and so that's going to make it Laravel's problem on the way its implemented...

I dont see an obvious fix though, because it would have to be a change in default behaviour in Laravel - which we might not want.

I'd leave this open and see if someone has a good solution for it...

[sisve]

I do not see how this is related to the linked Carbon issue. They are talking about how the some calls to static methods now have a microsecond value; but that is the expected behavior in this issue. Here the problem is that the microseconds disappears.

I can reproduce this using the mysql grammar, so this does not depend on a mssql installation. I got stuck on reproducing this until I realized I had to add the field to the $dates array. This means that HasAttributes::setAttribute will convert the provided value into a string using the current query grammars; and no query grammars support microsecond precision.

framework/src/Illuminate/Database/Eloquent/Concerns/HasAttributes.php

Lines 551 to 556 in ca75ec8

	// If an attribute is listed as a "date", we'll convert it from a DateTime
	// instance into a form proper for storage on the database tables using
	// the connection grammar's date format. We will auto set the values.
	elseif ($value && $this->isDateAttribute($key)) {
	$value = $this->fromDateTime($value);
	}

class MyModel extends Model {
    protected $dates = [ 'date_field' ];
}

$value = Carbon::now();
$model = new MyModel();
$model->date_field = $value;
dd(array(
    'before' => $value,
    'after' => $model->date_field
));

// array:2 [▼
//     "before" => Carbon {#3097 ▼
//         +"date": "2018-05-15 17:15:48.768878"
//         +"timezone_type": 3
//         +"timezone": "UTC"
//     }
//     "after" => Carbon {#3117 ▼
//         +"date": "2018-05-15 17:15:48.000000"
//         +"timezone_type": 3
//         +"timezone": "UTC"
//     }
// ]

[laurencei]

I do not see how this is related to the linked Carbon issue. They are talking about how the some calls to static methods now have a microsecond value; but that is the expected behavior in this issue. Here the problem is that the microseconds disappears.

The problem is we have always been dropping the microseconds. But it was never noticed or an issue, because Carbon was also dropping the microseconds.

Now Carbon has changed behaviour and passes them through - highlighting the issue.

It's not a bug, - its a change in behaviour by Carbon - and now it affects Laravel downstream.

[laurencei]

Technically we are no worse off than before. We've always dropped the microseconds - its not a new bug.

The issue is Carbon tests will just "highlight" the issue now.

So it would seem like something worth targeting for 5.7 to change and incorporate microseconds (if at all).

[staudenmeir]

It looks like we can't really support microseconds on MySQL at the moment.

Saving works, but fetching doesn't: There is a four-year-old bug that removes microseconds when using PDO::ATTR_EMULATE_PREPARES => false (which Laravel does by default).

[tillkruss]

Can this be closed then?

[antonkomarev]

Thats the only reason why I've started to use integers when I need to store datetime with microseconds.

$model->setAttribute('cursor', Carbon::now()->format('Uu'));

[antonkomarev]

It will be great if everybody who experienced this issue will vote in PHP Bug №67122. It doesn't require registration and possibly will add importance for this bug.

[laurencei]

@tillkruss - i think it needs to stay open as it's going to come up as people upgrade their apps and run into this issue...

And leaving it open might give other people a chance to come up with a creative solution...

[antonkomarev]

@laurencei I've found possible solution of this issue and posted it to the PHP bug tracker, but sadly I don't have an access to propose these changes directly to the PHP sources.

[antonkomarev]

Here is a PR which should fix this issue: php/php-src#3245

[sisve]

@a-komarev That PR may fix some related problems, but that PR alone will not fix this issue. Laravel still formats date fields using the query grammar's date format, and it doesn't support milli-/microseconds.

[antonkomarev]

There is a lot of progress with PHP bugfix: php/php-src#3257
It could be released in PHP 7.3 alpha versions.

[strongholdmedia]

@sisve I am not totally sure of that being true.
I am uncertain about the factory helper stuff, but we definitely use Laravel/Eloquent with timestamps and Carbon and it definitely produced an issue exactly like this one in appearance until we turned PDO::ATTR_EMULATE_PREPARES on.

[sisve]

@strongholdmedia Look at my reproduction just a few posts up. It doesn't do database calls at all, just creates an Eloquent mode, set's a date field and reads it back again. #24214 (comment)

[strongholdmedia]

Ah, yes, now I remember.
There is also an issue with Carbon, IIRC.
Something along the "format specifier for microseconds does not work like Laravel uses it" line.

[gitBC]

Would love to have full support of fractional seconds as well, this would include ANY date/time and timestamp methods. My use case is to fetch by created_at where there are several records a second added. Something along these lines:

public function mostRecentEvent()
    {
        return Event::all()
            ->orderBy('occurred_at', 'desc')
            ->limit(1)->first();
    }

I've tried a few permutations of modifying the $cast and $dates and creating custom mutators and even overriding the Model::asDateTime() functions to no avail

[driesvints]

Seems like the php/php-src#3257 was merged for the 7.3 release. If anyone can verify if this is fixed for the latest 7.3 RC that'd be great.

I'm going to close this because this seems to be related to PHP rather than Laravel. People can still find this issue in the tracker.

[staudenmeir]

@driesvints Yes, this is working in PHP 7.3.

[strongholdmedia]

Glad it finally turned out to be the same issue.
Whoever can not wait for the final 7.3 can turn on emulated prepares.
(Just remember to turn it (back) off afterwards.)