Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MySQL 5.6 Fractional Seconds #14359

Merged
merged 3 commits into from Mar 12, 2014
Merged

Conversation

@arthurnn
Copy link
Member

arthurnn commented Mar 12, 2014

Adding microseconds support for mysql 5.6
This should not affect mysql 5.5 so IMO we dont need a version check.

this is PR #8240 + tests and Changelog entry.

review @rafaelfranca @jeremy @tenderlove

miyagawa and others added 3 commits Nov 12, 2012
You might want to branch it to include this only for 5.6, but
passing these values to < 5.6 doesn't cause issues either.
rafaelfranca added a commit that referenced this pull request Mar 12, 2014
MySQL 5.6  Fractional Seconds
@rafaelfranca rafaelfranca merged commit 79ceae6 into rails:master Mar 12, 2014
@arthurnn arthurnn deleted the arthurnn:mysql2_56_franc_sec branch Mar 12, 2014
@Silex
Copy link

Silex commented May 13, 2014

Is there an installable rails version to benefit from this? I'm using rails 4.1.0

@arthurnn
Copy link
Member Author

arthurnn commented May 13, 2014

@Silex not actually, we will need to use Rails edge(master) for now, as this will be part of 4.2.x only

@Silex
Copy link

Silex commented May 13, 2014

@arthurnn: Thanks. It is really annoying not being able to store milliseconds from a time... Do you know if your patch also takes care about migrations? Right now I'm unable to create a DATETIME(6) even when using t.datetime :fetched_at, precision: 6 (by default it seems to use second resolution).

@arthurnn
Copy link
Member Author

arthurnn commented May 13, 2014

Yep, you should be able to, look at the migration on the tests in this PR, https://github.com/rails/rails/pull/14359/files#diff-95d3dfc38be7275845454d444b0826f9R678, so you should be able to do:

t.datetime : fetched_at, limit: 6
@Silex
Copy link

Silex commented May 13, 2014

Ah, limit it is! Thanks this works even on my current version.

@jeremy
Copy link
Member

jeremy commented Aug 17, 2014

Note that this causes undesirable rounding behavior on MySQL 5.6 with lower-precision temporal fields.

In 5.5, when you insert 2014-08-17 12:30:00.999999 the fractional part is ignored. In 5.6, it's rounded to 2014-08-17 12:30:01: http://bugs.mysql.com/bug.php?id=68760

This has undesirable consequences like breaking apps that rely on updated_at for optimistic locking and causing spurious unit test failures.

Ideally, we'd format the datetime string according to the precision of the datetime field. Provide microseconds, milliseconds, tens of seconds, etc. No more precision than needed.

@jeremy
Copy link
Member

jeremy commented Aug 17, 2014

@sodabrew
Copy link
Contributor

sodabrew commented Dec 1, 2014

Yay! I'm super glad this got merged at long last. I added the corresponding support to the mysql2 gem ages ago per Miyagawa's suggestion. Users will need mysql2 >= 0.3.12 for DATETIME and >= 0.3.17 for TIME with microseconds.

@MaxGabriel
Copy link

MaxGabriel commented Jan 3, 2015

@jeremy This is breaking our tests on Rails 4.2 like how you described. An INSERT is done with created_at 2015-01-02 23:23:17.716890, then a SELECT is done for rows with created_at BETWEEN '2014-12-26 23:23:17.829018' AND '2015-01-02 23:23:17.829087'.

On paper this looks like it should work, but MySQL stores the 23:23:17.xxxxxx as 23:23:18, so the SELECT doesn't find the row that was just inserted.

We're on MySQL 5.6.16, but the column isn't set to store fractional seconds. Was a pain to track down the issue because this causes non-deterministic test failures :(

@rafaelfranca
Copy link
Member

rafaelfranca commented Jan 3, 2015

@MaxGabriel I believe I just fixed this issue. Could you check 4-2-stable?

@MaxGabriel
Copy link

MaxGabriel commented Jan 3, 2015

@rafaelfranca That fixes it! Thanks very much 👍

@arthurnn
Copy link
Member Author

arthurnn commented Jan 5, 2015

❤️

@trobrock
Copy link

trobrock commented Feb 20, 2015

@rafaelfranca we are upgrading our app to 4.2.0 right now and are experiencing the issue that @MaxGabriel was seeing, our db doesnt store the microseconds, but the query generated looks like

SELECT `orders`.* FROM `orders` WHERE (`orders`.`charged_on` BETWEEN '2015-02-20 18:35:44.376956' AND '2015-02-20 18:35:44.376957')

which causes test failures for us.

@rafaelfranca
Copy link
Member

rafaelfranca commented Feb 20, 2015

@trobrock have you checked 4-2-stable?

@trobrock
Copy link

trobrock commented Feb 20, 2015

@rafaelfranca I saw your PR in the changelog at the 4.2.0 tag so I assumed it was in there, let me try the branch.

@trobrock
Copy link

trobrock commented Feb 20, 2015

@rafaelfranca looks like 4-2-stable fixed it, but according to this https://github.com/rails/rails/blob/v4.2.0/activerecord/lib/active_record/connection_adapters/mysql2_adapter.rb#L77 the fix should be in 4.2.0 that is on rubygems correct?

@rafaelfranca
Copy link
Member

rafaelfranca commented Feb 20, 2015

Not really. The fix was in another place.

@trobrock
Copy link

trobrock commented Feb 20, 2015

@rafaelfranca ahh, do you have a link to that fix so I can track when it makes it to the gem? Until then I'll use 4-2-stable, thanks for the help.

@rafaelfranca
Copy link
Member

rafaelfranca commented Feb 20, 2015

No, I don't have it. It will make into the gem in 4.2.1

@miyagawa
Copy link
Contributor

miyagawa commented Feb 20, 2015

@trobrock I believe the fix is #18067

@trobrock
Copy link

trobrock commented Feb 20, 2015

@rafaelfranca @miyagawa thanks for the help.

@sodabrew
Copy link
Contributor

sodabrew commented Feb 20, 2015

You will also need mysql2 gem >= 0.3.12 for DATETIME and >= 0.3.18 for TIME fields. (A bug in argument count caused us to fail to select milliseconds from a TIME field into Ruby Time object prior to the 0.3.18 release).
https://github.com/brianmario/mysql2/releases/tag/0.3.18

kamipo added a commit to kamipo/rails that referenced this pull request Feb 28, 2015
Common methods in both mysql adapters are should be added to
`AbstractMysqlAdapter`, but some methods had been added to
`Mysql2Adapter`. (8744632, 0306f82, rails#14359)

Some methods already moved from `Mysql2Adapter` to
`AbstractMysqlAdapter`. (rails#17601, rails#17998)

Common methods in both mysql adapters are remaining only the `explain`
method in `Mysql2Adapter`.
@dre-hh

This comment has been minimized.

Copy link

dre-hh commented on df5a38f Mar 5, 2015

passing these values to < 5.6 doesn't cause issues either.

Unfortunately it does. Let's consider following usecase. you insert some records, which recieve a created at.

 INSERT INTO `recent_contact_requests` (`requester_id`, `requestee_id`, `created_at`, `updated_at`) VALUES (2, 3, '2014-11-23 13:41:38.394797', '2014-11-22 13:41:38.394797')
INSERT INTO `recent_contact_requests` (`requester_id`, `requestee_id`, `created_at`, `updated_at`) VALUES (1, 3, '2014-11-24 13:41:38.394797', '2014-11-23 13:41:38.394797')
INSERT INTO `recent_contact_requests` (`requester_id`, `requestee_id`, `created_at`, `updated_at`) VALUES (1, 2, '2014-11-22 13:41:38.394797', '2014-11-24 13:41:38.394797')
RecentContactRequest.where('created_at < ?', 100.days.ago)

which will generate the query

SELECT `recent_contact_requests`.* FROM `recent_contact_requests` WHERE (created_at < '2014-11-24 13:41:38.394797')

So you would expect it to return the first 2 records but not the last one, because the it's created is exactly2014-11-24 13:41:38.39479 but not less than it.

However this is not what will happen on mysql below 5.6. Below 5.6 all the records will be stored without the microsecond timestamp. So the last record will be stored with created_at: "2014-11-22 13:41:38".

And for the comparison of dates mysql, will just do a lexical comparison. So 2014-11-22 13:41:38 is actually less than 2014-11-24 13:41:38.394797 and the query will return the last record also.

You might want to branch it to include this only for 5.6

I.d.k why, but we experience this behavior on 5.6 either. The timestamps have been truncated on insert.

This implementation of microseconds also ignores the fact, that a a precision was set manually.

You can define the precision of timestamps for db queries by setting Time::DATE_FORMATS[:db]. If you then pass a datetime like object to an AR query it will automatically respect this setting.

So

# set date format to have 3 places for microseconds
Time::DATE_FORMATS[:db] = '%Y-%m-%d %H:%M:%S.%3N'
#fire up a query
RecentContactRequest.where('created_at < ?', 100.days.ago)

will result in

SELECT `recent_contact_requests`.* FROM `recent_contact_requests` WHERE (created_at < '2014-11-24 13:41:38.394.394797')

See issue #19223

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

9 participants
You can’t perform that action at this time.