Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Switch date and time handling to use CPAN DateTime modules #409

Open
gdevenyi opened this issue Jun 14, 2019 · 12 comments
Open

Proposal: Switch date and time handling to use CPAN DateTime modules #409

gdevenyi opened this issue Jun 14, 2019 · 12 comments

Comments

@gdevenyi
Copy link
Contributor

It looks like based on #406 and #395 and maybe others that the time/date handling for snapshot naming has become a bit of a problem.

Perl offers a particularly useful module DateTime, for both handling dates and times, generating standard formatting, and doing math with these objects in a sensible and standards-compliant way. It also supports sorting and comparisons between DateTime objects in perl compatible ways for tests.

Short example, get the current time in ISO UTC format as a string, then re-parse that string back into a DateTime object and extract a bunch of sub-date details

use DateTime;
use DateTime::Format::ISO8601;

$mydatetime = DateTime->now()->iso8601();

print("$mydatetime \n");

$dt = DateTime::Format::ISO8601->parse_datetime( $mydatetime );

use DateTime;
use DateTime::Format::ISO8601;

$mydatetime = DateTime->now()->iso8601();

print("$mydatetime \n");

$dt = DateTime::Format::ISO8601->parse_datetime( $mydatetime );

$year   = $dt->year;
$month  = $dt->month;          # 1-12
 
$day    = $dt->day;            # 1-31
 
$dow    = $dt->day_of_week;    # 1-7 (Monday is 1)
 
$hour   = $dt->hour;           # 0-23
$minute = $dt->minute;         # 0-59
 
$second = $dt->second;         # 0-61 (leap seconds!)
 
$doy    = $dt->day_of_year;    # 1-366 (leap years)
 
$doq    = $dt->day_of_quarter; # 1..
 
$qtr    = $dt->quarter;        # 1-4

print("$year $month $day $dow $hour $minute $second $doy $doq $qtr\n");

What's the exact difference in time between now and ~ a year from now.

use DateTime;
use DateTime::Duration;

$today = DateTime->now();
$later = DateTime->new(
    year      => 2020,
    month     => 06,
    day       => 26,
    hour      => 1,
    minute    => 30,
    second    => 0,
    time_zone => 'UTC',
);

$dur = $later->subtract_datetime_absolute($today);
print($dur->in_units( 'seconds'));
@gdevenyi
Copy link
Contributor Author

gdevenyi commented Jun 14, 2019

@phreaker0
Copy link
Collaborator

@gdevenyi do you intend to port the entire code? The hard part will be detection the DST crossover and handle the duplicate snapshot accordingly.

@gdevenyi
Copy link
Contributor Author

@phreaker0 I don't yet use sanoid, only syncoid, so I'm not sure I can effectively work on dev of such a change, however I am interested in contributing that in the future.

Regarding the DST crossovers and such, I believe DateTime already intrinsically handles this if you use UTC (default), and use the built-in date/time comparisons available in DateTime (see https://metacpan.org/pod/DateTime#Ambiguous-Local-Times ).

This is one of the reasons I'm suggesting going this direction for the code.

@phreaker0
Copy link
Collaborator

I specially meant the handling of the duplicate hour in the DST crossover case, DateTime handles it but it needs to be detected nevertheless because a duplicate snapshot name is not possible :-) and the later should be suffixed so no hourly snapshot is lost.

@gdevenyi
Copy link
Contributor Author

I'm not sure what you mean here, if we use DateTime in UTC there is never a DST crossover because UTC doesn't have DST events. This would simplify checking further.

There's still the issue of leap seconds but thats a pretty tiny corner case...

@phreaker0
Copy link
Collaborator

sanoid currently supports UTC (which is the default for packages and recommend install instruction) and local timezones. In the later case DST crossover is possible and needs to be handled.

Local timezones are handy if for example you have an network file server and you are exposing snapshots directly so users can restore older versions themselves. Having the snapshot name/time in the local timezone makes it a lot easier because most non technical users have a hard time with utc :-)

@gdevenyi
Copy link
Contributor Author

Ah, okay, makes sense. This is also a pretty simple solution, we need to properly store the daylight savings status when storing localtime.

Take, for example, "eastern time" https://en.wikipedia.org/wiki/Eastern_Time_Zone

Standard Time (EST) when observing standard time (autumn/winter) are 5 hours behind Coordinated Universal Time (UTC−05:00).

Eastern Daylight Time (EDT), when observing daylight saving time (spring/summer), is 4 hours behind Coordinated Universal Time (UTC−04:00).

iso8601 mandiates a storage format for local time:
https://en.wikipedia.org/wiki/ISO_8601#Time_zone_designators

The offset from UTC is appended to the time in the same way that 'Z' was above, in the form ±[hh]:[mm], ±[hh][mm], or ±[hh].

So, as of right now, where I am, the localtime is:

use DateTime;
use DateTime::Format::ISO8601::Format;
 
my $format = DateTime::Format::ISO8601::Format->new();

my $dt = DateTime->now->set_time_zone( 'local' );
$mydatetime = $format->format_datetime($dt);

print("$mydatetime \n");
> perl localtime.pl
2019-06-18T16:55:15-04:00

Then internally, we do all our math in UTC, after using the DateTime parsers and conversion to UTC.

This will handle when daylight savings flips over, and guarantees that we never have identically named snapshots because we embed our timezone info.

@gdevenyi
Copy link
Contributor Author

I'll update PR #410 to add a localtime option to demonstrate this further.

@gdevenyi
Copy link
Contributor Author

I've updated PR #410 to demonstrate UTC vs localtime generation, here's the difference in how the datestamps are generated:

use DateTime;
use DateTime::Format::ISO8601::Format;
 
my $format = DateTime::Format::ISO8601::Format->new();

my $dt = DateTime->now->set_time_zone( 'local' );
$mydatetime = $format->format_datetime($dt);

$dt->set_time_zone('UTC');
$myutctime = $format->format_datetime($dt);

print("$mydatetime \n");
print("$myutctime \n");
> perl printtime.pl
2019-06-19T12:43:47-04:00 
2019-06-19T16:43:47Z

^As we can see here, when my timezone goes from EDT to EDT, the name of the snapshot will go from -04:00 to -05:00, preventing the double-naming. Both these formats are back-parsable from ISO8601 format using DateTime::Format::ISO8601->parse_datetime($string) so we can use DateTime to compare the difference between any two snapshots using reliable maths.

@phreaker0
Copy link
Collaborator

@gdevenyi I didn't think about changing the snapshot name format, this would make it really easy. Let's see if @jimsalterjrs is open for such a change.

@timkgh
Copy link

timkgh commented Nov 7, 2021

It's that time of the year when I'm reminded about this issue.

@putnam
Copy link

putnam commented Jan 27, 2022

Rather than create a new ticket, I just wanted to point out the datetime strings used in sanoid snaps vs. syncoid ephemeral snaps are oddly very different:

xyz/xyz@syncoid_abc_2022-01-26:23:07:34-GMT-08:00
xyz/xyz@autosnap_2022-01-27_07:30:04_frequently   

- vs. _, timezone indicator vs. not, and why not just use the ISO standard with Z?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants