UTC Date not converted to Local if in Folder #422
Comments
|
From what I read from the specifications the DOS date/time of .zip files CANNOT store locale information, so whatever you find there should be assumed to be local time and should NOT be modified when unarchiving. However there are 2 extra fields that can be set, see the go code here So if you have this in your archive then the third extra timestamp UTC time should be used instead of the DOS time, then convert the date depending on the locale of the client. The thing is you already seem do this for files in the root of the archive, but not for the ones inside Folders. |
|
Good catch! Thanks @philip-firstorder. Due to differences in the headers, the "Correct.Date.zip" file is handled with This should be fixed here: https://github.com/MacPaw/XADMaster I'll try to check the code. |
|
I'm seeing that the normal behavior is to set the local time as the DOS date time. In this case the Go library is setting this as UTC, so probably this might also be fixed. |
|
|
|
I managed to set the DOS time in local time like this, and the go library also sets the Expanded flags accordingly. // Modified date should be set in LOCAL format of the hosting server
// This will be converted by the archive/zip package to 3 formats:
// file last modified on (DOS date/time): 2019 Jun 25 18:46:36 (up to 2 seconds loss because of DOS conversion)
// file last modified on (UT extra field modtime): 2019 Jun 25 18:46:37 local (ignored by some unzip software)
// file last modified on (UT extra field modtime): 2019 Jun 25 16:46:37 UTC (ignored by some unzip software)
loc, err := time.LoadLocation("Europe/Berlin")
if err != nil {
// Location data needs to be added to Dockerfile for dev/prod with
// RUN apk add --no-cache tzdata
// see: http://pouzek.si/blog/go-loadlocation-docker/
return 0, fmt.Errorf("Can't load time Location: %v", err)
}
// Set file meta in the zip
fileHeader := &zip.FileHeader{
Method: zip.Store, // no compression, use for fastest speed,
Modified = modified.Time().In(loc),
} |
|
I think for Go this must be fixed in the writer rather than the user script. In the APPNOTE it only says MSDOS time, so technically using the UTC is ok, but since all other compressors seem to use local most extractors will expect it to be the local.
|
|
The reasons why Go wants the Modified field in local are:
So I think Go is right here. What is not OK is the specification of zip itself, because they got too lazy to solve this problem when they first implemented MS-DOS times, and because they didn't deprecate it in 30 years. |
|
I'm not sure I understand your point. I meant here: The first date, the DOS one, should be local instead of UTC, since most extractors expect that to be local. Looking at the writer code I see this: // Contrary to the FileHeader.SetModTime method, we intentionally
// do not convert to UTC, because we assume the user intends to encode
// the date using the specified timezone. A user may want this control
// because many legacy ZIP readers interpret the timestamp according
// to the local timezone.
//
// The timezone is only non-UTC if a user directly sets the Modified
// field directly themselves. All other approaches sets UTC.
fh.ModifiedDate, fh.ModifiedTime = timeToMsDosTime(fh.Modified)Even if they say "legacy ZIP readers", I think most readers (even Go I assume) will use the extended values instead of the DOS one, so keep that time to local will be better, in my opinion. |
|
Ah, sorry, my example above was BEFORE I implemented in go the DOS date/time in local format, if for example you look on the other 5GB archive zipinfos you'll see it's all in local time. Otherwise I wouldn't have been able to detect this bug, as both the DOS and UT extra fields would have had the same local date, giving me the impression that all is well. So the problem is actually that Keka doesn't read from the UT extended time for the files in Folders. |
#422 (comment) So |
|
Yes, then you can set this as solved I guess, that was the only problem. |
|
I did set it as fixed (the label) but I prefer to close it when 1.1.18 is released. |
|
Also would be awesome to change the writer.go behavior to set the DOS date in local instead of UTC. |
They won't change it, because it would overwrite whatever custom timezone the user specifies. I fixed this on my side already. |
|
What they say there is that if the user informed // If Modified is set, this takes precedence over MS-DOS timestamp fields.
if !fh.Modified.IsZero() {
// Contrary to the FileHeader.SetModTime method, we intentionally
// do not convert to UTC, because we assume the user intends to encode
// the date using the specified timezone. A user may want this control
// because many legacy ZIP readers interpret the timestamp according
// to the local timezone.
//
// The timezone is only non-UTC if a user directly sets the Modified
// field directly themselves. All other approaches sets UTC.
fh.ModifiedDate, fh.ModifiedTime = timeToMsDosTime(fh.Modified)But the problem when // Modified is the modified time of the file.
//
// When reading, an extended timestamp is preferred over the legacy MS-DOS
// date field, and the offset between the times is used as the timezone.
// If only the MS-DOS date is present, the timezone is assumed to be UTC.
//
// When writing, an extended timestamp (which is timezone-agnostic) is
// always emitted. The legacy MS-DOS date field is encoded according to the
// location of the Modified time.
Modified time.Time
ModifiedTime uint16 // Deprecated: Legacy MS-DOS date; use Modified instead.
ModifiedDate uint16 // Deprecated: Legacy MS-DOS time; use Modified instead.But they are also converting the DOS date to UTC in the reader, so I'm not sure they would like to change this. msdosModified := msDosTimeToTime(f.ModifiedDate, f.ModifiedTime)
f.Modified = msdosModified
if !modified.IsZero() {
f.Modified = modified.UTC()
// If legacy MS-DOS timestamps are set, we can use the delta between
// the legacy and extended versions to estimate timezone offset.
//
// A non-UTC timezone is always used (even if offset is zero).
// Thus, FileHeader.Modified.Location() == time.UTC is useful for
// determining whether extended timestamps are present.
// This is necessary for users that need to do additional time
// calculations when dealing with legacy ZIP formats.
if f.ModifiedTime != 0 || f.ModifiedDate != 0 {
f.Modified = modified.In(timeZone(msdosModified.Sub(modified)))
}
}Since they also set the extended dates, I will keep that as fixed with the |
|
I got go‘s point: set only Modified field to the local timezone and they take care of everything else. That‘s why they deprecated the other time fields. unar fix is enough. Thanks! |
|
@philip-firstorder version 1.1.19 has all fixes, released now in the WEB, in a few hours in the App Store :) |
|
Very nice, I just tested it and confirm it fixes this bug and can also open the 5GB archive without any warning. I will send my users to use Keka :) |
I archived a a file with modified date
2019-06-25 11:58:57.201 +0000 UTCusing https://golang.org/pkg/archive/zip/If I put the file inside a folder and the folder is archived, then the UTC date is not converted to Local Date when unarchiving, to reproduce you can unarchive this png.
Bad Date.zip
But if the file is directly archived in the root, then the date is correctly converted to Local Date.
Correct Date.zip
The correct time using my Germany local +2 Timezone should be 13:58, but if the file is inside a folder it remains 11:58 when unarchived, seems a bug to me.
The text was updated successfully, but these errors were encountered: