Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default date of nix store to increase compatibility? #60446

Closed
tbenst opened this issue Apr 30, 2019 · 5 comments
Closed

Change default date of nix store to increase compatibility? #60446

tbenst opened this issue Apr 30, 2019 · 5 comments

Comments

@tbenst
Copy link
Contributor

@tbenst tbenst commented Apr 30, 2019

Issue description

I often see problems like https://nixos.org/nixpkgs/manual/#python-setup.py-bdist_wheel-cannot-create-.whl due to the default timestamp on NixOS. It seems that dates after 1980 are widely supported across UNIX and DOS, whereas prior to 1980 is often not supported by DOS programs that are later ported to UNIX.

Given that Jan 1, 1970 is completely arbitrary and made up, can we change to the DOS date for nix store to increase compatibility? eg SOURCE_DATE_EPOCH=315532800 instead of 1.

@vcunat

This comment has been minimized.

@volth

This comment has been minimized.

Copy link
Contributor

@volth volth commented Apr 30, 2019

I understood it as the issue with 1970-01-01 00:00:01 timestamp which is fixed by many hacks across nixpkgs:

ensureNewerSourcesHook = { year }: makeSetupHook {}
(writeScript "ensure-newer-sources-hook.sh" ''
postUnpackHooks+=(_ensureNewerSources)
_ensureNewerSources() {
'${findutils}/bin/find' "$sourceRoot" \
'!' -newermt '${year}-01-01' -exec touch -h -d '${year}-01-02' '{}' '+'
}
'');
# Zip file format only allows times after year 1980, which makes e.g. Python wheel building fail with:
# ValueError: ZIP does not support timestamps before 1980
ensureNewerSourcesForZipFilesHook = ensureNewerSourcesHook { year = "1980"; };

# Set the Epoch to 1980; otherwise the Python wheel/zip code
# gets very angry
preConfigure = ''
find . -type f | while read file; do
touch -d @315532800 $file;
done
'';

# Set all timestamps to Jan 1 1980, which is the earliest date the zip format supports...
find $input-tmp -exec touch -t 198001010000.00 {} +

# timestamp need to come after 1980 for zipfiles and nix store is set to epoch
prePatch = ''
substituteInPlace cx_Freeze/freezer.py --replace "os.stat(module.file).st_mtime" "time.time()"
'';

# Cannot build wheel otherwise (zip 1980 issue)
SOURCE_DATE_EPOCH=315532800;

(ensureNewerSourcesHook { year = "1980"; })

# avoid crash with 'ValueError: ZIP does not support timestamps before 1980'
substituteInPlace mx.py --replace \
'zipfile.ZipInfo(arcname, time.localtime(os.path.getmtime(join(root, f)))[:6])' \
'zipfile.ZipInfo(arcname, time.strptime ("1 Jan 1980", "%d %b %Y" )[:6])'

setting $SOURCE_DATE_EPOCH to the latest mtime does not help in many cases (when the timestamped files are inside a zip-archive, or just has been copied from nix store, etc)

But changing 1970-01-01 00:00:01 to, say, 2000-01-01 00:00:01 would fix the 1980-issue as well as other vague issues (overflow/underflow of timestamp arithmetics, ...)

@vcunat

This comment has been minimized.

Copy link
Member

@vcunat vcunat commented Apr 30, 2019

It's just switching from one arbitrary date to another. When doing this, I'd anticipate that some "new modern framework" will come that will assume that anything before 2010 can be ignored. The shift might help in practice, though... but I suspect nixpkgs/stdenv won't be enough and we'd need to shift nix store dates as well, i.e. break its compatibility.

@tbenst

This comment has been minimized.

Copy link
Contributor Author

@tbenst tbenst commented Apr 30, 2019

Yes, thanks @volth. @vcunat you are right that developers are free to hardcode other date assumptions and my lay perception is also that we would have to shift nix store dates to fix

@c0bw3b c0bw3b changed the title Change default date of nix store to increase compatability? Change default date of nix store to increase compatibility? May 4, 2019
@FRidh

This comment has been minimized.

Copy link
Member

@FRidh FRidh commented Jul 17, 2019

I don't see this happening so closing.

@FRidh FRidh closed this Jul 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.