Make etc/ci/upload_nightly.sh responsible for putting datetimes into filenames #12163

imjacobclark · 2016-07-02T09:36:30Z

This is part of #11980, allowing for more reproducible builds, increasing the security and integrity of the release process.

This particular change ensures the date and time is on the files that are available for download from the nightly-release s3 bucket and removes that responsibility from the packager.

See #12128 for more details.

@aneeshusa

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #12128 (github issue number if applicable).

There are tests for these changes OR
These changes do not require tests because they are part of the CI infrastructure

This change is

highfive · 2016-07-02T09:36:36Z

Heads up! This PR modifies the following files:

@wafflespeanut: python/servo/package_commands.py
@aneeshusa: etc/ci/upload_nightly.sh


        upload_nightly.sh responsible for date on nightlies


        syntax fix


        resolve build failure


        resolve build failure

bors-servo · 2016-07-02T17:50:51Z

☔ The latest upstream changes (presumably #12156) made this pull request unmergeable. Please resolve the merge conflicts.

aneeshusa · 2016-07-03T00:20:57Z

This should also incorporate the change in #12088/wait for that to merge.

aneeshusa · 2016-07-03T00:21:44Z

etc/ci/upload_nightly.sh

 upload() {
-    local package_filename 
+    local package_filename current_date_time


Let's rename current_date_time to nightly_timestamp.

Also rename package_filename to nightly_filename since it'll include the timestamp per my other comment.

aneeshusa · 2016-07-03T00:22:55Z

etc/ci/upload_nightly.sh

    package_filename="$(basename "${2}")"
+    current_date_time="$(date -u +"%Y-%m-%dT%H:%M:%SZ")"


Put this line before the one that sets package_filename, and then include nightly_timestamp when you set package_filename.


        Merge branch 'master' of https://github.com/servo/servo into 12128-up…

…loadnightlysh-responsible-date-times Conflicts: etc/ci/upload_nightly.sh

imjacobclark · 2016-07-03T13:04:02Z

@aneeshusa Resolve the merge conflict and review comments, just need to wait on #12088 landing.

Is #12088 still relevant in that file if this PR moves the time/date logic into upload_nightly.sh?

bors-servo · 2016-07-03T13:36:42Z

☔ The latest upstream changes (presumably #12088) made this pull request unmergeable. Please resolve the merge conflicts.

aneeshusa · 2016-07-03T13:37:33Z

Well, it just merged, so that's a moot point now :)

That PR changes the colons in the timestamp to dashes because the tar command will interpret the filename with colons as containing a remote server address with the colons and try to connect out, instead of knowing that it is a local file. I wish tar would just get smarter, but in the meantime we're switching to dashes so tar doesn't get confused. Any replacement for that functionality also needs to incorporate that change.

imjacobclark · 2016-07-06T08:12:53Z

@aneeshusa I will port the functionality from the Python into upload_nightly.sh, there seems to be a few conflicts on this PR too which I'll resolve shortly.

aneeshusa · 2016-08-10T01:55:20Z

#12088 merged so this is no longer blocked.

@imjacobclark are you still working on this? Do you have any questions?

imjacobclark · 2016-08-10T08:02:07Z

@aneeshusa still working on this, had a bit of a hectic few weeks, however will pick this up again now. Pretty straightforward, I'll catch up on IRC if there are any issues.

KiChjang · 2016-10-13T22:42:44Z

@imjacobclark Is this still being worked on?

ducks · 2016-10-18T02:19:14Z

I'd like to help finish this one if it's up for grabs.

It looks like the datetime got moved to the upload_nightly.sh but still uses colons.
Was that the only outstanding change needed besides a rebase?

aneeshusa · 2016-10-27T17:39:01Z

@rjgoldsborough Yes, please feel free to submit a PR for this!

This needs a rebase, but if you look at python/servo/package_commands.py you'll see we replace colons by dashes, which is also behavior we need to keep.

wafflespeanut · 2016-11-04T08:45:35Z

Closing based on inactivity (@rjgoldsborough has started working on it in #14058)

highfive assigned SimonSapin Jul 2, 2016

highfive added the S-awaiting-review label Jul 2, 2016

imjacobclark mentioned this pull request Jul 2, 2016

Make etc/ci/upload_nightly.sh responsible for putting datetimes into filenames #12128

Closed

upload_nightly.sh responsible for date on nightlies

Unverified

The key that signed this is expired.

GPG key ID: D77F08FC8050BD8B Learn about signing commits

Loading status checks…

4d966ee

imjacobclark force-pushed the imjacobclark:12128-uploadnightlysh-responsible-date-times branch from 534fea9 to 4d966ee Jul 2, 2016

imjacobclark added 3 commits Jul 2, 2016

syntax fix

Unverified

The key that signed this is expired.

GPG key ID: D77F08FC8050BD8B Learn about signing commits

Loading status checks…

1a0abd2

resolve build failure

Unverified

The key that signed this is expired.

GPG key ID: D77F08FC8050BD8B Learn about signing commits

Loading status checks…

2cda535

resolve build failure

Unverified

The key that signed this is expired.

GPG key ID: D77F08FC8050BD8B Learn about signing commits

Loading status checks…

c7676d5

highfive added the S-needs-rebase label Jul 2, 2016

aneeshusa reviewed Jul 3, 2016
View changes

aneeshusa added S-needs-code-changes S-blocked-on-external and removed S-awaiting-review labels Jul 3, 2016

aneeshusa assigned aneeshusa and unassigned SimonSapin Jul 3, 2016

Merge branch 'master' of https://github.com/servo/servo into 12128-up…

Loading status checks…

834a6a5

…loadnightlysh-responsible-date-times Conflicts: etc/ci/upload_nightly.sh

highfive added S-awaiting-review and removed S-needs-code-changes labels Jul 3, 2016

highfive added the S-needs-rebase label Jul 3, 2016

aneeshusa added S-needs-code-changes and removed S-awaiting-review labels Jul 18, 2016

aneeshusa removed the S-blocked-on-external label Aug 10, 2016

wafflespeanut closed this Nov 4, 2016

servo / servo

Make etc/ci/upload_nightly.sh responsible for putting datetimes into filenames #12163

Make etc/ci/upload_nightly.sh responsible for putting datetimes into filenames #12163

imjacobclark commented Jul 2, 2016 •

edited

highfive commented Jul 2, 2016

bors-servo commented Jul 2, 2016

aneeshusa commented Jul 3, 2016 •

edited

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

imjacobclark commented Jul 3, 2016 •

edited

bors-servo commented Jul 3, 2016

aneeshusa commented Jul 3, 2016

imjacobclark commented Jul 6, 2016

aneeshusa commented Aug 10, 2016

imjacobclark commented Aug 10, 2016

KiChjang commented Oct 13, 2016

ducks commented Oct 18, 2016

aneeshusa commented Oct 27, 2016

wafflespeanut commented Nov 4, 2016 •

edited

		package_filename="$(basename "${2}")"
		current_date_time="$(date -u +"%Y-%m-%dT%H:%M:%SZ")"

servo / servo

Join GitHub today

Make etc/ci/upload_nightly.sh responsible for putting datetimes into filenames #12163

Make etc/ci/upload_nightly.sh responsible for putting datetimes into filenames #12163

Conversation

imjacobclark commented Jul 2, 2016 • edited

highfive commented Jul 2, 2016

bors-servo commented Jul 2, 2016

aneeshusa commented Jul 3, 2016 • edited

This comment has been minimized.

aneeshusa Jul 3, 2016

This comment has been minimized.

aneeshusa Jul 3, 2016

This comment has been minimized.

aneeshusa Jul 3, 2016

imjacobclark commented Jul 3, 2016 • edited

bors-servo commented Jul 3, 2016

aneeshusa commented Jul 3, 2016

imjacobclark commented Jul 6, 2016

aneeshusa commented Aug 10, 2016

imjacobclark commented Aug 10, 2016

KiChjang commented Oct 13, 2016

ducks commented Oct 18, 2016

aneeshusa commented Oct 27, 2016

wafflespeanut commented Nov 4, 2016 • edited

imjacobclark commented Jul 2, 2016 •

edited

aneeshusa commented Jul 3, 2016 •

edited

imjacobclark commented Jul 3, 2016 •

edited

wafflespeanut commented Nov 4, 2016 •

edited