Create `mach bootstrap` based on Mozilla's mozboot bootstrapper #12916

UK992 · 2016-08-17T23:51:16Z

Fixes #12914
I've made this few weeks ago, its an example how could everything looks like.
It downloads and setup all needed dependencies for MSVC.
It's has version in case if some dependencies need to be updated.
Zip files and folder in zip need to be named <dep>-<version>.
Also if cmake already exist in PATH, it won't download it again.

I want opinion on that, if this is right approaches and how to improve it.

cc @vvuk

This change is

highfive · 2016-08-17T23:51:19Z

Heads up! This PR modifies the following files:

@wafflespeanut: python/servo/command_base.py, python/servo/bootstrap_commands.py

saschanaz · 2016-08-18T00:01:38Z

Can the command window be kept when error occurred? Users won't see the error messages unless they manually run cmd.

UK992 · 2016-08-18T01:13:28Z

@saschanaz I don't understand, can you please show which part of code did you mean?

saschanaz · 2016-08-18T01:22:41Z

@UK992 I was talking about mach.bat ECHO lines but never mind, the error message was thrown from mach rather than mach.bat. Is there a way to keep cmd window when mach throws any error?

(The error message was about the nonexistence of python virtualenv)

bors-servo · 2016-08-22T23:27:15Z

☔ The latest upstream changes (presumably #12956) made this pull request unmergeable. Please resolve the merge conflicts.

wafflespeanut · 2016-08-23T05:07:39Z

python/servo/bootstrap_commands.py

+            # Don't download CMake if already exists in PATH
+            if dep_name == "cmake":
+                if spawn.find_executable(dep_name):
+                    continue


This could be moved out of the loop?

Hardly, considering that it compares with the names in the list.

UK992 · 2016-08-31T19:25:05Z

In last commit i moved to mach bootstrap command based on Mozilla's mozboot, still needs some cleanups, but its for example how could servo bootstrapper looks like. I also added Windows GNU platform.
And can be easily extended to other platforms, since most work it's already done.

wafflespeanut · 2016-09-02T04:43:53Z

python/servo/bootstrapper/base.py

+                                  % __name__)
+
+    def which(self, name):
+        """Python implementation of which.


Can't we just use distutils.spawn.find_executable? (like we do in mach_bootstrap.py)

wafflespeanut · 2016-09-02T04:46:45Z

I don't have much ideas/suggestions regarding MSVC bootstrapping (apart from python'ish style nits, which I'll address in the end). Pinging @larsbergstrom @vvuk and @metajack to look into this sometime...

larsbergstrom · 2016-09-02T13:48:27Z

appveyor.yml

-           $env:FFMPEG_LIBS="avformat:avcodec:avutil" ;
-        }'
-  - if %BUILD_ENV%==msvc call "C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64\vcvars64.bat"
+#  - ps: 'if ($env:BUILD_ENV -eq "msvc") {


Can you change this to instead do ./mach bootstrap-msvc and see if that works on AppVeyor?

bootstrap-msvc is replaced by bootstrap in newer commit, which also support gnu builds, but for msvc build, command is being run in ensure_bootstrapped. But i did that for gnu builds.
EDIT: doesn't work for gnu

larsbergstrom · 2016-09-02T13:49:07Z

python/servo/bootstrap_commands.py

+            return print("Unsupported platform.")
+
+        msvc_deps_dir = path.join(self.context.sharedir, "msvc-dependencies")
+        msvc_deps_url = "https://dl.dropboxusercontent.com/u/25971865/msvc-deps/"


We will definitely want to move these to one of the Servo S3 buckets. I like that you've versioned the files, which is something we've definitely found a need for in the past.

bors-servo · 2016-09-07T16:22:24Z

✌️ @UK992 can now approve this pull request

UK992 · 2016-09-07T18:31:57Z

@larsbergstrom done!
@wafflespeanut Can you please take a look again for any python'ish nits?


        Create `mach bootstrap` based on Mozilla's mozboot bootstrapper


        Update appveyor.yml

UK992 · 2016-09-07T20:46:59Z

I've made some renamings:
check_installed_packages -> ensure_system_packages
install_packages -> install_system_packages

wafflespeanut · 2016-09-08T15:00:04Z

I just had a brief overview on this, and I'm cool with most of the stuff. Let's land this thing happily and if there's something dirty, we'll cleanup later 😄

larsbergstrom · 2016-09-08T15:01:51Z

@bors-servo r=larsbergstrom,wafflespeanut

bors-servo · 2016-09-08T15:01:52Z

📌 Commit eff91be has been approved by larsbergstrom,wafflespeanut

bors-servo · 2016-09-08T17:15:07Z

⌛ Testing commit eff91be with merge 16db0ed...


        Auto merge of #12916 - UK992:msvc-dependencies, r=larsbergstrom,waffl…

@vvuk

…espeanut Create `mach bootstrap` based on Mozilla's mozboot bootstrapper Fixes #12914 I've made this few weeks ago, its an example how could everything looks like. It downloads and setup all needed dependencies for MSVC. It's has version in case if some dependencies need to be updated. Zip files and folder in zip need to be named ``<dep>-<version>``. Also if cmake already exist in PATH, it won't download it again. I want opinion on that, if this is right approaches and how to improve it. cc @vvuk  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/12916)

bors-servo · 2016-09-08T18:06:19Z

💔 Test failed - mac-rel-wpt

highfive · 2016-09-08T18:06:23Z

  ▶ ERROR [expected OK] /webgl/conformance-1.0.3/conformance/ogles/GL/build/build_001_to_008.html
  └   → gl.getProgramInfoLog is not a function

wafflespeanut · 2016-09-08T18:21:58Z

@bors-servo retry #13165

bors-servo · 2016-09-08T18:57:24Z

⌛ Testing commit eff91be with merge daa30e6...


        Auto merge of #12916 - UK992:msvc-dependencies, r=larsbergstrom,waffl…

@vvuk

…espeanut Create `mach bootstrap` based on Mozilla's mozboot bootstrapper Fixes #12914 I've made this few weeks ago, its an example how could everything looks like. It downloads and setup all needed dependencies for MSVC. It's has version in case if some dependencies need to be updated. Zip files and folder in zip need to be named ``<dep>-<version>``. Also if cmake already exist in PATH, it won't download it again. I want opinion on that, if this is right approaches and how to improve it. cc @vvuk  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/12916)

bors-servo · 2016-09-08T19:53:04Z

☀️ Test successful - arm32, arm64, linux-dev, linux-rel, mac-dev-unit, mac-rel-css, mac-rel-wpt, windows-dev

highfive assigned metajack Aug 17, 2016

highfive added the S-awaiting-review label Aug 17, 2016

UK992 force-pushed the UK992:msvc-dependencies branch from 9021bcd to d5cd00d Aug 18, 2016

saschanaz mentioned this pull request Aug 18, 2016

How to build from MSVC branch? #12917

Closed

UK992 force-pushed the UK992:msvc-dependencies branch 2 times, most recently from ba60869 to 0551280 Aug 18, 2016

highfive added the S-needs-rebase label Aug 22, 2016

UK992 force-pushed the UK992:msvc-dependencies branch from 0551280 to 6d36610 Aug 22, 2016

jdm removed the S-needs-rebase label Aug 23, 2016

wafflespeanut reviewed Aug 23, 2016
View changes

UK992 force-pushed the UK992:msvc-dependencies branch from 37dc444 to 2814f61 Aug 31, 2016

UK992 force-pushed the UK992:msvc-dependencies branch 5 times, most recently from bbbbbe9 to 0bcb7ac Sep 1, 2016

wafflespeanut reviewed Sep 2, 2016
View changes

larsbergstrom reviewed Sep 2, 2016
View changes

larsbergstrom self-assigned this Sep 2, 2016

UK992 force-pushed the UK992:msvc-dependencies branch 3 times, most recently from 90dac0b to ecaa97c Sep 2, 2016

UK992 force-pushed the UK992:msvc-dependencies branch from 1bd1fc0 to 922937c Sep 7, 2016

UK992 changed the title ~~WIP: MSVC dependencies auto-download~~ Create `mach bootstrap` based on Mozilla's mozboot bootstrapper Sep 7, 2016

UK992 added 2 commits Aug 1, 2016

Create `mach bootstrap` based on Mozilla's mozboot bootstrapper

fc35555

Update appveyor.yml

Loading status checks…

eff91be

UK992 force-pushed the UK992:msvc-dependencies branch from 922937c to eff91be Sep 7, 2016

highfive unassigned metajack Sep 8, 2016

highfive added S-awaiting-merge and removed S-awaiting-review labels Sep 8, 2016

highfive added S-tests-failed and removed S-awaiting-merge labels Sep 8, 2016

highfive added S-awaiting-merge and removed S-tests-failed labels Sep 8, 2016

Coder206 mentioned this pull request Sep 8, 2016

Adding Documentation for MSVC #13206

Closed

bors-servo merged commit eff91be into servo:master Sep 8, 2016
3 checks passed

3 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

highfive removed the S-awaiting-merge label Sep 8, 2016

UK992 deleted the UK992:msvc-dependencies branch Jan 26, 2017

servo / servo

Create `mach bootstrap` based on Mozilla's mozboot bootstrapper #12916

Create `mach bootstrap` based on Mozilla's mozboot bootstrapper #12916

UK992 commented Aug 17, 2016 •

edited

highfive commented Aug 17, 2016

saschanaz commented Aug 18, 2016

UK992 commented Aug 18, 2016 •

edited

saschanaz commented Aug 18, 2016 •

edited

bors-servo commented Aug 22, 2016

This comment has been minimized.

This comment has been minimized.

UK992 commented Aug 31, 2016 •

edited

This comment has been minimized.

This comment has been minimized.

wafflespeanut commented Sep 2, 2016

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

bors-servo commented Sep 7, 2016

UK992 commented Sep 7, 2016 •

edited

UK992 commented Sep 7, 2016

wafflespeanut commented Sep 8, 2016

larsbergstrom commented Sep 8, 2016

bors-servo commented Sep 8, 2016

bors-servo commented Sep 8, 2016

bors-servo commented Sep 8, 2016

highfive commented Sep 8, 2016

wafflespeanut commented Sep 8, 2016

bors-servo commented Sep 8, 2016

bors-servo commented Sep 8, 2016

servo / servo

Join GitHub today

Create `mach bootstrap` based on Mozilla's mozboot bootstrapper #12916

Create `mach bootstrap` based on Mozilla's mozboot bootstrapper #12916

Conversation

UK992 commented Aug 17, 2016 • edited

highfive commented Aug 17, 2016

saschanaz commented Aug 18, 2016

UK992 commented Aug 18, 2016 • edited

saschanaz commented Aug 18, 2016 • edited

bors-servo commented Aug 22, 2016

This comment has been minimized.

wafflespeanut Aug 23, 2016

This comment has been minimized.

UK992 Sep 1, 2016

UK992 commented Aug 31, 2016 • edited

This comment has been minimized.

wafflespeanut Sep 2, 2016

This comment has been minimized.

UK992 Sep 2, 2016

wafflespeanut commented Sep 2, 2016

This comment has been minimized.

larsbergstrom Sep 2, 2016

This comment has been minimized.

UK992 Sep 2, 2016 • edited

This comment has been minimized.

larsbergstrom Sep 2, 2016

bors-servo commented Sep 7, 2016

UK992 commented Sep 7, 2016 • edited

UK992 commented Sep 7, 2016

wafflespeanut commented Sep 8, 2016

larsbergstrom commented Sep 8, 2016

bors-servo commented Sep 8, 2016

bors-servo commented Sep 8, 2016

bors-servo commented Sep 8, 2016

highfive commented Sep 8, 2016

wafflespeanut commented Sep 8, 2016

bors-servo commented Sep 8, 2016

bors-servo commented Sep 8, 2016

UK992 commented Aug 17, 2016 •

edited

UK992 commented Aug 18, 2016 •

edited

saschanaz commented Aug 18, 2016 •

edited

UK992 commented Aug 31, 2016 •

edited

UK992 Sep 2, 2016 •

edited

UK992 commented Sep 7, 2016 •

edited