Initial Windows support #320

larsbergstrom · 2016-04-16T01:05:01Z

I don't expect it to succeed yet (windows was broken by a recent servo commit), so homu is not gated on it. But, like arm64, I'd like to get it building regularly so that I can see the output.

This change is

aneeshusa · 2016-04-16T01:10:01Z

buildbot/master/files/config/master.cfg

@@ -231,6 +232,16 @@ arm64_factory = create_servo_factory([
    steps.Compile(command=["./mach", "build", "--rel", "--target=aarch64-unknown-linux-gnu"], env=arm64_compile_env),
 ])

+windows_compile_env = dict({'PATH': 'C:\\msys64\\mingw64\\bin;C:\\msys64\\usr\\bin\\C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Amazon\\cfn-bootstrap\\',


Let's use a raw string to make this nicer.

aneeshusa · 2016-04-16T01:12:09Z

Just curious: did you make any progress with #302?

larsbergstrom · 2016-04-16T01:13:23Z

buildbot/master/files/config/master.cfg

@@ -232,7 +232,7 @@ arm64_factory = create_servo_factory([
    steps.Compile(command=["./mach", "build", "--rel", "--target=aarch64-unknown-linux-gnu"], env=arm64_compile_env),
 ])

-windows_compile_env = dict({'PATH': 'C:\\msys64\\mingw64\\bin;C:\\msys64\\usr\\bin\\C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Program Files\\Amazon\\cfn-bootstrap\\',
+windows_compile_env = dict({'PATH': r'C:\msys64\mingw64\bin;C:\msys64\usr\bin\C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Amazon\cfn-bootstrap',


@aneeshusa like this?

Yes, much easier to work with. (For some reason GitHub highlights these weird, though.)

Did you remove the slash at the end on purpose?

larsbergstrom · 2016-04-16T01:15:14Z

And re: #302, it's going to be WAY harder than expected. buildbot requires an incredibly manual setup process (involving a bunch of manual registry key edits, a few aborted service start/restarts to create initial keys with proper permissions, etc.), so I just muscled my way through the setup manually.

When we go to stand up a second machine and we're not in urgent-Windows-mode, I'd like to look at it again, because nobody should have to go through that manually.

aneeshusa · 2016-04-16T01:17:34Z

I figured as much, that sounds painful. Which version of Windows is this machine running? I think that would be valuable to document (on the wiki).

larsbergstrom · 2016-04-16T01:18:41Z

Will do! I went with Windows Server 2012. I like that it's got a longer support cycle, but the lack of a start button is incredibly disturbing :-)

aneeshusa · 2016-04-16T01:24:59Z

buildbot/master/files/config/master.cfg

+windows_compile_env = dict({'PATH': r'C:\msys64\mingw64\bin;C:\msys64\usr\bin\C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Program Files\Amazon\cfn-bootstrap',
+                            'MSYSTEM': 'MINGW64',
+                            'MSYS': 'winsymlinks=lnk'},
+                           **common_test_env)


What about the CARGO_HOME and SERVO_CACHE_DIR environment variables we set for Linux and OS X?

(I think there's a port of ccache to windows, but we can worry about that later.)

I'm skipping them because we only have one build type, at least for now, and I'm trying to get something that works similarly to the current AppVeyor config (https://github.com/servo/servo/blob/master/appveyor.yml).

I really wish that we could use ccache, but it doesn't work on Windows due to known build problems with the components we pick up from Firefox. I already spent a bunch of time trying to get it working (and failing) on AppVeyor and locally.

Do we need something similar to servo/servo#10528?

Luckily, I have 100GB of disk on this machine :-) AppVeyor's cache is only 500MB.

That said, we have no cache reduction on any of the machines, so they will all slowly accrete more cargo packages & rust/cargo binaries over time until they fill all available disk. Opening #321

aneeshusa · 2016-04-16T01:41:45Z

This LGTM after a squash.

Side note: As far as I can tell we haven't actually deployed any of the last 3-4 PRs to our buildbot machines. I'm sure you and @edunham both have enough on your plates already; the Homu queue only has one build right now, so can you show me exactly how to restart Buildbot cleanly to deploy this when it finishes?


        Initial Windows support

larsbergstrom · 2016-04-16T01:44:17Z

@bors-servo r=aneeshusa

bors-servo · 2016-04-16T01:44:19Z

📌 Commit 9916006 has been approved by aneeshusa

bors-servo · 2016-04-16T01:44:20Z

⌛ Testing commit 9916006 with merge d268ef1...


        Auto merge of #320 - larsbergstrom:windows, r=aneeshusa

@edunham

Initial Windows support r? @edunham @aneeshusa I don't expect it to succeed yet (windows was broken by a recent servo commit), so homu is not gated on it. But, like arm64, I'd like to get it building regularly so that I can see the output.  --- This change is [<img src="https://reviewable.io/review_button.svg" height="35" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/320)

larsbergstrom · 2016-04-16T01:46:22Z

@aneeshusa I typically just wait for the queue to empty, log in, and then highstate. I've found that the "careful" buildbot restarts that we described have a downside that we end up with a buildbot master process that is running as an orphaned child process instead of as a child of the service framework, so I don't use it very often.

I think that if we change the salt rules to push out the files but only send a SIGHUP to a running buildbot process, that should have the best effect, and could be integrated with normal salt rules.

aneeshusa · 2016-04-16T01:47:45Z

OK, can I try deploying these changes? I'm logged into a tmux instance on servo-master1 if you want to watch.

larsbergstrom · 2016-04-16T01:50:26Z

@aneeshusa Sure! Please go right ahead, and thanks for all of the late-Friday-night reviews & chatter :-)

bors-servo · 2016-04-16T01:57:42Z

☀️ Test successful - travis

aneeshusa · 2016-04-16T02:38:14Z

Just deployed. BTW, I turned off the automatic buildbot-master restart in #307, so I also restarted it manually with service buildbot-master restart.

I'm running tail +F twistd.log in tmux and I'm seeing a lot of this:

invalid login from unknown user 'servo-windows1'

As well as the same message for servo-macpro1. I think we're missing some authentication?

larsbergstrom · 2016-04-16T02:57:00Z

I know that macpro1 should not be working, but servo-windows1 should be working because of this:
https://github.com/servo/saltfs/blob/master/buildbot/master/files/config/master.cfg#L27-L28

I'm not sure what's going wrong here...

larsbergstrom · 2016-04-16T03:07:37Z

Hrm, I re-ran the steps here:
https://github.com/servo/servo/wiki/Buildbot-administration#dealing-with-troubles

And now it's seeing servo-windows1. However, the buildbot process seems to be running as root instead of as the servo user.

aneeshusa · 2016-04-16T03:14:48Z

Which steps in particular did you run? It's weird that a single restart didn't work properly.

larsbergstrom · 2016-04-16T03:17:24Z

I attempted to run:

# su - servo
# buildbot restart --clean --nodaemon /home/servo/buildbot/master &

But it told me that the .pid file was not owned by the servo user. So I just ran it as root and the service turned over nicely :-)

aneeshusa · 2016-04-16T03:18:00Z

Also, I think something may be wrong with our buildbot-master.conf file. service buildbot-master status says that it's not running, and checking in syslog says it was terminated with status 1, so maybe it's being invoked incorrectly?

aneeshusa reviewed Apr 16, 2016
View changes

larsbergstrom reviewed Apr 16, 2016
View changes

aneeshusa reviewed Apr 16, 2016
View changes

Initial Windows support

Loading status checks…

9916006

larsbergstrom force-pushed the larsbergstrom:windows branch from 358de78 to 9916006 Apr 16, 2016

bors-servo merged commit 9916006 into servo:master Apr 16, 2016
2 checks passed

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

aneeshusa mentioned this pull request Apr 16, 2016

Keep homebrew updated on OS X #323

Closed

servo / saltfs

Initial Windows support #320

Initial Windows support #320

larsbergstrom commented Apr 16, 2016

This comment has been minimized.

aneeshusa commented Apr 16, 2016

This comment has been minimized.

This comment has been minimized.

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

bors-servo commented Apr 16, 2016

bors-servo commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

bors-servo commented Apr 16, 2016

aneeshusa commented Apr 16, 2016 •

edited

larsbergstrom commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

servo / saltfs

Join GitHub today

Initial Windows support #320

Initial Windows support #320

Conversation

larsbergstrom commented Apr 16, 2016

This comment has been minimized.

aneeshusa Apr 16, 2016

aneeshusa commented Apr 16, 2016

This comment has been minimized.

larsbergstrom Apr 16, 2016

This comment has been minimized.

aneeshusa Apr 16, 2016

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

This comment has been minimized.

aneeshusa Apr 16, 2016

This comment has been minimized.

larsbergstrom Apr 16, 2016

This comment has been minimized.

aneeshusa Apr 16, 2016

This comment has been minimized.

larsbergstrom Apr 16, 2016

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

bors-servo commented Apr 16, 2016

bors-servo commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

bors-servo commented Apr 16, 2016

aneeshusa commented Apr 16, 2016 • edited

larsbergstrom commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

larsbergstrom commented Apr 16, 2016

aneeshusa commented Apr 16, 2016

aneeshusa commented Apr 16, 2016 •

edited