Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mac builder updates #674

Closed
larsbergstrom opened this issue May 22, 2017 · 10 comments
Closed

Mac builder updates #674

larsbergstrom opened this issue May 22, 2017 · 10 comments

Comments

@larsbergstrom
Copy link
Contributor

@larsbergstrom larsbergstrom commented May 22, 2017

Few items here:

@aneeshusa Are you done with your testing on servo-mac1?

cc @edunham

@aneeshusa
Copy link
Member

@aneeshusa aneeshusa commented May 22, 2017

It's currently finals week, I don't see myself using servo-mac1 anymore, feel free to pave :)
Any reason we'll no longer be using servo-macpro1?

We may need to update the Buildbot config so Buildbot knows about the new builders.
Also, @metajack filed #650 recently, so there may be some bumps while highstating.

@larsbergstrom
Copy link
Contributor Author

@larsbergstrom larsbergstrom commented May 23, 2017

@edunham Added the rest of the steps to push this through

@edunham
Copy link
Contributor

@edunham edunham commented May 24, 2017

None of the versions of xcode and command line tools in the app store were compatible with 10.10.5, and macstadium started all the new ones out at 10.10.0, so I got to spend a lot of today poking at GUI installers over a slow VNC connection.

Mac8 has a nasty habit of setting its locale to Croatian in new accounts when they're created. I manually set it back to en-us for administrator and Servo, but just be aware it may do this if we recreate the Servo account.

saltstack/salt#34658 was keeping them from Salting, as noted in Jack's ticket, because they couldn't retrieve the FailedLoginAttempts. We don't care about thoes in the slightest, so I have done a thing I know is bad practices as a stopgap and patched the Salt on each mac to shut up about it:

def _get_account_policy_data_value(name, key):
    '''
    Return the value for a key in the accountPolicy section of the user's plist
    file. For use by this module only

    :param str name: The username
    :param str key: The accountPolicy key

    :return: the value contained within the key
    :rtype: str

    :raises: CommandExecutionError on user not found or any other unknown error
    '''
    cmd = 'dscl . -readpl /Users/{0} accountPolicyData {1}'.format(name, key)
    try:
        ret = salt.utils.mac_utils.execute_return_result(cmd)
    except CommandExecutionError as exc:
        if 'eDSUnknownNodeName' in exc.strerror:
            raise CommandExecutionError('User not found: {0}'.format(name))
        ret = "" #TODO FIXME THIS IS AWFUL
        #raise CommandExecutionError('Unknown error: {0}'.format(exc.strerror))

    return ret

Current state is that 2, 6 and 7 have the patch and are highstating happily. 5 and 8 are supposed to have everything installed correctly but they aren't pinging from Salt so I need to VNC back in and troubleshoot. 9 is 2/3 of the way through initial setup (I started on it later than the others because Macstadium was delayed paving it, as I'd initially typo'd the IP) and needs:

  • install brew and git and pip
  • download salt files
  • Add the following to the com.saltstack.salt.minion.plist file just after the Label section
  • coreaudiod / mdworker commands

from https://github.com/servo/servo/wiki/SaltStack-Administration and then it should be good to go (I skipped ahead on user account creation).

The only hitch at the moment is that none of macs 5-9 have 1280x960 available as a resolution. Allegedly there's a key you can press to pick an arbitrary resolution but I couldn't get it to do that through VNC. I've set them all to 1600x900, which was the closest available resolution, for the time being.

@edunham
Copy link
Contributor

@edunham edunham commented May 24, 2017

5 is patched and back up. Turns out it had lost the salt master's IP from its hosts file.

@edunham
Copy link
Contributor

@edunham edunham commented May 24, 2017

8 had the same issue, and is now also patched and highstated. I tentatively blame the fact that I ran part of the system updates after manually adding Salt to the hosts file for its disappearance, whereas on 6 and 7 the updates showed up earlier so I happened to run them sooner in the process.

bors-servo added a commit that referenced this issue May 25, 2017
Add 4/5 of the new macs

See #674 (comment).

Test that the 4 completed new builders are actually working, and free up a bit of capacity, while I finish up with mac9.

Also format these lines to have less icky diffs as we add and remove others.

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/677)
<!-- Reviewable:end -->
@edunham
Copy link
Contributor

@edunham edunham commented May 25, 2017

"deploy" turns out to mean "buildbot reconfig" rather than just highstate at the moment.

macs 5-8 are now in Buildbot's rotation.

I have verified that 5-8 are logged in as Tom Servo and have no outstanding updates.

@edunham edunham mentioned this issue May 25, 2017
bors-servo added a commit that referenced this issue May 25, 2017
Mac9 is ready

* Return mac1 to rotation
* Remove macpro1
* Add mac9
* Switch the whole thing to a nice list comprehension

This finishes up the work from #674

<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/679)
<!-- Reviewable:end -->
@edunham
Copy link
Contributor

@edunham edunham commented May 26, 2017

Ok we're all like
image now. Macs 9 and 1 are back in; macpro1 is out.

http://build.servo.org/buildslaves

I think we should be good to spin down the pro, but let's leave it for a couple days so we can swap it back in if something goes horribly and unexpectedly wrong with the new builders as they do their first few builds.

@larsbergstrom
Copy link
Contributor Author

@larsbergstrom larsbergstrom commented May 27, 2017

Awesome - this is all super! BTW, IIRC macstadium asked that we do all shutdown requests via explicit support tickets rather than the UI so that the billing, etc. don't get messed up.

@edunham
Copy link
Contributor

@edunham edunham commented May 31, 2017

It's shut down, and I emailed the list with the update.

@edunham edunham closed this May 31, 2017
@aneeshusa
Copy link
Member

@aneeshusa aneeshusa commented Jun 1, 2017

I've deleted the servo-macpro1 minion key from the Salt master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.