Mac builders fails to compile harfbuzz #9504

KiChjang · 2016-02-02T08:43:41Z

It's the autoconf failure, but now it happens on our mac build machines as well.

--- stderr
configure: WARNING:
  You will not be able to create source packages with 'make dist'
  because gtk-doc >= 1.15 is not found.
make[1]: warning: -jN forced in submake: disabling jobserver mode.
/Users/servo/.cargo/registry/src/github.com-0a35038f75765ae4/harfbuzz-sys-0.1.2/harfbuzz/missing: line 81: autoheader: command not found
WARNING: 'autoheader' is missing on your system.
         You should only need it if you modified 'acconfig.h' or
         'configure.ac' or m4 files included by 'configure.ac'.
         The 'autoheader' program is part of the GNU Autoconf package:
         <http://www.gnu.org/software/autoconf/>
         It also requires GNU m4 and Perl in order to run:
         <http://www.gnu.org/software/m4/>
         <http://www.perl.org/>
make[1]: *** [/Users/servo/.cargo/registry/src/github.com-0a35038f75765ae4/harfbuzz-sys-0.1.2/harfbuzz/config.h.in] Error 127
make: *** [all] Error 2
thread '<main>' panicked at 'assertion failed: Command::new("make").args(&["-R", "-f", "makefile.cargo",
                            &format!("-j{}" , env:: var ( "NUM_JOBS" ) .
                                     unwrap (
                                     ))]).status().unwrap().success()', /Users/servo/.cargo/registry/src/github.com-0a35038f75765ae4/harfbuzz-sys-0.1.2/build.rs:14
stack backtrace:
   1:        0x10735a038 - sys::backtrace::tracing::imp::write::h268482af15af0a80Y8t
   2:        0x10735dbb5 - panicking::default_handler::_$u7b$$u7b$closure$u7d$$u7d$::closure.42271
   3:        0x10735d7b6 - panicking::default_handler::h97bc664f7e2bfeb5Ocy
   4:        0x10734b956 - sys_common::unwind::begin_unwind_inner::h98c0975c9b6b2fa0Zbt
   5:        0x1073461c7 - sys_common::unwind::begin_unwind::h6740538633853985845
   6:        0x10734129a - main::h4996c082ce17a6c8iaa
   7:        0x10735d3e2 - sys_common::unwind::try::try_fn::h9072539541001871658
   8:        0x1073595db - __rust_try
   9:        0x10735d291 - rt::lang_start::hacfbaccc148e05fdU4x
  10:        0x107346b29 - main

cc @larsbergstrom

paulrouget · 2016-02-02T09:01:53Z

Isn't it just a matter of linking autoconf?

nox · 2016-02-02T09:02:40Z

@paulrouget What do you mean here by linking?

paulrouget · 2016-02-02T09:04:06Z

Install and brew link autoconf (brew link autoconf)

larsbergstrom · 2016-02-02T13:07:38Z

Yes, it looks like our saltstack rules are not properly installing autoconf. I just brought a builder back online from scratch, and the auto-provisioning did not work correctly.

cc @edunham @aneeshusa

larsbergstrom · 2016-02-02T13:37:00Z

OK, it looks like autoconf213 got linked first (instead?) and this caused autoconf to not get linked and thus autoheader was not available. Maybe an ordering issue in our salt config files? I manually fixed servo-mac1 so we should be able to proceed.

administrators-Mac-mini:~ root# su - administrator
administrators-Mac-mini:~ administrator$ brew link autoconf
Linking /usr/local/Cellar/autoconf/2.69... 
Error: Could not symlink share/info/autoconf.info
Target /usr/local/share/info/autoconf.info
is a symlink belonging to autoconf213. You can unlink it:
  brew unlink autoconf213

To force the link and overwrite all conflicting files:
  brew link --overwrite autoconf

To list all files that would be deleted:
  brew link --overwrite --dry-run autoconf
administrators-Mac-mini:~ administrator$ brew link --overwrite autoconf
Linking /usr/local/Cellar/autoconf/2.69... 19 symlinks created
administrators-Mac-mini:~ administrator$ which autoheader
/usr/local/bin/autoheader

aneeshusa · 2016-02-02T13:45:12Z

I don't think the config is running the 'brew link' on autoconf, so I'll investigate adding that. Does automake need to be installed before autoconf, or can they be installed together (i.e. in the same command?)

larsbergstrom · 2016-02-02T13:48:29Z

@aneeshusa I think they can be installed together - the conflict seemed to be with autoconf vs. autoconf213 (in particular, autoconf213 being installed first and blocking the link of autoconf).

Thanks a ton for all your help with this!

aneeshusa · 2016-02-02T13:51:10Z

Do we need to install both autoconf and autoconf213? Based on the current master, it looks like we're only installing autoconf213.

larsbergstrom · 2016-02-02T13:56:47Z

I thought that we needed both. autoconf213 is required for the SpiderMonkey JS engine; everything else (like harfbuzz) uses normal autoconf.

They are definitely both installed on all our machines.

Maybe autoconf is a dependency of automake?

aneeshusa · 2016-02-02T13:58:25Z

Yes, it seems you're right: automake depends on autoconf

aneeshusa · 2016-02-02T14:09:37Z

Do you know if there is a way to query if a homebrew package is linked? I couldn't find anything like that in the docs.

larsbergstrom · 2016-02-02T14:16:15Z

Hrm, I don't see anything, either. brew info and the others don't seem to show it.

aneeshusa · 2016-02-02T14:18:48Z

Ok, I should have a PR shortly then, but it'll have to be a bit hacky.

aneeshusa · 2016-02-02T14:18:57Z

Actually, can we just check the symlink manually?

larsbergstrom · 2016-02-02T14:20:52Z

Aha, that sounds like a great plan :-)

aneeshusa · 2016-02-02T14:21:25Z

I have to go to my first class of the semester, but I should have a quick fix for this today, and a better fix in the next few days.

larsbergstrom · 2016-02-02T14:22:19Z

That's fantastic! Thanks again for all your help, and good luck with classes :-)

aneeshusa · 2016-02-02T16:23:08Z

I'm trying to understand brew link's behavior better in the case of updates. Is there a spare Mac builder that I can ssh into and mess around on (i.e. one that's not being used for builds)?

larsbergstrom · 2016-02-02T16:25:12Z

Unfortunately, they're all pretty slammed right now. I requested a new one from macstadium this morning, though, and as soon as I have it provisioned enough for you to ssh in, I'll let you know! I don't know how long it will take for them to bring it online.

aneeshusa · 2016-02-02T16:30:53Z

Ok, thank you! We should also use it for some manual testing when reviewing the PR.


        Link autoconf after installing autoconf213 on OS X

Use --overwrite because autoconf and autoconf213 are installed, and either one may have been linked beforehand. They overlap in the files they provide, and Homebrew will refuse to destructively overwrite any symlinks without an explicit flag. There's no built-in Salt module for brew link, so for now this is a quick fix. This provides some idempotence for repeated Salt highstate runs by checking the directory where Homebrew keeps track of linked packages (and linking autoconf if it is not linked). It's unknown whether this approach properly handles updates to one or both packages. A better approach would be to write a custom Salt state (perhaps homebrew.linked); even if this state works properly, a custom state would be more declarative and could be upstreamed to Salt proper. Closes servo#213. Closes servo/servo#9504.


        Link autoconf after installing autoconf213 on OS X

Use --overwrite because autoconf and autoconf213 are installed, and either one may have been linked beforehand. They overlap in the files they provide, and Homebrew will refuse to destructively overwrite any symlinks without an explicit flag. There's no built-in Salt module for brew link, so for now this is a quick fix. This provides some idempotence for repeated Salt highstate runs by checking the directory where Homebrew keeps track of linked packages (and linking autoconf if it is not linked). It's unknown whether this approach properly handles updates to one or both packages. A better approach would be to write a custom Salt state (perhaps homebrew.linked); even if this state works properly, a custom state would be more declarative and could be upstreamed to Salt proper. Closes servo#210. Closes servo/servo#9504.

larsbergstrom · 2016-02-03T01:32:56Z

@aneeshusa You should be able to ssh as root into servo-macpro1.servo.org. I'm doing some basic timing runs, but feel free to do whatever you need to the box! It's not hooked up to our CI yet.

larsbergstrom · 2016-02-04T22:10:54Z

@aneeshusa Did you have a chance to do some manual testing of these rules? Or is there something that you'd like me to manually test?

Thanks!

aneeshusa · 2016-02-05T05:17:06Z

I got caught up in school the last two days - I should be able to poke around today.

How can I run servo builds manually on the Mac (to ensure that the builds using autoconf and autoconf213 work properly)?

larsbergstrom · 2016-02-05T15:59:15Z

No worries at all! You can test the build by doing:

su - servo
source ~/exports
cd ~/servo
rm -rf target
./mach build -d

Also, we may need to coordinate a bit with @mbrubeck, as he's also doing some testing of intermittent test failures. Let me know how you were going to do the tests if your classes get a bit overwhelming - I'm happy to help out! I just don't really know how to run salt "standalone" w/o the master to test pending changes.

aneeshusa · 2016-02-05T18:05:18Z

@larsbergstrom I updated the wiki page with instructions on how to run Salt remotely and locally - let me know if that helps.

@mbrubeck, let me know if/when you hop on the machine for some testing; I'll also make sure to check the output of w to see if anyone else is logged in.

aneeshusa · 2016-02-05T22:08:45Z

FYI: This is my fault for not updating the documentation earlier, but @larsbergstrom we should be using the install_salt script from the repo to install salt, instead of just brew install saltstack. The install script keeps the Salt version pinned, which is helpful because sometimes there are compatibility problems between versions (see this commit for an example) so installing the latest from brew isn't a good idea. I've changed to the right Salt version on servo-macpro1 and updated the wiki page.

aneeshusa · 2016-02-05T23:48:53Z

@larsbergstrom I can't seem to build successfully - I'm getting a lot of errors that say error: couldn't create a temp dir: No such file or directory (os error 2). It looks like the TMPDIR variable that's being set in ~/exports points to a nonexistent directory.

larsbergstrom · 2016-02-06T00:27:15Z

Gah! Just unset TMPDIR. Sorry about that!

On Fri, Feb 5, 2016 at 5:48 PM, Aneesh Agrawal notifications@github.com
wrote:

@larsbergstrom https://github.com/larsbergstrom I can't seem to build
successfully - I'm getting a lot of errors that say error: couldn't
create a temp dir: No such file or directory (os error 2). It looks like
the TMPDIR variable that's being set in ~/exports points to a nonexistent
directory.

—
Reply to this email directly or view it on GitHub
#9504 (comment).

larsbergstrom · 2016-02-06T02:21:16Z

@aneeshusa Thanks a ton for fixing up the wiki instructions! I'm sorry - I had seen the "warning, this needs an update" comment there, but when I ran through and fixed up the worst of the bits that were wrong, I assumed that was it. Thanks again for all your help with our infra!


        Auto merge of #211 - aneeshusa:link-autoconf-correctly, r=larsbergstrom

Link autoconf after installing autoconf213 on OS X This needs to be tested manually on one of the Mac builders before merging. Use --overwrite because autoconf and autoconf213 are installed, and either one may have been linked beforehand. They overlap in the files they provide, and Homebrew will refuse to destructively overwrite any symlinks without an explicit flag. There's no built-in Salt module for brew link, so for now this is a quick fix. This provides some idempotence for repeated Salt highstate runs by checking the directory where Homebrew keeps track of linked packages (and linking autoconf if it is not linked). It's unknown whether this approach properly handles updates to one or both packages. A better approach would be to write a custom Salt state (perhaps homebrew.linked); even if this state works properly, a custom state would be more declarative and could be upstreamed to Salt proper. Closes servo/servo#9504.  [<img src="https://reviewable.io/review_button.svg" height="40" alt="Review on Reviewable"/>](https://reviewable.io/reviews/servo/saltfs/211)

KiChjang added the I-bustage label Feb 2, 2016

mmatyas mentioned this issue Feb 2, 2016

Fix build error on AArch64 #9492

Merged

This was referenced Feb 2, 2016

Update rust-url #9498

Merged

Bump quickersort version. #9500

Merged

fix #8461 #9471

Merged

Quality: Fix CS #9483

Merged

Ensure that text nodes don't inherit opacity. #9503

Merged

Update ipc-channel to get deadlock fix for Linux. #9496

Merged

jdm mentioned this issue Feb 2, 2016

Use the select macro on the paint thread. #9493

Merged

larsbergstrom mentioned this issue Feb 2, 2016

Update web-platform-tests to revision 7f2f85a88f434798e9d643427b34b05fab8278c6 #9458

Merged

aneeshusa mentioned this issue Feb 2, 2016

Link autoconf after installing autoconf213 on OS X servo/saltfs#211

Merged

bors-servo closed this in servo/saltfs#211 Feb 6, 2016

2015xli mentioned this issue Feb 13, 2016

Helps on the build issue on Mac OS X servo/saltfs#221

Closed

servo / servo

Mac builders fails to compile harfbuzz #9504

Mac builders fails to compile harfbuzz #9504

KiChjang commented Feb 2, 2016

paulrouget commented Feb 2, 2016

nox commented Feb 2, 2016

paulrouget commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 3, 2016

larsbergstrom commented Feb 4, 2016

aneeshusa commented Feb 5, 2016

larsbergstrom commented Feb 5, 2016

aneeshusa commented Feb 5, 2016

aneeshusa commented Feb 5, 2016

aneeshusa commented Feb 5, 2016

larsbergstrom commented Feb 6, 2016

larsbergstrom commented Feb 6, 2016

servo / servo

Join GitHub today

Mac builders fails to compile harfbuzz #9504

Mac builders fails to compile harfbuzz #9504

Comments

KiChjang commented Feb 2, 2016

paulrouget commented Feb 2, 2016

nox commented Feb 2, 2016

paulrouget commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 2, 2016

aneeshusa commented Feb 2, 2016

larsbergstrom commented Feb 3, 2016

larsbergstrom commented Feb 4, 2016

aneeshusa commented Feb 5, 2016

larsbergstrom commented Feb 5, 2016

aneeshusa commented Feb 5, 2016

aneeshusa commented Feb 5, 2016

aneeshusa commented Feb 5, 2016

larsbergstrom commented Feb 6, 2016

larsbergstrom commented Feb 6, 2016