Unable to build thrift-0.9.2 on Yosemite

[ascandella]

Greetings. I'm trying to update the thrift formula to 0.9.2 (from 0.9.1) since it contains many useful changes (https://github.com/apache/thrift/blob/master/CHANGES)

I started with a brew edit thrift and munged the URL and SHA1 of the stable block. I ended up adding a depends_on "bison" => :build to the stable block as well to squash errors about bad bison versions (although that seems like it should not be necessary due to the --without-tests exclusion...).

Here are my gist logs trying to build it: https://gist.github.com/7c9a718147444505a313

It appears to error-out at tr1/functional, which seems to be a c++11 issue? I'm more of a Python/Ruby guy myself so I'm not very helpful here, but I'd like to see if I can help contribute a working 0.9.2 formula.

Any tips?

[ascandella]

This seems to be a known-ish issue that was allegedly fixed in 0.9.1: https://issues.apache.org/jira/browse/THRIFT-2266
https://issues.apache.org/jira/browse/THRIFT-2229

[DomT4]

Unless the situation has changed, you may need to patch the tr1 stuff. It has caused issues before on newer systems. There seems to be an upstream patch here, which may still work, as well as additional patches for a version back here. Check to see what's still relevant, what has been merged, etc and rebase accordingly.

[ascandella]

Yea, I optimistically read that comment to mean that all the patches would be unnecessary once 0.9.2 is released. I'll figure out what combination is still necessary and try to get a working PR together. Thanks.

[DomT4]

Ah, if only things were that cooperative, heh. No worries. Feel free to shout if it drives you too crazy.

[ascandella]

Interestingly enough, this seems to be limited to the tests -- lib/cpp/test/processor/ProcessorTest.cpp seems to be the only place requiring tr1 directly. It shouldn't be building this file in the first place, due to the --without-tests flag, and if I ignore the errors (via brew install -vd thrift) I can continue on and everything is built fine (without any patches!).

So I think it's close, but still need to figure out why that test file is trying to get built despite the flag to ./configure

[DomT4]

If it's just the one test element, you could always kill that particular file and see what the ./configure process does without it. If we're not using the tests, we can always kill them off for now if they're causing problems and doing so resolves them.

[ascandella]

When you say "kill them off for now", are you suggesting adding a step to install that actually rm's the file? Or what's the least hacky way to implement this? I feel like the --without-tests should work but it does not seem to be doing the trick.

I'll have some more time to look at this issue this weekend.

[DomT4]

Yeah, essentially. If something should be behaving a certain way in the configure script, but some file is blocking that, it isn't necessary a no-go just to axe that troublesome file temporarily if it solves the problem, with rm. Something like:

rm "tests/troublesome_test"
system "./configure", "--prefix=blahblah", "--without-tests", "--so-on-so-forth"
system "make", "install"

If it doesn't work, It doesn't work, but if it does Homebrew isn't always anti-merging something like that until upstream have a fuller fix/grip on the recurring issue. In those situations as long as there's an upstream bug report linked above the rm it's easy enough for future updaters to monitor over time.

[DomT4]

Or you could try completely commenting out those troublesome lines in that file with an inreplace or a patch, and seeing if the configure process builds successfully with those lines commented out.

[ascandella]

Opened an issue on Thrift here: https://issues.apache.org/jira/browse/THRIFT-2827 to make sure I'm not going about this totally the wrong way.

[ascandella]

Got the first issue resolved and merged into master. Uncovered another issue: https://issues.apache.org/jira/browse/THRIFT-2799 that seems to be fixable by patching the Makefile.am as mentioned in the ticket.

Now getting this when installing with my formula:

/bin/sh ../../../libtool  --tag=CXX   --mode=link clang++ -Wall -Wextra -pedantic -g -O2 -L/usr/local/Cellar/boost/1.56.0/lib  -o processor_test processor/ProcessorTest.o processor/EventLog.o processor/ServerThread.o libprocessortest.la ../../../lib/cpp/libthrift.la ../../../lib/cpp/libthriftnb.la -L/usr/local/Cellar/boost/1.56.0/lib -levent -lboost_unit_test_framework -lssl -lcrypto -lpthread
libtool: link: clang++ -Wall -Wextra -pedantic -g -O2 -o .libs/processor_test processor/ProcessorTest.o processor/EventLog.o processor/ServerThread.o -Wl,-bind_at_load  -L/usr/local/Cellar/boost/1.56.0/lib ./.libs/libprocessortest.a ../../../lib/cpp/.libs/libthrift.dylib ../../../lib/cpp/.libs/libthriftnb.dylib -levent -lboost_unit_test_framework -lssl -lcrypto -lpthread
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Which, when I drop into a shell, shows:

clang++ -v -Wall -Wextra -pedantic -g -O2 -o .libs/processor_test processor/ProcessorTest.o processor/EventLog.o processor/ServerThread.o -Wl,-bind_at_load  -L/usr/local/Cellar/boost/1.56.0/lib ./.libs/libprocessortest.a ../../../lib/cpp/.libs/libthrift.dylib ../../../lib/cpp/.libs/libthriftnb.dylib -levent -lboost_unit_test_framework -lssl -lcrypto -lpthread
Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin14.0.0
Thread model: posix
 "/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -dynamic -arch x86_64 -macosx_version_min 10.10.0 -o .libs/processor_test -L/usr/local/Cellar/boost/1.56.0/lib processor/ProcessorTest.o processor/EventLog.o processor/ServerThread.o -bind_at_load ./.libs/libprocessortest.a ../../../lib/cpp/.libs/libthrift.dylib ../../../lib/cpp/.libs/libthriftnb.dylib -levent -lboost_unit_test_framework -lssl -lcrypto -lpthread -lc++ -lSystem /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/clang/6.0/lib/darwin/libclang_rt.osx.a
Undefined symbols for architecture x86_64:
  "_main", referenced from:
     implicit entry/start for main executable
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

However I can build fine from the command line (e.g. without homebrew) and I'm a little out of my element with the interactions of clang and c++11 and boost, which seems to be the issue.

Fairly sure I could hack around it by not trying to compile the tests, but I'm learning more as I dig in and feel there is probably a reasonable solution that includes building the tests. Will dig in some more tomorrow.

[DomT4]

Nice one 👍.

After the last depends_on line, try adding:

  fails_with :clang do
    cause "Clang compile fails"
  end

And then building.

[ascandella]

Tried that, same error. Seems like it should build under clang (and does when I do the ./configure; make dance outside of homebrew). I'm sure there's enough different in my environment variables between running manually and inside of brew but I'm not really clear where to investigate next.

[cambridgemike]

@sectioneight Do you have a working formula for installing 0.9.2 on yosemite? I'm in the same boat and was going to tackle updating it myself, but looks like you've gotten most of the way there.

Do you have something you could share?

[ascandella]

@cambridgemike It's not fully working but it's getting there:

https://github.com/sectioneight/homebrew/commits/update-thrift

ascandella@313aac6

Basically there is a single piece that is not working well: ProcessorTest.cpp. If you run my branch with brew install -vd thrift and ignore the errors you hit linking processor_test you will have a working 0.9.2. Both of the patches (the first one accepted into thrift master, the second one still being tracked) have to do with this test not compiling well under clang.

I'm pretty inexperience with clang and boost so I'm not sure the best way to fix this one... one option is to "patch" the makefiles to not try to build this file at all (since it's just a test) but the engineer in me gets sad giving up like that.

Thoughts?

[cambridgemike]

Hm, the patch data didn't apply cleanly

==> Downloading http://archive.apache.org/dist/thrift/0.9.2/thrift-0.9.2.tar.gz
Already downloaded: /Library/Caches/Homebrew/thrift-0.9.2.tar.gz
==> Verifying thrift-0.9.2.tar.gz checksum
tar xf /Library/Caches/Homebrew/thrift-0.9.2.tar.gz
==> Patching
patching file lib/cpp/test/processor/ProcessorTest.cpp
patching file lib/cpp/test/Makefile.am
Hunk #1 succeeded at 99 with fuzz 1.
Hunk #4 FAILED at 124.
Hunk #5 FAILED at 230.
patch unexpectedly ends in middle of line
2 out of 5 hunks FAILED -- saving rejects to file lib/cpp/test/Makefile.am.rej
patch unexpectedly ends in middle of line
Error: Failure while executing: /usr/bin/patch -g 0 -f -p1

Maybe it's fallen out of sync?

[ascandella]

Weird. thrift-0.9.2 should be frozen at the release... I just wiped my cache and tried again and it applied fine. Also the sha1sum is saved in the formula. Maybe I'm taking crazy pills.

[ascandella]

==> Downloading http://archive.apache.org/dist/thrift/0.9.2/thrift-0.9.2.tar.gz
/usr/bin/curl -fLA Homebrew 0.9.5 (Ruby 2.0.0-481; Mac OS X 10.10.1) http://archive.apache.org/dist/thrift/0.9.2/thrift-0.9.2.tar.gz -C 0 -o /Library/Caches/Homebrew/thrift-0.9.2.tar.gz.incomplete
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 7864k  100 7864k    0     0   102k      0  0:01:16  0:01:16 --:--:-- 1884k
==> Verifying thrift-0.9.2.tar.gz checksum
tar xf /Library/Caches/Homebrew/thrift-0.9.2.tar.gz
==> Patching
patching file lib/cpp/test/processor/ProcessorTest.cpp
patching file lib/cpp/test/Makefile.am
==> ./configure --disable-debug --prefix=/usr/local/Cellar/thrift/0.9.2 --libdir=/usr/local/Cellar/thrift/0.9.2/lib --without-ruby --without-tests --without-php_extension --without-python --without-haskell --without-java --without-pe
rl --without-php --without-erlang

[cambridgemike]

Huh... yeah, IDK, I guess it was me. I checked out 0.9.2 from the thrift repo and made the same changes to Makefile.am and generated a patch, it looked just like yours, but I got the same errors. Some googling indicated it has to do with line endings. shrug

I removed the hunks that were failing and compiled, ignoring the errors.. Seems like it's working. So thanks!

[ascandella]

Fixed in #34433