Cannot download thrift-0.5.0-finagle on OS X 10.10

[Yasumoto]

Digging through https://issues.apache.org/jira/browse/AURORA-867, looks like we need a build of finagle thrift at http://maven.twttr.com/twitter-commons/pants/build-support/bin/thrift/mac/10.10/0.5.0-finagle/thrift

[tw-172-25-132-201 aurora (master)]$ ./pants ./src/test/python:all
<snip>
Traceback (most recent call last):
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/commands/build.py", line 130, in _python_build
    debug=self._verbose)
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/python_builder.py", line 43, in build
    debug=debug).run()
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/test_builder.py", line 88, in run
    rv = self._run_tests([target], stdout, stderr)
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/test_builder.py", line 316, in _run_tests
    with self._test_runner(targets, stdout, stderr) as (pex, test_args):
  File "/Users/jsmith/Python/CPython-2.7.8/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/test_builder.py", line 295, in _test_runner
    builder = chroot.dump()
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/python_chroot.py", line 165, in dump
    self.MEMOIZED_THRIFTS[thr] = self._generate_thrift_requirement(thr)
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/python_chroot.py", line 135, in _generate_thrift_requirement
    return self._generate_requirement(library, PythonThriftBuilder)
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/python_chroot.py", line 127, in _generate_requirement
    sdist = builder.build(interpreter=self._interpreter)
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/code_generator.py", line 96, in build
    self.generate()
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/thrift_builder.py", line 123, in generate
    self.run_thrifts()
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/thrift_builder.py", line 69, in run_thrifts
    if not self._run_thrift(src):
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/backend/python/thrift_builder.py", line 74, in _run_thrift
    select_thrift_binary(self.config),
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/thrift_util.py", line 107, in select_thrift_binary
    return BinaryUtil(config=config).select_binary(thrift_supportdir, thrift_version, 'thrift')
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/binary_util.py", line 162, in select_binary
    with self.select_binary_stream(base_path, version, name) as stream:
  File "/Users/jsmith/Python/CPython-2.7.8/lib/python2.7/contextlib.py", line 17, in __enter__
    return self.gen.next()
  File "/Users/jsmith/.pex/install/pantsbuild.pants-0.0.24-py2-none-any.whl.0285abd76697304cfaec8a361df04094600f0e74/pantsbuild.pants-0.0.24-py2-none-any.whl/pants/binary_util.py", line 147, in select_binary_stream
    raise BinaryUtil.BinaryNotFound((base_path, version, name), accumulated_errors)
BinaryNotFound: Failed to fetch binary ('bin/thrift', '0.5.0-finagle', u'thrift') from any source: (Failed to fetch binary from http://maven.twttr.com/twitter-commons/pants/build-support/bin/thrift/mac/10.10/0.5.0-finagle/thrift: HTTP Error 404: Not Found)

[areitz]

As an alternative, I would be in favor of this patch to pants:

[prometheus pants (master)]$ git diff
diff --git a/src/python/pants/binary_util.py b/src/python/pants/binary_util.py
index 4d87cbb..33698d6 100644
--- a/src/python/pants/binary_util.py
+++ b/src/python/pants/binary_util.py
@@ -41,7 +41,7 @@ _PATH_BY_ID = {
   ('darwin', '11'):     ['mac', '10.7'],
   ('darwin', '12'):     ['mac', '10.8'],
   ('darwin', '13'):     ['mac', '10.9'],
-  ('darwin', '14'):     ['mac', '10.10'],
+  ('darwin', '14'):     ['mac', '10.9'],
 }

[Yasumoto]

That seems counter-intuitive.. why not just fix pants to actually support 10.10?

[areitz]

Pants supports 10.10 just fine, but it treats every OS X release as being binary incompatible. As far as I know, binaries compiled on 10.9 are ABI compatible with 10.10. Rather than forcing recompile (or honestly, copying) of binaries, we can tell pants that using 10.9 binaries on 10.10 is a-ok.

[Yasumoto]

Hm, I disagree on two points-

1. Once these issues are resolved, pants will support 10.10 just fine.

2. My intuition is still that a recompile would still lead to more correctness- I'm happy to help pitch in if there are docs I can follow.

Overall, the attitude signified by this approach is very concerning as a user of pants. If this is the stance of the pants-using community, then I'm going to start a vote on the apache mailing list to migrate off and find an alternative solution.

[Yasumoto]

Spent some time searching for a confirm/deny on ABI compatibility, and the closest I can find so far is from a Technical Q&A article. That said, if y'all have a more official document confirming compatibility, then we can point to that instead and confirm the path forward.

[ericzundel]

What I think we need to do is just copy the 10.9 binaries to a new 10.10
directory in the tool bootstrap repo. That will likely keep working, and
if someone does find an is specific problem, we can update the tool just in
the 10.10 release.
On Oct 17, 2014 3:09 PM, "Joe Smith" notifications@github.com wrote:

Spent some time searching for a confirm/deny on ABI compatibility, and the
closest I can find so far is from a Technical Q&A article
https://developer.apple.com/library/mac/qa/qa1118/_index.html. That
said, if y'all have a more official document confirming compatibility, then
we can point to that instead and confirm the path forward.

—
Reply to this email directly or view it on GitHub
#690 (comment).

[Yasumoto]

Hi all,

Took a look at the pants' pants.ini vs. the one used by aurora, and I've swapped out to use the same pants_support_baseurls which has fixed things up: https://reviews.apache.org/r/27066/diff/#

Thanks!
Joe

[ericzundel]

Just to follow up on this, the issue was that the aurora repo has pants.ini

pants_support_baseurls = [
    'http://maven.twttr.com/twitter-commons/pants/build-support',
  ]

The repo at that URL was not updated for mac OSX 10.10 binaries because the OSS pants had migrated to a github repo for the support binaries. Today we're setting up a new URL to supersede the github repo:

https://dl.bintray.com/pantsbuild/bin/build-support

[Yasumoto]

Exactly- thanks @ericzundel! :)