protobuf_gen breaks other goals if config flags are missing

[patricklaw]

$ ./pants goal binary src/main/python/apache/aurora/executor/bin:thermos_executor

20:55:57 00:00 [main]
               (To run a reporting server: ./pants goal server)
20:55:58 00:01   [bootstrap]
20:55:58 00:01   [setup]
20:55:58 00:01     [parse]
20:55:58 00:01   [jvm-locate]
20:55:58 00:01   [jvm-locate]
20:55:58 00:01   [jvm-locate]
20:55:58 00:01   [jvm-locate]
20:55:58 00:01   [jvm-locate]
               SUCCESS
Traceback (most recent call last):
  File "/home/josh/vendor-src/github.com/apache/incubator-aurora/build-support/pantsbuild.pants-0.0.20.pex/.bootstrap/_twitter_common_python/pex.py", line 223, in execute
    self.execute_entry(entry_point, args)
  File "/home/josh/vendor-src/github.com/apache/incubator-aurora/build-support/pantsbuild.pants-0.0.20.pex/.bootstrap/_twitter_common_python/pex.py", line 271, in execute_entry
    runner(entry_point)
  File "/home/josh/vendor-src/github.com/apache/incubator-aurora/build-support/pantsbuild.pants-0.0.20.pex/.bootstrap/_twitter_common_python/pex.py", line 294, in execute_pkg_resources
    runner()
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/bin/pants_exe.py", line 196, in main
    _run()
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/bin/pants_exe.py", line 177, in _run
    result = command.run(lock)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/commands/goal.py", line 262, in run
    return engine.execute(context, self.phases)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/engine/engine.py", line 50, in execute
    self.attempt(context, phases)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/engine/round_engine.py", line 170, in attempt
    phase_executors = list(self._prepare(context, phases))
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/engine/round_engine.py", line 164, in _prepare
    self._visit_phase(phase, context, phase_info_by_phase)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/engine/round_engine.py", line 156, in _visit_phase
    self._visit_phase(phase_dependency, context, phase_info_by_phase)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/engine/round_engine.py", line 156, in _visit_phase
    self._visit_phase(phase_dependency, context, phase_info_by_phase)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/engine/round_engine.py", line 156, in _visit_phase
    self._visit_phase(phase_dependency, context, phase_info_by_phase)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/engine/round_engine.py", line 120, in _visit_phase
    task = task_type(context, task_workdir)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/backend/codegen/tasks/protobuf_gen.py", line 55, in __init__
    'protoc'
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/binary_util.py", line 156, in select_binary
    binary_path = self.select_binary_base_path(base_path, version, name)
  File "/export/hdc3/home/josh/.pex/install/pantsbuild.pants-0.0.20-py2-none-any.whl.64b7789906d084affad728731eed3e74c6e5236d/pantsbuild.pants-0.0.20-py2-none-any.whl/pants/binary_util.py", line 108, in select_binary_base_path
    return os.path.join(base_path, *(middle_path + [version, name]))
  File "/usr/local/lib/python2.7/posixpath.py", line 77, in join
    elif path == '' or path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'

[ericzundel]

This may just be the tip of the iceberg. I fixed the protobuf defaults and ran into some more. I'll get this target to compile and upload a patch.

[patricklaw]

I also ran into a bunch: jvm, protobuf, ivy. Presumably we can do a much more stringent audit after the new options system lands.

[ericzundel]

I am moving our defaults from pants.ini into default= arguments to fix it. I think that helps us meet the desired endgoal of an empty pants.ini, doesn't it?

[patricklaw]

Yup! It's a good idea regardless.

[ericzundel]

protobuf-gen now has some reasonable defaults as of commit 35d8a74. However, I'm pretty sure the Aurora project is not out of the woods:

1. They don't have a BUILD.tools which means the tool external artifacts aren't resolving
2. After fixing that, I can't get any further in their build process on my Mac:

Cannot satisfy requirements: [PythonRequirement(mesos==0.18.0)]

I've asked for help in their IRC channel.

[patricklaw]

We resolved that issue internally, assuming it was for the same reason. We ended up needing to force the platforms and compatibility parameters to the python targets in question in order to get the dependency resolution to accept a linux-only mesos dep.

[ericzundel]

protobuf_gen does not need settings in pants.ini any more.