bazel_external: Issue from drake dir being on the Python path, and drake/common/__init__.py being auto-created

[EricCousineau-TRI]

This is the more specific version of this comment: #7871 (comment)

From @pvarin's StackOverflow post:
https://stackoverflow.com/q/61802860/7829525

To reproduce, see my updated example:
drake_bazel_external/apps/bar.py (branch)

Issue 1: Repositories are slapped onto PYTHONPATH.
Issue 2: Autocreated __init__.py

When running, this is the output that I see: (with <-- annotations added)

$ bazel build //apps:bar
$ alias bash-isolate='env -i HOME=$HOME DISPLAY=$DISPLAY SHELL=$SHELL TERM=$TERM USER=$USER PATH=/usr/local/bin:/usr/bin:/bin bash --norc'
$ bash-isolate -c bazel-bin/apps/bar

path:
  {source_tree}/apps
  {runfiles}
  {runfiles}/drake/bindings
  {runfiles}/lcmtypes_bot2_core/lcmtypes
  {runfiles}/lcmtypes_bot2_core
  {runfiles}/lcmtypes_robotlocomotion/lcmtypes
  {runfiles}/lcmtypes_robotlocomotion
  {runfiles}/meshcat_python/src
  {runfiles}/spdlog
  {runfiles}/meshcat_python
  {runfiles}/lcm
  {runfiles}/ignition_math
  {runfiles}/drake                            <-- Issue 1: Repositories are slapped onto PYTHONPATH.
  {runfiles}/drake_external_examples
  /usr/lib/python36.zip
  /usr/lib/python3.6
  /usr/lib/python3.6/lib-dynload
  /usr/lib/python3/dist-packages

common: {runfiles}/drake/common/__init__.py   <-- Issue 2: Autocreated __init__.py

Traceback (most recent call last):
  File "{runfiles}/drake_external_examples/apps/bar.py", line 24, in <module>
    from common import foo
ImportError: cannot import name 'foo'

Note that common should point to drake_external_examples/common/__init__.py, not drake/common/__init__.py.

FTR The overall correct decision is to import using the workspace name. Changing:

from common import foo

to

from drake_external_examples.common import foo

FYI @jwnimmer-tri

[jwnimmer-tri]

With the "Issue 1, 2" callouts above, I can understand what's happening and why it causes friction.

I think legacy_create_init mentions before were confusing me. I don't see how that's relevant here. For example, it would be easy to depend on something in drake that brought in even a real __init__.py that we wrote for ourselves. In general from drake.stuff import other_stuff should work.

It seems to me the only problem is Issue 1 biting us. To mitigate, we either need to (a) change how bazel python sets up the import path project-wide, or (b) show users how to correctly spell imports in the bazel world.

Since (a) seems like a project wide, rcfile-type setting, it's not fixable in Drake anyway. Maybe a prototypical bazelrc in drake-external-examples.

However, doing (b) should be easy and sufficient to resolve the question. I think the only action here is to add a py_library to drake-external-examples that shows how to import your own python code and pydrake code at the same time?

[EricCousineau-TRI]

I think the only action here is to add a py_library to drake-external-examples that shows how to import your own python code and pydrake code at the same time?

SGTM. Will submit strawman example.

[EricCousineau-TRI]

Also, for the record, I have one, but it's reeks of "unofficialness":
https://github.com/EricCousineau-TRI/repro/blob/39f79009a2e89b987f072276d1921a282f63e6a1/python/bazel_py_example/mid/py3_bin.py#L3

[jwnimmer-tri]

Eric is working on the PR for this, so changed assignment.

[jwnimmer-tri]

The PR RobotLocomotion/drake-external-examples#175 died on the vine. Closing this as "won't fix". Any anyone is interested, they can work on resurrecting that PR.