Python variables are not accessible when script is sourced from a file listed in buildfile.includes section

[davido]

Found on 63e0e52.

Including a python file (VERSION) directly works as expected, particularly variables that defined in it are exposed to the caller:

Given this VERSION file:

 $ cat VERSION
 PLUGIN_VERSION = '1.0'

and gerrit_plugin.bucklet:

 $ cat gerrit_plugin.bucklet
def gerrit_plugin(
    name,
    visibility = ['PUBLIC']):
  from os import path
  ROOT = get_base_path()
  f = '//' + path.join(ROOT, 'VERSION')
  include_defs(f)
  genrule(
    name = name + '__manifest',
    cmd = 'echo ' + PLUGIN_VERSION + '>$OUT',
    out = 'MANIFEST.MF',
  )

And this BUCK build file:

include_defs('//gerrit_plugin.bucklet')
gerrit_plugin(
  name = 'wip',
)

All is just fine:

 $ buck build :wip__manifest
 $ cat buck-out/gen/MANIFEST.MF
 1.0

When we make use of .buckconfig and buildfile.includes section there and include bucklet from a file that listed in this section, it doesn't work, e. g.:

$ cat .buckconfig
[buildfile]
  includes = //bucklets.defs

And include gerrit_plugin.bucklet not directly, but from bucklets.defs that is specified in .buckconfig file:

$ cat bucklets.defs
bucklets = [
  'gerrit_plugin.bucklet',
]

for bucklet in bucklets:
  include_defs('//%s' % bucklet)

And remove the include from BUCK:

 $ cat BUCK
 gerrit_plugin(
  name = 'wip',
)

It doesn't work any more, PLUGIN_VERSION cannt be accessed, even though the file '//VERSION' seems to be correctly imported:

$ buck build :wip__manifest
Using buckd.
[+] PARSING BUILD FILES...0,1s
Traceback (most recent call last):
  File "/home/davido/projects/buck_reproducer/buck-out/tmp/buck_run.pGbINKP8kg/buck6118602169058355247.py", line 1215, in <module>
    main()
  File "/home/davido/projects/buck_reproducer/buck-out/tmp/buck_run.pGbINKP8kg/buck6118602169058355247.py", line 583, in main
    buildFileProcessor.process(build_file.rstrip())
  File "/home/davido/projects/buck_reproducer/buck-out/tmp/buck_run.pGbINKP8kg/buck6118602169058355247.py", line 482, in process
    build_env['BUILD_FILE_SYMBOL_TABLE'])
  File "/home/davido/projects/buck_reproducer/./BUCK", line 9, in <module>
    name = 'wip',
  File "/home/davido/projects/buck_reproducer/gerrit_plugin.bucklet", line 10, in gerrit_plugin
    cmd = 'echo ' + PLUGIN_VERSION + '>$OUT',
NameError: global name 'PLUGIN_VERSION' is not defined
BUILD FAILED: Parse error for BUCK file ./BUCK: End of input at line 1 column 1

[davido]

If someone wonders why we would want to do that, here is the real change [1]

[1] https://gerrit-review.googlesource.com/58892

[davido]

With this reproducer it's easy to see the problem and the workaround: [1].

[1] https://github.com/davido/buck_include_defs_repro

[shs96c]

We've a patch landing for this soon. Thanks for the report and test.

[andrewjcg]

Thanks for such a detailed bug report! This undesirable behavior (and a few others) is caused by how we currently handle "include_defs" and python globals. We're working on remodeling this around python modules, which should fix a number of issues (and I've confirmed it fixes this issue), and hope to have it pushed soon.

[andrewjcg]

Heh, completely missed Simon's comment.