Buck parses build files in buck-out

[davido]

Trying to upgrade Buck to recent master: [1] reveals that buck test is inherently broken. It interferes with Eclipse output directory (written to buck-out/eclipse) with errors like:

$ buck test --include api
[-] PARSING BUCK FILES...FINISHED 1,0s
BUILD FAILED: //buck-out/eclipse/test/com/google/gerrit/acceptance/server/project:server-project-flaky: parameter 'srcs': no such file or directory 'buck-out/eclipse/test/com/google/gerrit/acceptance/server/project/ProjectWatchIT.java'

Note, that we are using Eclipse and telling it to use buck-out/eclipse as output drectory:

<?xml version="1.0" encoding="UTF-8"?>
<classpath>
    <classpathentry kind="src" output="buck-out/eclipse/test" path="gerrit-acceptance-framework/src/test/java"/>
    <classpathentry kind="src" output="buck-out/eclipse/test" path="gerrit-acceptance-tests/src/test/java"/>

So Eclipse is copying all files (except *.java) to this directory:

$ ls -1 buck-out/eclipse/test/com/google/gerrit/acceptance/server/project/
BUCK
CustomLabelIT.class
LabelTypeIT.class
ProjectWatchIT.class

My impression is that Buck is trying to parse Buck files from this output directory.
I've tracked down the problem to this commit:

35cb495 is the first bad commit

Steps to reproduce:

1. clone most recent Gerrit master: git clone https://gerrit.googlesource.com/gerrit
2. replace Buck version with 35cb495 in .buckversion
3. run buck test --include api -> all is fine
4. generate Eclipse project files with: tools/eclipse/project.py
5. start Eclipse, import gerrit project, refresh gerrit project in Eclipse to force building to buck-out/eclipse
6. run buck test --include api is failing with error mentioned above

[1] https://gerrit-review.googlesource.com/72682

[Coneko]

You shouldn't be writing or modifying files in buck-out, it will cause problems in general.

Can you change your eclipse project to output to a directory that isn't in buck-out?

It's pretty silly of Buck to be parsing build files in there though.

[davido]

We could probably introduce eclipse-outfor that, yes.

[davido]

Moving Eclipse output from buck-out fixed it. However, I figured out, that Buck even interferes with itself. All you need to shoot yourself in the foot, is to have such innocent rule:

genrule(
  name = 'foo',
  srcs = glob(['**']),
  [...]

Now you are screwed up, because Buck copying all files to foo__srcs, including BUCK file itself, and later, when you are trying to run buck test (note, that buck build * still works) it will complain that

//buck-out/.../foo: is not there or something

So now, all glob(['**']) must be extended with srcs = glob(['**'], excludes = ['BUCK']), as in : [1]. May be this should be added to the documentation? Now it's claiming that:

Buck automatically excludes its own output, e.g. <code>buck-out</code>,

https://gerrit-review.googlesource.com/#/c/72684/1/polygerrit-ui/app/BUCK@41

[Coneko]

Good point.

@shs96c something else to keep in mind for how the new FileHashCache should work.

[LegNeato]

From what I can gather not really knowing the code, DirectoryTraversal can take ignore paths:

buck/src/com/facebook/buck/io/DirectoryTraversal.java

Line 41 in ad05aa3

public DirectoryTraversal(Path root, ImmutableSet<Path> ignorePaths) {

It looks like some places it is called don't ever put ignore paths in:

https://github.com/facebook/buck/search?utf8=%E2%9C%93&q=DirectoryTraversal%28

Then again, the project filesystem appears to ignore it:

buck/src/com/facebook/buck/io/ProjectFilesystem.java

Line 209 in 5a3ccc0

builder.add(Paths.get(BuckConstant.BUCK_OUTPUT_DIRECTORY));

And parsing build files appears to use it:

buck/src/com/facebook/buck/model/FilesystemBackedBuildFileTree.java

Line 81 in fab3812

if (ignoredPaths.contains(dir)) {

Might be a good place to poke around and send in a patch if you find the issue!