Enable clang-tidy in CI

[goldsborough]

At long last, we will have clang-tidy enabled in CI. For a while I thought I could clean up the project enough to enable clang-tidy with all checks enabled, but I figure it's smarter to set up the minimal checks and at least have those in CI. We can fix more going forward.

@ezyang @apaszke

[ezyang]

The job doesn't seem to be failing on errors. On https://travis-ci.org/pytorch/pytorch/jobs/435471449 I see:

1.33s$ python tools/clang_tidy.py -c "$(which clang-tidy-6.0)" -d clang-tidy-build -p torch/csrc -r HEAD~1
/home/travis/build/pytorch/pytorch/torch/csrc/jit/interned_strings.h:9:10: error: 'torch/csrc/jit/generated/aten_interned_strings.h' file not found [clang-diagnostic-error]
#include "torch/csrc/jit/generated/aten_interned_strings.h"

[ezyang]

I have quibbles about how the Python script is written, but I'm requesting changes here because it doesn't seem like it is actually flagging errors as failures in CI.

[goldsborough]

What you're seeing are clang compiler errors (clang-diagnostic-error). Those are because I don't do a full build of PyTorch since that would take too long in CI. I only generate the .compile_commands.json and clang-tidy will happily lint 99% of code that it can parse and compile. The clang diagnostics can be ignored. It means we're not linting generated files, but I think that's fine. In any case, there is nothing broken here, it's expected. When there is an actual lint error (which I have verified of course), those will make the CI error.

[goldsborough]

Now that I think of it, I can just run our gen.py script to generate the missing files. I also do that in our doc CI build. In any case the lint job will take 2 minutes and not 20 minutes or however long it takes to build PyTorch

[ezyang]

CR on clang_tidy.py

• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L164 This is not the idiomatic way to pass extra arguments to an underlying executable, when there is a unique such executable (as is this case); these arguments should come after a -- in the command line. See Allow passing through arguments to unittest #11209 for an example of adding this functionality to a Python script using argparse.
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L207 This is not the idiomatic flag name; the idiomatic name is --dry-run with abbreviation -n. See for example https://linux.die.net/man/1/rsync https://linux.die.net/man/1/git-review https://linux.die.net/man/1/hg (Sometimes people accept -s, but that's short for --simulate)
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L59 This is not the correct way to query git programatically. You should use a plumbing command, rather than the porcelain command (see also Git - Plumbing and Porcelain <https://git-scm.com/book/en/v2/Git-Internals-Plumbing-and-Porcelain>_). Using plumbing will reduce the amount of preparatory work you have to do to get the command to give you something machine only (e.g., you don't have to pass --no-pager and --no-color). In this case, because you want to diff against the working tree, you should use git-diff-index. Read the manpage for this command to see how it can be done.
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L110 No, please don't do this. If you pass the base directory, this will walk recursively into the build directory and .git and grab you all of your build products too. Even if you use it on a subdirectory, it will still grab any build products. Use git-ls-files to get an accurate listing of all files in Git which are checked into source control and can be profitably linted.
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L232 This seems like an awfully circuitous way to implement recursive file removal.
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L183 The doc here should mention that regexes implicitly get $ appended to them
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L44 Uhhhhh no, please don't do this. Use https://docs.python.org/2/library/fnmatch.html#fnmatch.fnmatch instead
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L138 That's weird, http://manpages.ubuntu.com/manpages/xenial/man1/clang-tidy-3.7.1.html seems to imply that clang-tidy understands YAML here too. What's going on? In fact, clang-tidy seems to imply that it will automatically try to read .clang-tidy itself
• https://github.com/pytorch/pytorch/blob/5564d8c0af37d74bd9ff0f19d083d065244acc08/tools/clang_tidy.py#L17 I still find this function deeply dubious.

[ezyang]

(sorry, accidentally closed the PR with an incorrect comment, please disregard)

[ezyang]

Message me if you want to chat more about the verbose output situation. I want to understand what's going on here.

[goldsborough]

Hard to respond to your comments directly this way. I'll fix the code you commented on. I think the only open ended question was "why pass the .yaml file to clang-tidy". Believe me, I've spent many hours fighting clang tidy because it wouldn't pick up my .clang-tidy file. It seems a thing about different clang tidy versions. There's unanswered stackoverflow questions about this too. The best way to ensure that all the config gets picked up is to pass it in directly the way I do.

I'll see how hard it is to create a cmake target to generate the files. I think it's more important to have clang tidy in CI as soon as possible than for this to be 100% clean. We could literally already be finding tons of bugs right now

[goldsborough]

No, please don't do this. If you pass the base directory, this will walk recursively into the build directory and .git and grab you all of your build products too. Even if you use it on a subdirectory, it will still grab any build products. Use git-ls-files to get an accurate listing of all files in Git which are checked into source control and can be profitably linted.

Why would you pass the base directory? This gets e.g. torch/csrc as its argument, that would certainly only include source files?

[ezyang]

Believe me, I've spent many hours fighting clang tidy because it wouldn't pick up my .clang-tidy file. It seems a thing about different clang tidy versions. There's unanswered stackoverflow questions about this too. The best way to ensure that all the config gets picked up is to pass it in directly the way I do.

That's fine, but then it should be commented in the code :)

Why would you pass the base directory? This gets e.g. torch/csrc as its argument, that would certainly only include source files?

I've seen too many programs written using os.walk (it worked fine on my machine!) blow up to be that sympathetic to this argument :) It should work without weird performance cliffs in all cases, or we shouldn't provide it at all.

[ezyang]

In terms of priorities, the main things I want to see fixed are:

• Removal of footguns from the Python script (this is mostly from the perspective of, this kind of Python code tends to replicate itself in the codebase if let alone for a while, and I don't want the footguns replicating themselves.)
• Some resolution on the issue where clang-tidy will silently choke if compile commands are busted without erroring. I'm willing to be convinced this is not a problem in practice, but we've had multiple cases in the past where we silently stopped testing code and if there is something easy we can do now to prevent this from happening, I'd prefer it.

Everything else is non-essential.

[facebook-github-bot]

goldsborough has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.