Fetch TORCH_PYTHON_SRCS filelists from build_variables

[malfet]

In build-variables.bzl split filelist into libtorch_python_core_sources and libtorch_python_distributed_sources
Move jit passes from glob_libtorch_python_sources() to libtorch_core_jit_sources filelist

Validated that original TORCH_PYTHON_SRCS filelist matches one in build_varaiables.bzl by running the following script:

import os

def read_file(path):
    with open(path) as f:
        return f.read()

def get_cmake_torch_python_srcs():
    caffe2_cmake = read_file("torch/CMakeLists.txt")
    start = caffe2_cmake.find("set(TORCH_PYTHON_SRCS")
    end = caffe2_cmake.find(")", start)
    return caffe2_cmake[start:end+1]

def get_cmake_torch_python_srcs_list():
    _srcs = get_cmake_torch_python_srcs()
    unfiltered_list = [x.strip() for x in _srcs.split("\n") if len(x.strip())>0]
    return [x.replace("${TORCH_SRC_DIR}/","torch/") for x in unfiltered_list if 'TORCH_SRC_DIR' in x]

import imp
build_variables = imp.load_source('build_variables', 'tools/build_variables.bzl')
libtorch_python_sources = set(build_variables.libtorch_python_core_sources)
torch_python_srcs = set(get_cmake_torch_python_srcs_list())

print(set.difference(libtorch_python_sources, torch_python_srcs))

Test Plan: CI

[dr-ci]

💊 Build failures summary and remediations

As of commit 574ef72 (more details on the Dr. CI page):

---

• 1/1 failures possibly* introduced in this PR
  • 1/1 non-CircleCI failure(s)

---

ci.pytorch.org: 1 failed

• Failed: pr/py3.6-clang7-rocmdeb-ubuntu16.04

---

This comment was automatically generated by Dr. CI (expand for details).

Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions on the GitHub issue tracker.

See how this bot performed.

This comment has been revised 11 times.

[malfet]

pytorch_linux_xenial_cuda10_2_cudnn7_py3_gcc7_build
failure should be fixed by #37275

[malfet]

Rebasing, just to make sure everything is still green.

[facebook-github-bot]

@malfet has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@malfet is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

[facebook-github-bot]

@malfet merged this pull request in d294c06.