[FYI] PyTorch and Caffe2 repos getting closer together

[Yangqing]

Over the last year and a half the Caffe2 project has invested heavily in high-performance computation, mobile inference, and a variety of large-scale system research. Over the course we have made quite a few good choices:

• At Facebook we support both PyTorch and Caffe2 for the wide range of AI use cases.
• Caffe2 has focused on scalable systems and cross-platform support, and PyTorch has focused on research flexibility with a truly Pythonic interface.
• We have jointly pushed the ONNX project for smooth framework transfers.
• Caffe2 and PyTorch have shared the dev infrastructure for a while: such as the CI (https://ci.pytorch.org/) system.

With the increasing amount of shared code and common libraries (such as NNPACK as well as integration work such as MKLDNN), sharing the codebase and dev infra has been challenging. For example, we have ONNX bot and onnx-fb-universe repository to sync and test the master of several repositories.

As a step to bring our engineering efforts together, we are merging the Caffe2 repository into the PyTorch repository - we picked the PyTorch repo as the one to combine into, because this will incur minimal overhead for the python user community. These changes will tentatively be made this Friday (March 30, 2018) and @orionr is taking lead on the engineering fronts of repo merging.

We'll keep you updated about the progress on developer support. If you are a pytorch user, check out pytorch/pytorch#6032 as well.

What development workflow will change?

As a Caffe2 user/developer, the only difference you will encounter for now is to do git clone git@github.com:pytorch/pytorch.git for the new combined repo. Nothing else changes: your native build will continue to use CMake from the repo root, and the tools and binaries will stay in the same location for now.

What about my local changes?

As we are not moving files drastically (torch and caffe2 remain the top level folders), if your local change is applied on top of the current Caffe2 master commit, you most likely will be able to create a patch and apply it to the new repo's master commit.

What would the new codebase look like?

We are putting the root folder of the two repos together. Most of the folder structure will be a merge of both, but we will also consolidate some of the files - such as CMake files and build-related scripts - to make the codebase clean.

Does this mean I will have to use Python for deployment?

No. You will be able to continue using the native library as Caffe2 does today. We will continue to have both native and python libraries: native library eases cross-platform deployment, and python library eases quick iterations and experimentation.

What happens to native dependencies like glog, gflags and protobuf?

We are planning to make it optional, so you can choose to build the native lib with or without it. Specifically, for python deployment, we are working on a solution to link protobuf locally to avoid versioning conflicts - something that many of you have kindly shared feedback with us about in the repo.

What happens to the current issues and pull requests?

We will assist you to move the pull request code changes to the new repo. GitHub unfortunately does not have an automatic way to move issues across repos, so we are working on a way to transfer them or wrap them up. Please kindly post any new issues on the PyTorch repository.

[qfgaohao]

What does the future look like? Does it mean caffe2 will be the runtime library and C interface, and Pytorch will be the python interface and the extensive libraries?

[Yangqing]

@qfgaohao Yes, approximately so - we expect this to improve engineering efficiency quite a lot, as well as making model deployment more easy for AI research and production. We'll keep folks updated.

[ahkarami]

Dear @Yangqing,
Thank your for your nice work. I have 2 questions:
1- Does it mean that in the near future these changes, will make the deploying PyTorch models into production task easier? (i.e., converting PyTorch models into C++, or in other words, Has the plan of PyTorch team changed about designing a new C++ production-ready runtime on top of PyTorch?) [You can refer to Deployment in c++ project]
2- What is the future plan of PyTorch or Caffe2 for Serving Models? (I mean that is there any plan to release something like TensorFlow Serving or MXNet Model Server (MMS)?)

[lupesko]

@Yangqing what does this mean for ONNX in your view?

[sridharmahadevan]

What a blunder to link PyTorch -- a really elegant Deep Learning package -- with Caffe2, a far different environment that has almost nothing in common with PT. Sad to see this regressive step. Hope a separate version of PyTorch will continue to be supported.