Skip to content

some caffe2 header files are not included on a built torch package  #28753

@qzhong0605

Description

@qzhong0605

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

  1. Build a torch wheel package with the following command:
USE_FFMPEG=1 USE_NUMPY=1  USE_OPENCV=1 USE_ROCM=0 DEBUG=0 python setup.py bdist_wheel --python-tag py3
  1. Install the above built package:
cd dist && pip install torch-1.3.0a0+f622ec8-cp36-cp36m-linux_x86_64.whl
  1. The header files on math and threadpool are not included on the built package
> ls include/caffe2/utils
bench_utils.h  conversions.h  eigen_utils.h    map_utils.h    murmur_hash3.h   proto_wrap.h      smart_tensor_printer.h
cast.h         cpuid.h        filler.h         math-detail.h  proto_convert.h  signal_handler.h  string_utils.h
cblas.h        cpu_neon.h     fixed_divisor.h  math.h         proto_utils.h    simple_queue.h    zmq_helper.h

Expected behavior

The header files on math and threadpool should be included on the built package because they
are included on the other header files, such as on file torch/include/caffe2/utils/math.h

#include "caffe2/core/common.h"
#include "caffe2/core/types.h"
#include "caffe2/utils/math/broadcast.h"
#include "caffe2/utils/math/elementwise.h"
#include "caffe2/utils/math/reduce.h"
#include "caffe2/utils/math/transpose.h"
#include "caffe2/utils/math/utils.h"

Environment

Collecting environment information...
PyTorch version: 1.3.0a0+f622ec8
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: CentOS Linux release 7.5.1804 (Core)
GCC version: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-28)
CMake version: version 3.12.2

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 9.0.176
GPU models and configuration:
GPU 0: Tesla K80
GPU 1: Tesla K80
GPU 2: Tesla K80
GPU 3: Tesla K80

Nvidia driver version: 410.48
cuDNN version: cudnn/lib64/libcudnn.so.7.3.0

Versions of relevant libraries:
[pip3] numpy==1.16.2
[pip3] torch==1.3.0a0+f622ec8
[pip3] torchvision==0.2.3a0+ccbb322
[conda] Could not collect

A method to fix this bug

The setup.py can include the header files on caffe2/utils/math with adding the following codes to it:

diff --git a/setup.py b/setup.py
index 2838b65c0d..a50d06815d 100644
--- a/setup.py
+++ b/setup.py
@@ -801,6 +801,7 @@ if __name__ == '__main__':
                 'include/ATen/cudnn/*.h',
                 'include/ATen/detail/*.h',
                 'include/caffe2/utils/*.h',
+                'include/caffe2/utils/**/*.h',
                 'include/c10/*.h',
                 'include/c10/macros/*.h',
                 'include/c10/core/*.h',

Metadata

Metadata

Assignees

No one assigned

    Labels

    module: buildBuild system issuestriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate module

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions