caffe2 install VS2019 CUDA 10.1 lib\torch.lib : fatal error LNK1248: image size (10028FA9F) exceeds maximum allowable size (FFFFFFFF) #27840
Labels
Comments
elnazsn1988
changed the title
|
Related issue: #27215. Actually, the static build is harder to fix because splitting things out won't work. |
|
One thing you could try first is to set set MSVC_Z7_OVERRIDE=OFF |
|
@peterjc123 Im trying that now, will update on progress. Is there any simpler way to get caffe2 up and running? |
|
@elnazsn1988 Another thing you could try is to use an earlier version of VS 2017, because STL may be simpler and smaller. BTW, could you please sort all the object files by size, and post the list here? It helps us to reduce the size or find the root cause. |
|
@peterjc123 sure, apologies for the elemntary question but could you ellaborate on which files the object files refer to specifically? |
|
All the object files (with the extension |
|
@peterjc123 Hi - so now after using set MSVC_Z7_OVERRIDE=OFF , it builds succesfully, then when running test : python -c "from caffe2.python import workspace; print(workspace.NumCudaDevices())" it returns : is there a link where I can upload the .obj files to show you, there are a high number of files it seems. |
|
I don't need files, just file names. (Or file paths) You could just post to Github Gist and post the link here. |
|
@elnazsn1988 Update: You will need to |
|
@peterjc123 so I should set set BUILD_PYTHON=ON in the build_windows.bat file and rerun? this is the bottom of the windows_build.bat file in its current configuration : |
|
@peterjc123 checking for all the .obj files under pytorch/build/ yields: |
|
@peterjc123 @ezyang I came across this post : #6259 , should I delete all caffe2 and try with Anaconda? there seems to be a mismatch between files in Build and in Caffe2 itself, two proto files exist with one having cafe2_pb2 and the other not having it. When copying all and pasting to both folders, errors are removed. Am now stuck at No module named caffe2_pybind11_state and assuming this is also a setup error, would it be beneficial to try with anaconda? as I have 3 python versions (python 3.7 64bit and 3.7 32bit, 2.7 64 bit) is getting a bit impossible to track whats what. |
|
We need Python3 actually. |
|
@peterjc123 so I had previously used python 3.7, then read this tutorial : http://gianni.rosagallina.com/en/posts/2018/10/04/caffe2-gpu-windows-1.html#step-4-build and moved to python 2.7 as I was getting the same errors with pybind11_state missiing. Should I delete all caffe2 folders and pytorch folders, copy python_37 to the lib folder under build, and resubmit build_windows, or should I just change the python saved under build to python_37 and resubmit? |
|
No need for this, just ensure your |
|
Is it possible to have nightly static builds? Would avoid having these issues with static builds that keep popping up. |
|
Hey @peterjc123 , I am too trying to build caffe2 from source on windows and getting the same error. log: As you mentioned, How do I set the Thanks a lot! |
|
We resplit the cpu and cuda parts of PyTorch, so this error should no longer be happening on master. |
|
Hey @ezyang, I tried again with the master but still facing the same error. Also, as mentioned by @peterjc123 , how do I set the BUILD_PYTHON flag to ON. Here is my build_windows.bat file:
Could you please have a look at this! Thanks a lot for your help! |
|
Can you set |
|
Hey @ezyang, But now I am facing a new error again. log: (a small part of a long list of the same type of errors) This almost took around 2 hours to get to this point! I hope I am getting closer to building caffe2 on my machine!! Thanks a lot for your help!! |
|
Seems like fbgemm isn't working. I guess you can workaround it by setting |
|
Hey @ezyang, Thanks a lot! |
|
Hey @ezyang, Now I am running with this error after setting the log: |
|
@veer5551 This error is intermittent. Just retry a few times. |
|
Hey @peterjc123 and @ezyang, This is what I am getting now and the folder size is already 50+ GB!! log: Please if you could have a look into this and help me setup caffe2 for Python! Thanks a lot for your help! |
|
@veer5551 Why do you need that? PyTorch already contains the python interface of caffe2. |
|
@peterjc123 so does that mean If I have Pytorch Installed, I will be able to use Caffe2 via python interface forcaffe2 models? If Is that what is meant,
Where did I go wrong while using the caffe2 here, and what does this error signify? Thanks a lot! |
|
@veer5551 Oh, you may need to load all the DLLs under |
|
@veer5551 If this solves the problem, I'll put up a fix to resolve that. |
|
Thanks for the suggestion! Thanks! |
|
I guess that has worked! I am able to at least get a prediction score for a particular class for classification. My focus more would be on the detection part though. Well, I need to check for that now! Just one doubt, Thanks a lot once again! |
Yes, absolutely. But still I'll work on a fix to avoid doing that. |
|
@peterjc123, |
peterjc123
added a commit
to peterjc123/pytorch
that referenced
this issue
Jun 24, 2020
Summary: Addresses pytorch#27840 (comment). Contains a bunch of fixes (pytorch#39376 + pytorch#39334 + pytorch#38302 + pytorch#35362) Pull Request resolved: pytorch#39513 Differential Revision: D22190761 Pulled By: malfet fbshipit-source-id: b2d52f6cb16c233d16071e9c0670dfff7da2710e (cherry picked from commit e2201e2)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
馃悰 Bug
Hi all I'm trying to build Caffe2 on Pytorch from Binaries, have used Python 2.7 with VS 2019 and CUDA 10.1. Cmake donwloaded manually and also updated via Conda as read from other users posting my issue. BUILD_SHARED_LIB=OFF per instructions in build_windows.bat, get error :
To Reproduce
Steps to reproduce the behavior:
Warning Generated:
CMake Warning:
Manually-specified variables were not used by the project:
The text was updated successfully, but these errors were encountered: