-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
1.16.0 built from source on RHEL8 fails with: undefined symbol: _ZNSt10filesystem7__cxx114path14_M_split_cmptsEv #6047
Labels
Comments
This was referenced Mar 28, 2024
github-merge-queue bot
pushed a commit
that referenced
this issue
Apr 12, 2024
### Description Possible fix for #6047 based on https://stackoverflow.com/questions/33149878/experimentalfilesystem-linker-error/33159746#33159746 ### Motivation and Context I'm not a cmake expert so there may be a better way to do this. This did allow me to build ONNX 1.16 on RHEL 8. --------- Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: Charles Volzka <42243335+cjvolzka@users.noreply.github.com> Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com> Co-authored-by: G. Ramalingam <grama@microsoft.com>
gramalingam
added a commit
to gramalingam/onnx
that referenced
this issue
Apr 12, 2024
### Description Possible fix for onnx#6047 based on https://stackoverflow.com/questions/33149878/experimentalfilesystem-linker-error/33159746#33159746 ### Motivation and Context I'm not a cmake expert so there may be a better way to do this. This did allow me to build ONNX 1.16 on RHEL 8. --------- Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: Charles Volzka <42243335+cjvolzka@users.noreply.github.com> Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com> Co-authored-by: G. Ramalingam <grama@microsoft.com> Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
gramalingam
added a commit
to gramalingam/onnx
that referenced
this issue
Apr 12, 2024
### Description Possible fix for onnx#6047 based on https://stackoverflow.com/questions/33149878/experimentalfilesystem-linker-error/33159746#33159746 ### Motivation and Context I'm not a cmake expert so there may be a better way to do this. This did allow me to build ONNX 1.16 on RHEL 8. --------- Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: Charles Volzka <42243335+cjvolzka@users.noreply.github.com> Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com> Co-authored-by: G. Ramalingam <grama@microsoft.com> Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
cjvolzka
added a commit
that referenced
this issue
Apr 29, 2024
### Description Possible fix for #6047 based on https://stackoverflow.com/questions/33149878/experimentalfilesystem-linker-error/33159746#33159746 ### Motivation and Context I'm not a cmake expert so there may be a better way to do this. This did allow me to build ONNX 1.16 on RHEL 8. --------- Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: Charles Volzka <42243335+cjvolzka@users.noreply.github.com> Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com> Co-authored-by: G. Ramalingam <grama@microsoft.com> (cherry picked from commit 3f24ef3)
snnn
added a commit
to microsoft/onnxruntime
that referenced
this issue
May 24, 2024
### Description Use a common set of prebuilt manylinux base images to build the packages, to avoid building the manylinux part again and again. The base images can be used in GenAI and other projects too. This PR also updates the GCC version for inference python CUDA11/CUDA12 builds from 8 to 11. Later on I will update all other CUDA pipelines to use GCC 11, to avoid the issue described in onnx/onnx#6047 and microsoft/onnxruntime-genai#257 . ### Motivation and Context To extract the common part as a reusable build infra among different ONNX Runtime projects.
snnn
added a commit
to microsoft/onnxruntime
that referenced
this issue
Jun 3, 2024
…esystem implementation (#20893) ### Description This PR upgrades CUDA 11 build pipelines' GCC version from 8 to 11. ### Motivation and Context GCC8 has an experimental std::filesystem implementation which is not ABI compatible with the formal one in later GCC releases. It didn't cause trouble for us, however, ONNX community has encountered this issue much. For example, onnx/onnx#6047 . So this PR increases the minimum supported GCC version from 8 to 9, and removes the references to GCC's "stdc++fs" library. Please note we compile our code on RHEL8 and RHEL8's libstdc++ doesn't have the fs library, which means the binaries in ONNX Runtime's official packages always static link to the fs library. It is just a matter of which version of the library, an experimental one or a more mature one. And it is an implementation detail that is not visible from outside. Anyway, a newer GCC is better. It will give us the chance to use many C++20 features. #### Why we were using GCC 8? It is because all our Linux packages were built on RHEL8 or its equivalents. The default GCC version in RHEL8 is 8. RHEL also provides additional GCC versions from RH devtoolset. UBI8 is the abbreviation of Red Hat Universal Base Image 8, which is the containerized RHEL8. UBI8 is free, which means it doesn't require a subscription(while RHEL does). The only devtoolset that UBI8 provides is GCC 12, which is too new for being used with CUDA 11.8. And our CUDA 11.8's build env is a docker image from Nvidia that is based on UBI8. #### How the problem is solved Almalinux is an alternative to RHEL. Almalinux 8 provides GCC 11. And the CUDA 11.8 docker image from Nvidia is open source, which means we can rebuild the image based on Almalinux 8 to get GCC 11. I've done this, but I cannot republish the new image due to various complicated license restrictions. Therefore I put them at an internal location in onnxruntimebuildcache.azurecr.io.
gramalingam
added a commit
to gramalingam/onnx
that referenced
this issue
Jun 6, 2024
### Description Possible fix for onnx#6047 based on https://stackoverflow.com/questions/33149878/experimentalfilesystem-linker-error/33159746#33159746 ### Motivation and Context I'm not a cmake expert so there may be a better way to do this. This did allow me to build ONNX 1.16 on RHEL 8. --------- Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: Charles Volzka <42243335+cjvolzka@users.noreply.github.com> Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com> Co-authored-by: G. Ramalingam <grama@microsoft.com> Signed-off-by: Ganesan Ramalingam <grama@microsoft.com>
yf711
pushed a commit
to microsoft/onnxruntime
that referenced
this issue
Jun 18, 2024
### Description Use a common set of prebuilt manylinux base images to build the packages, to avoid building the manylinux part again and again. The base images can be used in GenAI and other projects too. This PR also updates the GCC version for inference python CUDA11/CUDA12 builds from 8 to 11. Later on I will update all other CUDA pipelines to use GCC 11, to avoid the issue described in onnx/onnx#6047 and microsoft/onnxruntime-genai#257 . ### Motivation and Context To extract the common part as a reusable build infra among different ONNX Runtime projects.
yf711
pushed a commit
to microsoft/onnxruntime
that referenced
this issue
Jun 19, 2024
…esystem implementation (#20893) ### Description This PR upgrades CUDA 11 build pipelines' GCC version from 8 to 11. ### Motivation and Context GCC8 has an experimental std::filesystem implementation which is not ABI compatible with the formal one in later GCC releases. It didn't cause trouble for us, however, ONNX community has encountered this issue much. For example, onnx/onnx#6047 . So this PR increases the minimum supported GCC version from 8 to 9, and removes the references to GCC's "stdc++fs" library. Please note we compile our code on RHEL8 and RHEL8's libstdc++ doesn't have the fs library, which means the binaries in ONNX Runtime's official packages always static link to the fs library. It is just a matter of which version of the library, an experimental one or a more mature one. And it is an implementation detail that is not visible from outside. Anyway, a newer GCC is better. It will give us the chance to use many C++20 features. #### Why we were using GCC 8? It is because all our Linux packages were built on RHEL8 or its equivalents. The default GCC version in RHEL8 is 8. RHEL also provides additional GCC versions from RH devtoolset. UBI8 is the abbreviation of Red Hat Universal Base Image 8, which is the containerized RHEL8. UBI8 is free, which means it doesn't require a subscription(while RHEL does). The only devtoolset that UBI8 provides is GCC 12, which is too new for being used with CUDA 11.8. And our CUDA 11.8's build env is a docker image from Nvidia that is based on UBI8. #### How the problem is solved Almalinux is an alternative to RHEL. Almalinux 8 provides GCC 11. And the CUDA 11.8 docker image from Nvidia is open source, which means we can rebuild the image based on Almalinux 8 to get GCC 11. I've done this, but I cannot republish the new image due to various complicated license restrictions. Therefore I put them at an internal location in onnxruntimebuildcache.azurecr.io.
baijumeswani
pushed a commit
to microsoft/onnxruntime
that referenced
this issue
Jun 20, 2024
Use a common set of prebuilt manylinux base images to build the packages, to avoid building the manylinux part again and again. The base images can be used in GenAI and other projects too. This PR also updates the GCC version for inference python CUDA11/CUDA12 builds from 8 to 11. Later on I will update all other CUDA pipelines to use GCC 11, to avoid the issue described in onnx/onnx#6047 and microsoft/onnxruntime-genai#257 . To extract the common part as a reusable build infra among different ONNX Runtime projects.
baijumeswani
pushed a commit
to microsoft/onnxruntime
that referenced
this issue
Jun 20, 2024
…esystem implementation (#20893) This PR upgrades CUDA 11 build pipelines' GCC version from 8 to 11. GCC8 has an experimental std::filesystem implementation which is not ABI compatible with the formal one in later GCC releases. It didn't cause trouble for us, however, ONNX community has encountered this issue much. For example, onnx/onnx#6047 . So this PR increases the minimum supported GCC version from 8 to 9, and removes the references to GCC's "stdc++fs" library. Please note we compile our code on RHEL8 and RHEL8's libstdc++ doesn't have the fs library, which means the binaries in ONNX Runtime's official packages always static link to the fs library. It is just a matter of which version of the library, an experimental one or a more mature one. And it is an implementation detail that is not visible from outside. Anyway, a newer GCC is better. It will give us the chance to use many C++20 features. It is because all our Linux packages were built on RHEL8 or its equivalents. The default GCC version in RHEL8 is 8. RHEL also provides additional GCC versions from RH devtoolset. UBI8 is the abbreviation of Red Hat Universal Base Image 8, which is the containerized RHEL8. UBI8 is free, which means it doesn't require a subscription(while RHEL does). The only devtoolset that UBI8 provides is GCC 12, which is too new for being used with CUDA 11.8. And our CUDA 11.8's build env is a docker image from Nvidia that is based on UBI8. Almalinux is an alternative to RHEL. Almalinux 8 provides GCC 11. And the CUDA 11.8 docker image from Nvidia is open source, which means we can rebuild the image based on Almalinux 8 to get GCC 11. I've done this, but I cannot republish the new image due to various complicated license restrictions. Therefore I put them at an internal location in onnxruntimebuildcache.azurecr.io.
andife
pushed a commit
to andife/onnx
that referenced
this issue
Jul 20, 2024
### Description Possible fix for onnx#6047 based on https://stackoverflow.com/questions/33149878/experimentalfilesystem-linker-error/33159746#33159746 ### Motivation and Context I'm not a cmake expert so there may be a better way to do this. This did allow me to build ONNX 1.16 on RHEL 8. --------- Signed-off-by: Charles Volzka <cjvolzka@us.ibm.com> Signed-off-by: Charles Volzka <42243335+cjvolzka@users.noreply.github.com> Co-authored-by: Thiago Crepaldi <thiago.crepaldi@microsoft.com> Co-authored-by: G. Ramalingam <grama@microsoft.com> Signed-off-by: Andreas Fehlner <fehlner@arcor.de>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Bug Report
Is the issue related to model conversion?
No
Describe the bug
importing ONNX, built from source, on RHEL8 fails with
ImportError: /usr/local/lib64/python3.9/site-packages/onnx/onnx_cpp2py_export.cpython-39-x86_64-linux-gnu.so: undefined symbol: _ZNSt10filesystem7__cxx114path14_M_split_cmptsEv
System information
Reproduction instructions
pip3 install onnx
as that installs pre-built wheel files. To replicate on x86 RHEL 8, usepip3 install --no-binary onnx onnx
Expected behavior
The text was updated successfully, but these errors were encountered: