Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add miopen install rpm script #26

Merged
merged 1 commit into from
Mar 10, 2023
Merged

Add miopen install rpm script #26

merged 1 commit into from
Mar 10, 2023

Conversation

jataylo
Copy link

@jataylo jataylo commented Mar 9, 2023

Companion PR to https://github.com/ROCmSoftwarePlatform/rocAutomation/pull/114

Adds new common/install_rocm_miopen_rpm.sh file which will install a given MIOpen rpm package from a given source.

export MIOPEN_RPM_SOURCE=https://repo.radeon.com/rocm/yum/5.4/main/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm

build_wheel.sh
In build_wheel.sh we have deployed an approach related to the MIOpen install approach from upstream https://github.com/pytorch/builder/blob/main/common/install_miopen.sh#L134

  1. wget the ${MIOPEN_RPM_SOURCE} and throw an error if the download fails
+ docker exec -t manylinux-rocm-py3.8 bash -c 'source /env && bash /builder/common/install_rocm_miopen_rpm.sh'
Running on Linux 89640942d125 5.15.0-58-generic #64-Ubuntu SMP Thu Jan 5 11:43:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux at Wed Mar  8 20:15:25 UTC 2023
+ EXTRACTDIR_ROOT=/extract_miopen_rpm
+ mkdir -p /extract_miopen_rpm
+ echo 'Creating temporary directory for rpm download...'
Creating temporary directory for rpm download...
+ wget -P /extract_miopen_rpm https://repo.radeon.com/rocm/yum/5.4/main/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
--2023-03-08 20:15:25--  https://repo.radeon.com/rocm/yum/5.4/main/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
Resolving repo.radeon.com (repo.radeon.com)... 13.82.220.49
Connecting to repo.radeon.com (repo.radeon.com)|13.82.220.49|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 83647440 (80M) [application/x-redhat-package-manager]
Saving to: ‘/extract_miopen_rpm/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm’


 0% [                                       ] 0           --.-K/s              
 0% [                                       ] 188,133      823KB/s             
 1% [                                       ] 1,416,933   3.02MB/s             
 3% [>                                      ] 3,317,477   4.72MB/s             
 6% [=>                                     ] 5,218,021   5.57MB/s             
 8% [==>                                    ] 7,118,565   6.08MB/s             
10% [===>                                   ] 9,035,493   6.43MB/s             
13% [====>                                  ] 10,952,421  6.62MB/s             
15% [=====>                                 ] 12,869,349  6.81MB/s             
17% [=====>                                 ] 14,802,661  6.97MB/s             
20% [======>                                ] 16,735,973  7.09MB/s             
22% [=======>                               ] 18,685,669  7.20MB/s             
24% [========>                              ] 20,618,981  7.29MB/s             
27% [=========>                             ] 22,585,061  7.37MB/s             
29% [==========>                            ] 24,534,757  7.43MB/s  eta 8s     
31% [===========>                           ] 26,500,837  8.07MB/s  eta 8s     
34% [============>                          ] 28,466,917  8.11MB/s  eta 8s     
36% [=============>                         ] 30,449,381  8.12MB/s  eta 8s     
38% [==============>                        ] 32,431,845  8.23MB/s  eta 8s     
41% [===============>                       ] 34,430,693  8.23MB/s  eta 6s     
43% [===============>                       ] 36,413,157  8.31MB/s  eta 6s     
45% [================>                      ] 37,953,253  8.34MB/s  eta 6s     
47% [=================>                     ] 39,821,029  8.35MB/s  eta 6s     
50% [==================>                    ] 41,852,645  8.44MB/s  eta 6s     
52% [===================>                   ] 43,884,261  8.51MB/s  eta 5s     
54% [====================>                  ] 45,424,357  8.44MB/s  eta 5s     
56% [=====================>                 ] 47,455,973  8.50MB/s  eta 5s     
59% [======================>                ] 49,602,277  8.67MB/s  eta 5s     
61% [======================>                ] 51,191,525  8.60MB/s  eta 5s     
63% [=======================>               ] 53,354,213  8.70MB/s  eta 4s     
66% [========================>              ] 55,566,053  8.86MB/s  eta 4s     
69% [=========================>             ] 57,794,277  8.99MB/s  eta 4s     
71% [==========================>            ] 59,514,597  8.93MB/s  eta 4s     
73% [===========================>           ] 61,873,893  9.12MB/s  eta 4s     
76% [============================>          ] 64,265,957  9.26MB/s  eta 2s     
79% [=============================>         ] 66,428,645  9.36MB/s  eta 2s     
81% [==============================>        ] 68,525,797  9.55MB/s  eta 2s     
84% [================================>      ] 71,081,701  9.75MB/s  eta 2s     
88% [=================================>     ] 73,719,525  9.94MB/s  eta 2s     
91% [==================================>    ] 76,185,317  10.2MB/s  eta 1s     
93% [===================================>   ] 78,405,349  10.2MB/s  eta 1s     
97% [====================================>  ] 81,239,781  10.5MB/s  eta 1s     
100%[======================================>] 83,647,440  10.7MB/s   in 8.9s   

2023-03-08 20:15:35 (8.92 MB/s) - ‘/extract_miopen_rpm/miopen-hip-2.19.0.50400-72.el7.x86_64.rpm’ saved [83647440/83647440]

+ echo 'MIOpen package download complete...'
MIOpen package download complete...
  1. Extract the rpm package with rpm2cpio
+ cd /extract_miopen_rpm
++ ls miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
+ miopen_rpm=miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
+ rpm2cpio miopen-hip-2.19.0.50400-72.el7.x86_64.rpm
+ cpio -idmv
./opt
./opt/rocm-5.4.0
./opt/rocm-5.4.0/bin
./opt/rocm-5.4.0/bin/MIOpenDriver
./opt/rocm-5.4.0/bin/install_precompiled_kernels.sh
./opt/rocm-5.4.0/lib
./opt/rocm-5.4.0/lib/libMIOpen.so.1
./opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
./opt/rocm-5.4.0/share
./opt/rocm-5.4.0/share/doc
./opt/rocm-5.4.0/share/doc/miopen-hip
./opt/rocm-5.4.0/share/doc/miopen-hip/LICENSE.txt
./opt/rocm-5.4.0/share/miopen
./opt/rocm-5.4.0/share/miopen/db
./opt/rocm-5.4.0/share/miopen/db/gfx1030_36.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx1030_36.db
./opt/rocm-5.4.0/share/miopen/db/gfx803_36.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx803_36.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx803_64.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx803_64.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_56.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_56.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_56.db
./opt/rocm-5.4.0/share/miopen/db/gfx900_64.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx900_64.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_60.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_60.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_60.db
./opt/rocm-5.4.0/share/miopen/db/gfx906_64.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx906_64.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90878.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90878.OpenCL.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90878.db
./opt/rocm-5.4.0/share/miopen/db/gfx90a68.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90a68.db
./opt/rocm-5.4.0/share/miopen/db/gfx90a6e.HIP.fdb.txt
./opt/rocm-5.4.0/share/miopen/db/gfx90a6e.db
1894145 blocks
  1. Move the extracted libMIOpen.so.1.0* file to the target libMIOpen.so.1.0* location
++ ls opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
+ source_file=opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
++ ls /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ dest_file=/opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ '[' -e opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400 ']'
+ '[' -e /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404 ']'
+ echo 'Source .so: opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400'
Source .so: opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400
+ echo 'Dest .so: /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404'
Dest .so: /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ cp opt/rocm-5.4.0/lib/libMIOpen.so.1.0.50400 /opt/rocm-5.4.4/lib/libMIOpen.so.1.0.50404
+ echo 'libMIOpen so file from RPM copied to existing MIOpen install...'
libMIOpen so file from RPM copied to existing MIOpen install...
  1. Removes the temporary directory
+ rm -rf /extract_miopen_rpm
+ echo 'Removed temporary directory..

Example CI log:
Build MIOpen RPM source test

@jithunnair-amd jithunnair-amd merged commit 82ccc1c into ROCm:main Mar 10, 2023
jithunnair-amd
jithunnair-amd reviewed Mar 22, 2023
View reviewed changes
common/install_rocm_miopen_rpm.sh
rpm2cpio ${miopen_rpm} | cpio -idmv

# Copy libMIOpen.so.1 over existing
source_file=$(ls opt/rocm-*/lib/libMIOpen.so.1.0*)
Copy link

@jithunnair-amd jithunnair-amd Mar 22, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May not be an immediate issue, but the hardcoding of the MIOpen .so version bugs me :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jithunnair-amd jithunnair-amd jithunnair-amd approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@jataylo @jithunnair-amd