Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Solver generic_search fail: ConvHipImplicitGemmBwdDataV1R1Xdlops and ConvHipImplicitGemmForwardV4R4Xdlops #427

Closed
alexandraBara opened this issue Sep 11, 2020 · 7 comments
Closed

Solver generic_search fail: ConvHipImplicitGemmBwdDataV1R1Xdlops and ConvHipImplicitGemmForwardV4R4Xdlops #427

alexandraBara opened this issue Sep 11, 2020 · 7 comments
Assignees
@atamazov @daniellowell @asroy @jane-zxy
Labels
bug
Projects
hip-igemm BUG tracker

Comments

@alexandraBara
Copy link
Contributor

After this issue #411 was solved i pulled in the changes and ran tuning again.
Ran into another issue that seems to be generic_search failing for 2 solvers. I will attach logs bellow
@alexandraBara
Copy link
Contributor Author

alexandraBara commented Sep 11, 2020 •
edited by atamazov

ConvHipImplicitGemmBwdDataV1R1Xdlops 
INFO - Executing: sudo docker run --device='/dev/kfd' --device='/dev/dri' -w /home/miopenpdb -v /home/miopenpdb:/home/miopenpdb --user=root --group-add video --privileged=true --rm miopentuna bash  -c "export MIOPEN_LOG_LEVEL=7 && export MIOPEN_FIND_ENFORCE=3 && export HIP_VISIBLE_DEVICES=7 && MIOpenDriver conv -V 0 -i 1 --forw 2 --pad_h 2 --out_channels 128 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 7 --conv_stride_w 1 --group_count 1 --in_channels 32 --in_w 7 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0 2>&1 "
INFO - Setting job id 11099079 state to running
INFO - MIOpenDriver conv -V 0 -i 1 --forw 2 --pad_h 2 --out_channels 128 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 7 --conv_stride_w 1 --group_count 1 --in_channels 32 --in_w 7 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0
INFO - MIOpen(HIP): Info [Handle] stream: 0x3a98f70, device_id: 0
INFO - MIOpen(HIP): Info [BackwardDataGetWorkSpaceSize]
INFO - MIOpen(HIP): Info2 [HipCompilerVersionImpl] Read version information from HIP package...
INFO - MIOpen(HIP): Info [HipCompilerVersionImpl] 3.6.20263
INFO - MIOpen(HIP): Info [AmdRocmMetadataVersionDetect] ROCm MD version AMDHSA_COv3, MIOpen version 2.7.0.8186-ab68183b
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Running: '/opt/rocm/llvm/bin/clang --version'
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] clang version 11.0.0 (/data/jenkins_workspace/compute-rocm-rel-3.6/external/llvm-project/clang f7b7e21a21d08df6971d2c77315a0e41b7639334)
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Target: x86_64-unknown-linux-gnu
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Thread model: posix
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] InstalledDir: /opt/rocm/llvm/bin
INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl]
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file /opt/rocm/miopen/share/miopen/db/miopen.db
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(config);
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(perf_db);
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.config/miopen/miopen_1.0.0.udb
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'config');
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'perf_db');
INFO - MIOpen(HIP): Trace [SQLitePerfDb] Database created successfully
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(config);
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(perf_db);
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf3x2 (not searchable)
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=128 H=7 W=7 K=32 n_groups=120 flags=7 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7
INFO - MIOpen(HIP): Info2 [GetSolution] ...flags=519 d_N_stride=25088 d_C_stride=196 f_K_stride=100 f_C_stride=3200 o_N_stride=6272 o_K_stride=196
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3 (db access disabled)
INFO - MIOpen(HIP): Info [GetPerformanceConfig] 120
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE enforced to NORMAL(1) due to MIOPEN_FIND_ENFORCE
INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE = NORMAL(1)
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm3x3U: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1U: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1UV2: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2f1: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2b1: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd11x11: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwdGen: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd3x3: Not applicable
INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd1x1: Not applicable
INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,8,8,1,1,8,2,1
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmForwardV4R4Xdlops: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenXdlopsFwdFp32: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenFwdXdlops: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmBwdDataV1R1Xdlops (db access disabled)
INFO - MIOpen(HIP): Info [EuristicInit] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Info [GetPerformanceConfigBase] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmBwdDataV1R1Xdlops: Success.
INFO - MIOpen(HIP): Info2 [BackwardDataGetWorkSpaceSize] 27697152
INFO - MIOpen(HIP): Info [FindConvBwdDataAlgorithm] requestAlgoCount = 2, workspace = 27697152
INFO - MIOpen(HIP): Info2 [FindRecordUnsafe] Looking for key 128-7-7-5x5-32-7-7-128-2x2-1x1-1x1-0-NCHW-FP32-B in file /home/miopenpdb/.config/miopen/gfx90878.HIP.2_7_0_8186-ab68183b.ufdb.txt
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.290088 ms
INFO - MIOpen(HIP): Info [TryLoad] Find-db regenerating.
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf3x2 (not searchable)
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=128 H=7 W=7 K=32 n_groups=120 flags=7 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7
INFO - MIOpen(HIP): Info2 [GetSolution] ...flags=519 d_N_stride=25088 d_C_stride=196 f_K_stride=100 f_C_stride=3200 o_N_stride=6272 o_K_stride=196
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info [GetValues] =ConvBinWinogradRxSf2x3:<values not found>
INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 138.718 ms
INFO - MIOpen(HIP): Info [FindSolutionImpl] Perf Db: record not found for: ConvBinWinogradRxSf2x3
INFO - MIOpen(HIP): Info [FindSolutionImpl] Starting search: ConvBinWinogradRxSf2x3, enforce: SEARCH(3), ALL(1)
INFO - MIOpen(HIP): Info [GetPerformanceConfig] 120
INFO - MIOpen(HIP): Warning [GenericSearch] ConvBinWinogradRxSf2x3: Searching the best solution among 120...
INFO - MIOpen(HIP): Info2 [GenericSearch] #0/0/120 1
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file
INFO - MIOpen(HIP): Info [KernDb] database not present
INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.cache/2.7.0.8186-ab68183b/gfx90878.ukdb
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:CREATE TABLE IF NOT EXISTS `kern_db` (`id` INTEGER PRIMARY KEY ASC,`kernel_name` TEXT NOT NULL,`kernel_args` TEXT NOT NULL,`kernel_blob` BLOB NOT NULL,`kernel_hash` TEXT NOT NULL,`uncompressed_size` INT NOT NULL);CREATE UNIQUE INDEX IF NOT EXISTS `idx_kern_db` ON kern_db(kernel_name, kernel_args, kernel_hash, uncompressed_size);
INFO - MIOpen(HIP): Info2 [KernDb] Database created successfully
INFO - MIOpen(HIP): Trace [Exec] 140138148409728:PRAGMA table_info(kern_db);
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args:  -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s.o') AND (kernel_args = ' -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 1.48071 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args:  -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  0/0/120 elapsed_time: 6.81911, best_time: 3.40282e+38, 1
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 6.81911 / 3.40282e+38 = 2.00396e-38
INFO - MIOpen(HIP): Info [GenericSearch] #0/0/120 6.81316 < 3.40282e+38 1
INFO - MIOpen(HIP): Info2 [GenericSearch] #1/0/120 2
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  1/0/120 elapsed_time: 3.39276, best_time: 6.81316, 2
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 3.39276 / 6.81316 = 0.497971
INFO - MIOpen(HIP): Info [GenericSearch] #1/0/120 3.40079 < 6.81316 2
INFO - MIOpen(HIP): Info2 [GenericSearch] #2/0/120 3
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  2/0/120 elapsed_time: 2.34317, best_time: 3.40079, 3
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 2.34317 / 3.40079 = 0.689008
INFO - MIOpen(HIP): Info [GenericSearch] #2/0/120 2.33082 < 3.40079 3
INFO - MIOpen(HIP): Info2 [GenericSearch] #3/0/120 4
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  3/0/120 elapsed_time: 1.70862, best_time: 2.33082, 4
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.70862 / 2.33082 = 0.733055
INFO - MIOpen(HIP): Info [GenericSearch] #3/0/120 1.69924 < 2.33082 4
INFO - MIOpen(HIP): Info2 [GenericSearch] #4/0/120 5
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  4/0/120 elapsed_time: 1.38638, best_time: 1.69924, 5
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.38638 / 1.69924 = 0.815883
INFO - MIOpen(HIP): Info [GenericSearch] #4/0/120 1.38174 < 1.69924 5
INFO - MIOpen(HIP): Info2 [GenericSearch] #5/0/120 6
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  5/0/120 elapsed_time: 1.17054, best_time: 1.38174, 6
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.17054 / 1.38174 = 0.847151
INFO - MIOpen(HIP): Info [GenericSearch] #5/0/120 1.16971 < 1.38174 6
INFO - MIOpen(HIP): Info2 [GenericSearch] #6/0/120 7
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  6/0/120 elapsed_time: 1.06495, best_time: 1.16971, 7
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 1.06495 / 1.16971 = 0.910434
INFO - MIOpen(HIP): Info [GenericSearch] #6/0/120 1.05541 < 1.16971 7
INFO - MIOpen(HIP): Info2 [GenericSearch] #7/0/120 8
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  7/0/120 elapsed_time: 0.84447, best_time: 1.05541, 8
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.84447 / 1.05541 = 0.800134
INFO - MIOpen(HIP): Info [GenericSearch] #7/0/120 0.843574 < 1.05541 8
INFO - MIOpen(HIP): Info2 [GenericSearch] #8/0/120 9
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  8/0/120 elapsed_time: 0.84287, best_time: 0.843574, 9
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.84287 / 0.843574 = 0.999166
INFO - MIOpen(HIP): Info [GenericSearch] #8/0/120 0.84351 < 0.843574 9
INFO - MIOpen(HIP): Info2 [GenericSearch] #9/0/120 10
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  9/0/120 elapsed_time: 0.740151, best_time: 0.84351, 10
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.740151 / 0.84351 = 0.877466
INFO - MIOpen(HIP): Info [GenericSearch] #9/0/120 0.739575 < 0.84351 10
INFO - MIOpen(HIP): Info2 [GenericSearch] #10/0/120 11
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  10/0/120 elapsed_time: 0.635832, best_time: 0.739575, 11
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.635832 / 0.739575 = 0.859726
INFO - MIOpen(HIP): Info [GenericSearch] #10/0/120 0.63516 < 0.739575 11
INFO - MIOpen(HIP): Info2 [GenericSearch] #11/0/120 12
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  11/0/120 elapsed_time: 0.634712, best_time: 0.63516, 12
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.634712 / 0.63516 = 0.999295
INFO - MIOpen(HIP): Info [GenericSearch] #11/0/120 0.634904 < 0.63516 12
INFO - MIOpen(HIP): Info2 [GenericSearch] #12/0/120 13
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  12/0/120 elapsed_time: 0.531194, best_time: 0.634904, 13
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.531194 / 0.634904 = 0.836652
INFO - MIOpen(HIP): Info [GenericSearch] #12/0/120 0.53097 < 0.634904 13
INFO - MIOpen(HIP): Info2 [GenericSearch] #13/0/120 14
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  13/0/120 elapsed_time: 0.531034, best_time: 0.53097, 14
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.531034 / 0.53097 = 1.00012
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.531066 >= 0.53097
INFO - MIOpen(HIP): Info2 [GenericSearch] #14/0/120 15
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  14/0/120 elapsed_time: 0.530714, best_time: 0.53097, 15
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.530714 / 0.53097 = 0.999518
INFO - MIOpen(HIP): Info [GenericSearch] #14/0/120 0.53065 < 0.53097 15
INFO - MIOpen(HIP): Info2 [GenericSearch] #15/0/120 16
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  15/0/120 elapsed_time: 0.425915, best_time: 0.53065, 16
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.425915 / 0.53065 = 0.802629
INFO - MIOpen(HIP): Info [GenericSearch] #15/0/120 0.425851 < 0.53065 16
INFO - MIOpen(HIP): Info2 [GenericSearch] #16/0/120 17
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  16/0/120 elapsed_time: 0.426395, best_time: 0.425851, 17
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.426395 / 0.425851 = 1.00128
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.427291 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #17/0/120 18
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  17/0/120 elapsed_time: 0.426235, best_time: 0.425851, 18
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.426235 / 0.425851 = 1.0009
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.426971 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #18/0/120 19
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  18/0/120 elapsed_time: 0.427355, best_time: 0.425851, 19
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.427355 / 0.425851 = 1.00353
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.427355 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #19/0/120 20
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  19/0/120 elapsed_time: 0.427195, best_time: 0.425851, 20
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.427195 / 0.425851 = 1.00316
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.427355 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #20/0/120 21
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  20/0/120 elapsed_time: 0.429435, best_time: 0.425851, 21
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.429435 / 0.425851 = 1.00842
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.429531 >= 0.425851
INFO - MIOpen(HIP): Info2 [GenericSearch] #21/0/120 22
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  21/0/120 elapsed_time: 0.323516, best_time: 0.425851, 22
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.323516 / 0.425851 = 0.759693
INFO - MIOpen(HIP): Info [GenericSearch] #21/0/120 0.32358 < 0.425851 22
INFO - MIOpen(HIP): Info2 [GenericSearch] #22/0/120 23
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  22/0/120 elapsed_time: 0.322876, best_time: 0.32358, 23
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.322876 / 0.32358 = 0.997824
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.32358 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #23/0/120 24
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  23/0/120 elapsed_time: 0.323996, best_time: 0.32358, 24
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.323996 / 0.32358 = 1.00129
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324348 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #24/0/120 25
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  24/0/120 elapsed_time: 0.322556, best_time: 0.32358, 25
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.322556 / 0.32358 = 0.996835
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324348 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #25/0/120 26
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  25/0/120 elapsed_time: 0.323996, best_time: 0.32358, 26
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.323996 / 0.32358 = 1.00129
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324476 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #26/0/120 27
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  26/0/120 elapsed_time: 0.325116, best_time: 0.32358, 27
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.325116 / 0.32358 = 1.00475
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.324732 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #27/0/120 28
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  27/0/120 elapsed_time: 0.327196, best_time: 0.32358, 28
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.327196 / 0.32358 = 1.01118
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.325244 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #28/0/120 29
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  28/0/120 elapsed_time: 0.326236, best_time: 0.32358, 29
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.326236 / 0.32358 = 1.00821
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.3255 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #29/0/120 30
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  29/0/120 elapsed_time: 0.326236, best_time: 0.32358, 30
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.326236 / 0.32358 = 1.00821
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.32582 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #30/0/120 31
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  30/0/120 elapsed_time: 0.324796, best_time: 0.32358, 31
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.324796 / 0.32358 = 1.00376
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.326492 >= 0.32358
INFO - MIOpen(HIP): Info2 [GenericSearch] #31/0/120 32
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  31/0/120 elapsed_time: 0.220478, best_time: 0.32358, 32
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.220478 / 0.32358 = 0.681371
INFO - MIOpen(HIP): Info [GenericSearch] #31/0/120 0.22067 < 0.32358 32
INFO - MIOpen(HIP): Info2 [GenericSearch] #32/0/120 33
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  32/0/120 elapsed_time: 0.223198, best_time: 0.22067, 33
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223198 / 0.22067 = 1.01146
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.221982 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #33/0/120 34
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  33/0/120 elapsed_time: 0.219998, best_time: 0.22067, 34
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.219998 / 0.22067 = 0.996956
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.221278 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #34/0/120 35
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  34/0/120 elapsed_time: 0.223838, best_time: 0.22067, 35
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223838 / 0.22067 = 1.01436
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22243 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #35/0/120 36
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  35/0/120 elapsed_time: 0.221278, best_time: 0.22067, 36
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.221278 / 0.22067 = 1.00276
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22179 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #36/0/120 37
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  36/0/120 elapsed_time: 0.223838, best_time: 0.22067, 37
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223838 / 0.22067 = 1.01436
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222718 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #37/0/120 38
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  37/0/120 elapsed_time: 0.221598, best_time: 0.22067, 38
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.221598 / 0.22067 = 1.00421
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222206 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #38/0/120 39
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  38/0/120 elapsed_time: 0.224158, best_time: 0.22067, 39
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224158 / 0.22067 = 1.01581
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22291 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #39/0/120 40
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  39/0/120 elapsed_time: 0.220798, best_time: 0.22067, 40
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.220798 / 0.22067 = 1.00058
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222974 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #40/0/120 41
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  40/0/120 elapsed_time: 0.223518, best_time: 0.22067, 41
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223518 / 0.22067 = 1.01291
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222782 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #41/0/120 42
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  41/0/120 elapsed_time: 0.223678, best_time: 0.22067, 42
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223678 / 0.22067 = 1.01363
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222846 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #42/0/120 43
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  42/0/120 elapsed_time: 0.223518, best_time: 0.22067, 43
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223518 / 0.22067 = 1.01291
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223102 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #43/0/120 44
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  43/0/120 elapsed_time: 0.220958, best_time: 0.22067, 44
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.220958 / 0.22067 = 1.00131
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.222942 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #44/0/120 45
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  44/0/120 elapsed_time: 0.221918, best_time: 0.22067, 45
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.221918 / 0.22067 = 1.00566
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22323 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #45/0/120 46
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  45/0/120 elapsed_time: 0.223358, best_time: 0.22067, 46
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223358 / 0.22067 = 1.01218
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223166 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #46/0/120 47
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  46/0/120 elapsed_time: 0.223998, best_time: 0.22067, 47
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223998 / 0.22067 = 1.01508
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22339 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #47/0/120 48
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  47/0/120 elapsed_time: 0.224158, best_time: 0.22067, 48
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224158 / 0.22067 = 1.01581
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223518 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #48/0/120 49
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  48/0/120 elapsed_time: 0.224478, best_time: 0.22067, 49
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224478 / 0.22067 = 1.01726
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223934 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #49/0/120 50
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  49/0/120 elapsed_time: 0.224798, best_time: 0.22067, 50
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224798 / 0.22067 = 1.01871
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223838 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #50/0/120 51
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  50/0/120 elapsed_time: 0.223678, best_time: 0.22067, 51
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223678 / 0.22067 = 1.01363
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223582 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #51/0/120 52
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  51/0/120 elapsed_time: 0.222078, best_time: 0.22067, 52
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.222078 / 0.22067 = 1.00638
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223966 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #52/0/120 53
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  52/0/120 elapsed_time: 0.223678, best_time: 0.22067, 53
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.223678 / 0.22067 = 1.01363
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.223678 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #53/0/120 54
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  53/0/120 elapsed_time: 0.224798, best_time: 0.22067, 54
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224798 / 0.22067 = 1.01871
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.224414 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #54/0/120 55
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  54/0/120 elapsed_time: 0.224958, best_time: 0.22067, 55
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.224958 / 0.22067 = 1.01943
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.224894 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #55/0/120 56
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  55/0/120 elapsed_time: 0.226558, best_time: 0.22067, 56
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226558 / 0.22067 = 1.02668
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.225438 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #56/0/120 57
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  56/0/120 elapsed_time: 0.225118, best_time: 0.22067, 57
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.225118 / 0.22067 = 1.02016
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.225342 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #57/0/120 58
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  57/0/120 elapsed_time: 0.225918, best_time: 0.22067, 58
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.225918 / 0.22067 = 1.02378
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22643 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #58/0/120 59
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  58/0/120 elapsed_time: 0.226238, best_time: 0.22067, 59
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226238 / 0.22067 = 1.02523
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.226814 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #59/0/120 60
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  59/0/120 elapsed_time: 0.226878, best_time: 0.22067, 60
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226878 / 0.22067 = 1.02813
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.22691 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #60/0/120 61
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  60/0/120 elapsed_time: 0.229438, best_time: 0.22067, 61
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.229438 / 0.22067 = 1.03973
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.228222 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #61/0/120 62
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  61/0/120 elapsed_time: 0.226878, best_time: 0.22067, 62
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.226878 / 0.22067 = 1.02813
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.229534 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #62/0/120 63
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  62/0/120 elapsed_time: 0.230558, best_time: 0.22067, 63
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.230558 / 0.22067 = 1.04481
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.230558 >= 0.22067
INFO - MIOpen(HIP): Info2 [GenericSearch] #63/0/120 64
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  63/0/120 elapsed_time: 0.122239, best_time: 0.22067, 64
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.122239 / 0.22067 = 0.553945
INFO - MIOpen(HIP): Info [GenericSearch] #63/0/120 0.122303 < 0.22067 64
INFO - MIOpen(HIP): Info2 [GenericSearch] #64/0/120 65
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  64/0/120 elapsed_time: 0.124479, best_time: 0.122303, 65
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.124479 / 0.122303 = 1.01779
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125151 >= 0.122303
INFO - MIOpen(HIP): Info2 [GenericSearch] #65/0/120 66
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  65/0/120 elapsed_time: 0.122239, best_time: 0.122303, 66
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.122239 / 0.122303 = 0.999477
INFO - MIOpen(HIP): Info [GenericSearch] #65/0/120 0.121887 < 0.122303 66
INFO - MIOpen(HIP): Info2 [GenericSearch] #66/0/120 67
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  66/0/120 elapsed_time: 0.123199, best_time: 0.121887, 67
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123199 / 0.121887 = 1.01076
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.122623 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #67/0/120 68
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  67/0/120 elapsed_time: 0.122719, best_time: 0.121887, 68
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.122719 / 0.121887 = 1.00683
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.122463 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #68/0/120 69
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  68/0/120 elapsed_time: 0.123039, best_time: 0.121887, 69
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123039 / 0.121887 = 1.00945
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.123039 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #69/0/120 70
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  69/0/120 elapsed_time: 0.123039, best_time: 0.121887, 70
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123039 / 0.121887 = 1.00945
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.123263 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #70/0/120 71
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  70/0/120 elapsed_time: 0.123999, best_time: 0.121887, 71
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.123999 / 0.121887 = 1.01733
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.124319 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #71/0/120 72
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  71/0/120 elapsed_time: 0.125599, best_time: 0.121887, 72
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125599 / 0.121887 = 1.03045
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.124415 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #72/0/120 73
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  72/0/120 elapsed_time: 0.126239, best_time: 0.121887, 73
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.126239 / 0.121887 = 1.03571
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125919 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #73/0/120 74
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  73/0/120 elapsed_time: 0.124639, best_time: 0.121887, 74
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.124639 / 0.121887 = 1.02258
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125535 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #74/0/120 75
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  74/0/120 elapsed_time: 0.125919, best_time: 0.121887, 75
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125919 / 0.121887 = 1.03308
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.126495 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #75/0/120 76
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  75/0/120 elapsed_time: 0.125919, best_time: 0.121887, 76
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125919 / 0.121887 = 1.03308
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.125823 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #76/0/120 77
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  76/0/120 elapsed_time: 0.125919, best_time: 0.121887, 77
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.125919 / 0.121887 = 1.03308
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.126463 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #77/0/120 78
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  77/0/120 elapsed_time: 0.126559, best_time: 0.121887, 78
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.126559 / 0.121887 = 1.03833
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.127487 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #78/0/120 79
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  78/0/120 elapsed_time: 0.127359, best_time: 0.121887, 79
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.127359 / 0.121887 = 1.04489
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.127455 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #79/0/120 80
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  79/0/120 elapsed_time: 0.129759, best_time: 0.121887, 80
INFO - MIOpen(HIP): Info2 [GenericSearch] #80/0/120 81
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  80/0/120 elapsed_time: 0.129439, best_time: 0.121887, 81
INFO - MIOpen(HIP): Info2 [GenericSearch] #81/0/120 82
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  81/0/120 elapsed_time: 0.130399, best_time: 0.121887, 82
INFO - MIOpen(HIP): Info2 [GenericSearch] #82/0/120 83
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  82/0/120 elapsed_time: 0.127839, best_time: 0.121887, 83
INFO - MIOpen(HIP): Info2 [GenericSearch] Finding average for: 0.127839 / 0.121887 = 1.04883
INFO - MIOpen(HIP): Info2 [GenericSearch] Average is not better: 0.129343 >= 0.121887
INFO - MIOpen(HIP): Info2 [GenericSearch] #83/0/120 84
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  83/0/120 elapsed_time: 0.128799, best_time: 0.121887, 84
INFO - MIOpen(HIP): Info2 [GenericSearch] #84/0/120 85
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  84/0/120 elapsed_time: 0.129119, best_time: 0.121887, 85
INFO - MIOpen(HIP): Info2 [GenericSearch] #85/0/120 86
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  85/0/120 elapsed_time: 0.130399, best_time: 0.121887, 86
INFO - MIOpen(HIP): Info2 [GenericSearch] #86/0/120 87
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  86/0/120 elapsed_time: 0.129279, best_time: 0.121887, 87
INFO - MIOpen(HIP): Info2 [GenericSearch] #87/0/120 88
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  87/0/120 elapsed_time: 0.131039, best_time: 0.121887, 88
INFO - MIOpen(HIP): Info2 [GenericSearch] #88/0/120 89
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  88/0/120 elapsed_time: 0.131679, best_time: 0.121887, 89
INFO - MIOpen(HIP): Info2 [GenericSearch] #89/0/120 90
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  89/0/120 elapsed_time: 0.134879, best_time: 0.121887, 90
INFO - MIOpen(HIP): Info2 [GenericSearch] #90/0/120 91
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  90/0/120 elapsed_time: 0.134399, best_time: 0.121887, 91
INFO - MIOpen(HIP): Info2 [GenericSearch] #91/0/120 92
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  91/0/120 elapsed_time: 0.133919, best_time: 0.121887, 92
INFO - MIOpen(HIP): Info2 [GenericSearch] #92/0/120 93
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  92/0/120 elapsed_time: 0.133759, best_time: 0.121887, 93
INFO - MIOpen(HIP): Info2 [GenericSearch] #93/0/120 94
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  93/0/120 elapsed_time: 0.136959, best_time: 0.121887, 94
INFO - MIOpen(HIP): Info2 [GenericSearch] #94/0/120 95
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  94/0/120 elapsed_time: 0.137439, best_time: 0.121887, 95
INFO - MIOpen(HIP): Info2 [GenericSearch] #95/0/120 96
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  95/0/120 elapsed_time: 0.137759, best_time: 0.121887, 96
INFO - MIOpen(HIP): Info2 [GenericSearch] #96/0/120 97
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  96/0/120 elapsed_time: 0.136159, best_time: 0.121887, 97
INFO - MIOpen(HIP): Info2 [GenericSearch] #97/0/120 98
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  97/0/120 elapsed_time: 0.138559, best_time: 0.121887, 98
INFO - MIOpen(HIP): Info2 [GenericSearch] #98/0/120 99
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  98/0/120 elapsed_time: 0.138239, best_time: 0.121887, 99
INFO - MIOpen(HIP): Info2 [GenericSearch] #99/0/120 100
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  99/0/120 elapsed_time: 0.138239, best_time: 0.121887, 100
INFO - MIOpen(HIP): Info2 [GenericSearch] #100/0/120 101
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  100/0/120 elapsed_time: 0.138239, best_time: 0.121887, 101
INFO - MIOpen(HIP): Info2 [GenericSearch] #101/0/120 102
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  101/0/120 elapsed_time: 0.139679, best_time: 0.121887, 102
INFO - MIOpen(HIP): Info2 [GenericSearch] #102/0/120 103
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  102/0/120 elapsed_time: 0.139039, best_time: 0.121887, 103
INFO - MIOpen(HIP): Info2 [GenericSearch] #103/0/120 104
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  103/0/120 elapsed_time: 0.140639, best_time: 0.121887, 104
INFO - MIOpen(HIP): Info2 [GenericSearch] #104/0/120 105
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  104/0/120 elapsed_time: 0.140959, best_time: 0.121887, 105
INFO - MIOpen(HIP): Info2 [GenericSearch] #105/0/120 106
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  105/0/120 elapsed_time: 0.141279, best_time: 0.121887, 106
INFO - MIOpen(HIP): Info2 [GenericSearch] #106/0/120 107
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  106/0/120 elapsed_time: 0.142399, best_time: 0.121887, 107
INFO - MIOpen(HIP): Info2 [GenericSearch] #107/0/120 108
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  107/0/120 elapsed_time: 0.142079, best_time: 0.121887, 108
INFO - MIOpen(HIP): Info2 [GenericSearch] #108/0/120 109
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  108/0/120 elapsed_time: 0.140799, best_time: 0.121887, 109
INFO - MIOpen(HIP): Info2 [GenericSearch] #109/0/120 110
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  109/0/120 elapsed_time: 0.141919, best_time: 0.121887, 110
INFO - MIOpen(HIP): Info2 [GenericSearch] #110/0/120 111
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  110/0/120 elapsed_time: 0.142239, best_time: 0.121887, 111
INFO - MIOpen(HIP): Info2 [GenericSearch] #111/0/120 112
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  111/0/120 elapsed_time: 0.144959, best_time: 0.121887, 112
INFO - MIOpen(HIP): Info2 [GenericSearch] #112/0/120 113
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  112/0/120 elapsed_time: 0.141919, best_time: 0.121887, 113
INFO - MIOpen(HIP): Info2 [GenericSearch] #113/0/120 114
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  113/0/120 elapsed_time: 0.142079, best_time: 0.121887, 114
INFO - MIOpen(HIP): Info2 [GenericSearch] #114/0/120 115
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  114/0/120 elapsed_time: 0.141919, best_time: 0.121887, 115
INFO - MIOpen(HIP): Info2 [GenericSearch] #115/0/120 116
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  115/0/120 elapsed_time: 0.143359, best_time: 0.121887, 116
INFO - MIOpen(HIP): Info2 [GenericSearch] #116/0/120 117
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  116/0/120 elapsed_time: 0.144159, best_time: 0.121887, 117
INFO - MIOpen(HIP): Info2 [GenericSearch] #117/0/120 118
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  117/0/120 elapsed_time: 0.144799, best_time: 0.121887, 118
INFO - MIOpen(HIP): Info2 [GenericSearch] #118/0/120 119
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  118/0/120 elapsed_time: 0.145279, best_time: 0.121887, 119
INFO - MIOpen(HIP): Info2 [GenericSearch] #119/0/120 120
INFO - MIOpen(HIP): Trace [GenericSearch] ##(n_current, n_failed, n_runs_total):  119/0/120 elapsed_time: 0.145919, best_time: 0.121887, 120
INFO - MIOpen(HIP): Warning [GenericSearch] Done: 120/0/120, best #65 0.121887 66
INFO - MIOpen(HIP): Warning [GenericSearch] ...Score: 1.19848 (default time 0.146079)
INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR IGNORE INTO config( layout,data_type,direction,spatial_dim,in_channels,in_h,in_w,in_d,fil_h,fil_w,fil_d,out_channels,batchsize,pad_h,pad_w,pad_d,conv_stride_h,conv_stride_w,conv_stride_d,dilation_h,dilation_w,dilation_d,bias,group_count ) VALUES( ?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?);
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info2 [UpdateUnsafe] 1 rows updated
INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO perf_db(config, solver, params, arch, num_cu) VALUES((SELECT id FROM config WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) ) ) , ? , ? , ? , ?);
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1,ConvBinWinogradRxSf2x3,66,gfx908,120]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:66
INFO - MIOpen(HIP): Info2 [Measure] Db::Update time: 3.31869 ms
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v16_5_0_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: conv_3x3_wheel_alpha_v9_0_15.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'conv_3x3_wheel_alpha_v9_0_15.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v16_5_0_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.09883 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: conv_3x3_wheel_alpha_v9_0_15.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.53101 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v21_1_0_gfx9_fp32_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.8889 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v16_5_0_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConvRxSf3x2
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxSf3x2: miopenSp3AsmConvRxSf3x2: 0.444955 < 3.40282e+38
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConv_v21_1_0_gfx9_fp32_stride1
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 G=1 C=128 H=7 W=7 K=32 n_groups=66 flags=1543 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7 d_buf.byte_stride.nk=25088 d_buf.byte_stride.c=196 d_buf.byte_stride.h=28 d_buf.byte_stride.w=4 f_buf.byte_stride.nk=100 f_buf.byte_stride.c=3200 f_buf.byte_stride.h=20 f_buf.byte_stride.w=4 o_buf.byte_stride.nk=6272 o_buf.byte_stride.c=196 o_buf.byte_stride.h=28 o_buf.byte_stride.w=4 d_buf.byte_stride.g=25088 o_buf.byte_stride.g=6272 f_buf.byte_stride.g=409600
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp32_stride1: 0.146079 < 0.444955
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConvRxSU
INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=128 H=7 W=7 K=32 n_groups=120 flags=7 R=5 S=5 pad_H=2 pad_W=2 out_H=7 out_W=7
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxS: miopenSp3AsmConvRxSU: 0.184478 >= 0.146079
INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB and solver ConvBinWinogradRxSf2x3
INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvBinWinogradRxSf2x3 registered as find 1.0 best for miopenConvolutionBwdDataAlgoWinograd in 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB
INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp32_stride1: 0.146079, workspce_sz = 0
INFO - MIOpen(HIP): Info [SetValues] 128-7-7-5x5-32-7-7-128-2x2-1x1-1x1-0-NCHW-FP32-B, content inserted: miopenConvolutionBwdDataAlgoWinograd:ConvBinWinogradRxSf2x3,0.146079,0,miopenConvolutionBwdDataAlgoWinograd,<unused>
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm3x3U: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1U: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1UV2: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2f1: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2b1: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd11x11: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwdGen: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd3x3: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd1x1: Not applicable
INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,8,8,1,1,8,2,1
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvOclDirectFwd
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:66
INFO - MIOpen(HIP): Info [GetValues] =ConvOclDirectFwd:<values not found>
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info [GetValues] =ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 135.471 ms
INFO - MIOpen(HIP): Info2 [FindSolutionImpl] Perf Db: record loaded: ConvOclDirectFwd
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd: Success.
INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'MIOpenConvDirUni.cl.o') AND (kernel_args = ' -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908');
INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.077527 ms
INFO - MIOpen(HIP): Info2 [LoadBinary] Unable to load binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [SaveBinary] Saving binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=0 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=32 -DMLO_N_INPUTS=128 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=7 -DMLO_OUT_HEIGHT=7 -DMLO_OUT_BATCH_STRIDE=1568 -DMLO_OUT_CHANNEL_STRIDE=49 -DMLO_OUT_STRIDE=7 -DMLO_IN_WIDTH=7 -DMLO_IN_HEIGHT=7 -DMLO_IN_BATCH_STRIDE=6272 -DMLO_IN_CHANNEL_STRIDE=49 -DMLO_IN_STRIDE=7 -DMLO_IN_TILE0=8 -DMLO_IN_TILE1=8 -DMLO_GRP_TILE0=8 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=1 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=2 -DMLO_N_OUT_TILES_PERSTACK=2 -DMLO_N_IN_TILES_PERSTACK=1 -DMLO_N_READ_PROCS=64 -DMLO_ALU_VTILE0=8 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=0 -DMIOPEN_USE_FP32=1 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO kern_db(kernel_name, kernel_args, kernel_blob, kernel_hash, uncompressed_size) VALUES(?, ?, ?, ?, ?);
INFO - MIOpen(HIP): Info2 [Measure] Db::StoreRecord time: 6.39631 ms
INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: MIOpenConvUni
INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvOclDirectFwd: MIOpenConvUni: 0.388957 < 3.40282e+38
INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB and solver ConvOclDirectFwd
INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvOclDirectFwd registered as find 1.0 best for miopenConvolutionBwdDataAlgoDirect in 128x7x7x5x5x32x7x7x128xNCHWxFP32x2x2x1x1x1x1x1xB
INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvOclDirectFwd: MIOpenConvUni: 0.388957, workspce_sz = 0
INFO - MIOpen(HIP): Info [SetValues] 128-7-7-5x5-32-7-7-128-2x2-1x1-1x1-0-NCHW-FP32-B, content inserted: miopenConvolutionBwdDataAlgoDirect:ConvOclDirectFwd,0.388957,0,miopenConvolutionBwdDataAlgoDirect,<unused>
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmForwardV4R4Xdlops: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenXdlopsFwdFp32: Not applicable
INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmV4R4GenFwdXdlops: Not applicable
INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmBwdDataV1R1Xdlops
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:66
INFO - MIOpen(HIP): Info [GetValues] =ConvHipImplicitGemmBwdDataV1R1Xdlops:<values not found>
INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP32,B,2,128,7,7,1,5,5,1,32,128,2,2,0,1,1,0,1,1,0,0,1]
INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,8,8,8,1,1,2,1,2
INFO - MIOpen(HIP): Info [GetValues] =ConvHipImplicitGemmBwdDataV1R1Xdlops:<values not found>
INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 151.451 ms
INFO - MIOpen(HIP): Info [FindSolutionImpl] Perf Db: record not found for: ConvHipImplicitGemmBwdDataV1R1Xdlops
INFO - MIOpen(HIP): Info [FindSolutionImpl] Starting search: ConvHipImplicitGemmBwdDataV1R1Xdlops, enforce: SEARCH(3), ALL(1)
INFO - MIOpen(HIP): Info [EuristicInit] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Info [GetPerformanceConfigBase] 32,128,8,32,64,4,0,1
INFO - MIOpen(HIP): Warning [GenericSearch] ConvHipImplicitGemmBwdDataV1R1Xdlops: Searching the best solution among 0 (spare)...
INFO - MIOpen(HIP): Warning [GenericSearch] Done: 0/0/0, best #0 3.40282e+38 4,4,1,4,4,1,0,0
**INFO - MIOpen(HIP): Error [FindSolutionImpl] Search failed for: ConvHipImplicitGemmBwdDataV1R1Xdlops: /root/dMIOpen/src/include/miopen/generic_search.hpp:575: Search failed
INFO - MIOpen(HIP): Info [EuristicInit] 32,128,8,32,64,4,0,1**

@alexandraBara
Copy link
Contributor Author

alexandraBara commented Sep 11, 2020 •
edited by atamazov

ConvHipImplicitGemmForwardV4R4Xdlops 
 INFO - Executing: sudo docker run --device='/dev/kfd' --device='/dev/dri' -w /home/miopenpdb -v /home/miopenpdb:/home/miopenpdb --user=root --group-add video --privileged=true --rm miopentuna bash  -c "export MIOPEN_LOG_LEVEL=7 && export MIOPEN_FIND_ENFORCE=3 && export HIP_VISIBLE_DEVICES=7 && MIOpenDriver convfp16 -V 0 -i 1 --forw 1 --pad_h 2 --out_channels 64 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 14 --conv_stride_w 1 --group_count 1 --in_channels 24 --in_w 14 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0 2>&1 " 
 INFO - Setting job id 11100890 state to running
 INFO - MIOpenDriver convfp16 -V 0 -i 1 --forw 1 --pad_h 2 --out_channels 64 --fil_w 5 --dilation_w 1 --fil_h 5 --in_h 14 --conv_stride_w 1 --group_count 1 --in_channels 24 --in_w 14 --dilation_h 1 --conv_stride_h 1 --pad_w 2 --batchsize 128 --pad_mode default --mode conv --fil_d 1 --in_d 1 --spatial_dim 2 --conv_stride_d 1 --dilation_d 1 --pad_d 0 --trans_output_pad_d 0
 INFO - MIOpen(HIP): Info [Handle] stream: 0x2f7ff70, device_id: 0
 INFO - MIOpen(HIP): Info [ForwardGetWorkSpaceSize]
 INFO - MIOpen(HIP): Info2 [HipCompilerVersionImpl] Read version information from HIP package...
 INFO - MIOpen(HIP): Info [HipCompilerVersionImpl] 3.6.20263
 INFO - MIOpen(HIP): Info [AmdRocmMetadataVersionDetect] ROCm MD version AMDHSA_COv3, MIOpen version 2.7.0.8186-ab68183b
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Running: '/opt/rocm/llvm/bin/clang --version'
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] clang version 11.0.0 (/data/jenkins_workspace/compute-rocm-rel-3.6/external/llvm-project/clang f7b7e21a21d08df6971d2c77315a0e41b7639334)
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Target: x86_64-unknown-linux-gnu
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] Thread model: posix
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl] InstalledDir: /opt/rocm/llvm/bin
 INFO - MIOpen(HIP): Info2 [ValidateGcnAssemblerImpl]
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file /opt/rocm/miopen/share/miopen/db/miopen.db
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(config);
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(perf_db);
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.config/miopen/miopen_1.0.0.udb
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'config');
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:SELECT name FROM sqlite_master WHERE type = 'table' AND (name = 'perf_db');
 INFO - MIOpen(HIP): Trace [SQLitePerfDb] Database created successfully
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(config);
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(perf_db);
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Not applicable
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3 (db access disabled)
 INFO - MIOpen(HIP): Info [GetPerformanceConfig] 120
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
 INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE enforced to NORMAL(1) due to MIOPEN_FIND_ENFORCE
 INFO - MIOpen(HIP): Info [GetFindModeValueImpl] MIOPEN_FIND_MODE = NORMAL(1)
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1U: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm1x1UV2: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2f1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvAsm5x10u2v2b1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd11x11: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwdGen: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd3x3: Not applicable
 INFO - MIOpen(HIP): Info2 [GetWorkspaceSize] ConvOclDirectFwd1x1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,16,16,2,2,8,2,1
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmForwardV4R4Xdlops (db access disabled)
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [GetPerformanceConfig] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvHipImplicitGemmForwardV4R4Xdlops: Success.
 INFO - MIOpen(HIP): Info2 [ForwardGetWorkSpaceSize] 235200
 INFO - MIOpen(HIP): Info [FindConvFwdAlgorithm] requestAlgoCount = 2, workspace = 235200
 INFO - MIOpen(HIP): Info2 [FindRecordUnsafe] Looking for key 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F in file /home/miopenpdb/.config/miopen/gfx90878.HIP.2_7_0_8186-ab68183b.ufdb.txt
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.909458 ms
 INFO - MIOpen(HIP): Info [TryLoad] Find-db regenerating.
 INFO - MIOpen(HIP): Info2 [GetKernels] 0 kernels for key: miopenIm2d2Col "c24i14_14w5_5p2_2s1_1d1_1t0"
 INFO - MIOpen(HIP): Info2 [AddKernel] Key: miopenIm2Col "c24i14_14w5_5p2_2s1_1d1_1t0"
 INFO - MIOpen(HIP): Info2 [AddKernelDumpKernelParams] runcl MIOpenIm2d2Col.cl -k Im2d2Col -dumpilisa -r 10 if#0: if#0: if#0: iv#0 12288,1,1/256,1,1  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing system database file
 INFO - MIOpen(HIP): Info [KernDb] database not present
 INFO - MIOpen(HIP): Info2 [SQLiteBase] Initializing user database file /home/miopenpdb/.cache/2.7.0.8186-ab68183b/gfx90878.ukdb
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:CREATE TABLE IF NOT EXISTS `kern_db` (`id` INTEGER PRIMARY KEY ASC,`kernel_name` TEXT NOT NULL,`kernel_args` TEXT NOT NULL,`kernel_blob` BLOB NOT NULL,`kernel_hash` TEXT NOT NULL,`uncompressed_size` INT NOT NULL);CREATE UNIQUE INDEX IF NOT EXISTS `idx_kern_db` ON kern_db(kernel_name, kernel_args, kernel_hash, uncompressed_size);
 INFO - MIOpen(HIP): Info2 [KernDb] Database created successfully
 INFO - MIOpen(HIP): Trace [Exec] 140589208588672:PRAGMA table_info(kern_db);
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: MIOpenIm2d2Col.cl ;args:  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'MIOpenIm2d2Col.cl.o') AND (kernel_args = ' -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.128953 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Unable to load binary for: MIOpenIm2d2Col.cl ;args:  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [SaveBinary] Saving binary for: MIOpenIm2d2Col.cl ;args:  -DNUM_CH_PER_WG=1 -DNUM_IM_BLKS_X=1 -DNUM_IM_BLKS=2 -DLOCAL_MEM_SIZE=432 -DSTRIDE_GT_1=0 -DTILE_SZ_X=32 -DTILE_SZ_Y=8 -DUSE_IM_OFF_GUARD=1 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO kern_db(kernel_name, kernel_args, kernel_blob, kernel_hash, uncompressed_size) VALUES(?, ?, ?, ?, ?);
 INFO - MIOpen(HIP): Info2 [Measure] Db::StoreRecord time: 7.56533 ms
 INFO - MIOpen(HIP): Info2 [CallGemm] gemm_desc: {isColMajor 0, transA 0, transB 0, m 64, n 196, k 600, lda 600, ldb 196, ldc 196, batch_count 1, strideA 0, strideB 0, strideC 0, alpha 1, beta 0, dataType 0}
 INFO - MIOpen(HIP): Info2 [CallGemm] gemm_desc: {isColMajor 0, transA 0, transB 0, m 64, n 196, k 600, lda 600, ldb 196, ldc 196, batch_count 1, strideA 0, strideB 0, strideC 0, alpha 1, beta 0, dataType 0}
 INFO - MIOpen(HIP): Info2 [dummy_memset] dummy gpu memset
 INFO - MIOpen(HIP): Info [SetValues] 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F, content inserted: miopenConvolutionFwdAlgoGEMM:gemm,8.8064,235200,rocBlas,<unused>
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinograd3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf3x2: Not applicable
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxSf2x3
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwdFused:8,8,16,16,2,2,8,2,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info [GetValues] =ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 138.273 ms
 INFO - MIOpen(HIP): Info2 [FindSolutionImpl] Perf Db: record loaded: ConvBinWinogradRxSf2x3
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxSf2x3: Success.
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvBinWinogradRxS (not searchable)
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvBinWinogradRxS: Success.
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<3-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<4-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<5-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvMPBidirectWinograd<6-3>: Not applicable
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v21_1_0_gfx9_fp16_dot2_edc_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v21_1_0_gfx9_fp16_dot2_edc_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: Conv_Winograd_v14_3_3_fp16dot_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'Conv_Winograd_v14_3_3_fp16dot_stride1.s.o') AND (kernel_args = '-Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.8644 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v21_1_0_gfx9_fp16_dot2_edc_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 3.61705 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Sucessfully loaded binary for: Conv_Winograd_v14_3_3_fp16dot_stride1.s ;args: -Wa,-defsym,ROCM_METADATA_VERSION=5 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConv_v21_1_0_gfx9_fp16_dot2_edc_stride1
 INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 G=1 C=24 H=14 W=14 K=64 n_groups=98 flags=1536 R=5 S=5 pad_H=2 pad_W=2 out_H=14 out_W=14 d_buf.byte_stride.nk=9408 d_buf.byte_stride.c=392 d_buf.byte_stride.h=28 d_buf.byte_stride.w=2 f_buf.byte_stride.nk=1200 f_buf.byte_stride.c=50 f_buf.byte_stride.h=10 f_buf.byte_stride.w=2 o_buf.byte_stride.nk=25088 o_buf.byte_stride.c=392 o_buf.byte_stride.h=28 o_buf.byte_stride.w=2 d_buf.byte_stride.g=9408 o_buf.byte_stride.g=25088 f_buf.byte_stride.g=76800
 INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp16_dot2_edc_stride1: 0.111999 < 3.40282e+38
 INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: miopenSp3AsmConvRxSU
 INFO - MIOpen(HIP): Info2 [GetSolution]  N=128 C=24 H=14 W=14 K=64 n_groups=120 flags=0 R=5 S=5 pad_H=2 pad_W=2 out_H=14 out_W=14
 INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvBinWinogradRxS: miopenSp3AsmConvRxSU: 0.120479 >= 0.111999
 INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF and solver ConvBinWinogradRxSf2x3
 INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvBinWinogradRxSf2x3 registered as find 1.0 best for miopenConvolutionFwdAlgoWinograd in 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF
 INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvBinWinogradRxSf2x3: miopenSp3AsmConv_v21_1_0_gfx9_fp16_dot2_edc_stride1: 0.111999, workspce_sz = 0
 INFO - MIOpen(HIP): Info [SetValues] 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F, content inserted: miopenConvolutionFwdAlgoWinograd:ConvBinWinogradRxSf2x3,0.111999,0,miopenConvolutionFwdAlgoWinograd,<unused>
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm3x3U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1U: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm1x1UV2: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2f1: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm7x7c3h224w224k64u2v2p3q3f1: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvAsm5x10u2v2b1: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd11x11: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwdGen: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd3x3: Not applicable
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd1x1: Not applicable
 INFO - MIOpen(HIP): Info2 [GetPerformanceConfig] Returns: 8,8,16,16,2,2,8,2,1
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvOclDirectFwd
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwdFused:8,8,16,16,2,2,8,2,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info [GetValues] =ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 133.11 ms
 INFO - MIOpen(HIP): Info2 [FindSolutionImpl] Perf Db: record loaded: ConvOclDirectFwd
 INFO - MIOpen(HIP): Info2 [SearchForAllSolutions] ConvOclDirectFwd: Success.
 INFO - MIOpen(HIP): Info2 [LoadBinary] Loading binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT kernel_blob, kernel_hash, uncompressed_size FROM kern_db WHERE (kernel_name = 'MIOpenConvDirUni.cl.o') AND (kernel_args = ' -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908');
 INFO - MIOpen(HIP): Info2 [Measure] Db::FindRecord time: 0.085592 ms
 INFO - MIOpen(HIP): Info2 [LoadBinary] Unable to load binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [SaveBinary] Saving binary for: MIOpenConvDirUni.cl ;args:  -DMLO_HW_WAVE_SZ=64 -DMLO_DIR_FORWARD=1 -DMLO_FILTER_SIZE0=5 -DMLO_FILTER_SIZE1=5 -DMLO_FILTER_PAD0=2 -DMLO_FILTER_PAD1=2 -DMLO_FILTER_STRIDE0=1 -DMLO_FILTER_STRIDE1=1 -DMLO_N_OUTPUTS=64 -DMLO_N_INPUTS=24 -DMLO_BATCH_SZ=128 -DMLO_OUT_WIDTH=14 -DMLO_OUT_HEIGHT=14 -DMLO_OUT_BATCH_STRIDE=12544 -DMLO_OUT_CHANNEL_STRIDE=196 -DMLO_OUT_STRIDE=14 -DMLO_IN_WIDTH=14 -DMLO_IN_HEIGHT=14 -DMLO_IN_BATCH_STRIDE=4704 -DMLO_IN_CHANNEL_STRIDE=196 -DMLO_IN_STRIDE=14 -DMLO_IN_TILE0=16 -DMLO_IN_TILE1=16 -DMLO_GRP_TILE0=16 -DMLO_GRP_TILE1=8 -DMLO_OUT_TILE0=1 -DMLO_OUT_TILE1=2 -DMLO_N_STACKS=1 -DMLO_N_OUT_TILES=8 -DMLO_N_OUT_TILES_PERSTACK=8 -DMLO_N_IN_TILES_PERSTACK=4 -DMLO_N_READ_PROCS=128 -DMLO_ALU_VTILE0=16 -DMLO_ALU_VTILE1=8 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -DMLO_CONV_BIAS=0 -DMIOPEN_USE_FP16=1 -DMIOPEN_USE_FP32=0 -DMIOPEN_USE_INT8=0 -DMIOPEN_USE_INT8x4=0 -DMIOPEN_USE_BFP16=0 -DMIOPEN_USE_INT32=0 -DMIOPEN_USE_RNE_BFLOAT16=1 -mcpu=gfx908
 INFO - MIOpen(HIP): Info2 [Prepare] INSERT OR REPLACE INTO kern_db(kernel_name, kernel_args, kernel_blob, kernel_hash, uncompressed_size) VALUES(?, ?, ?, ?, ?);
 INFO - MIOpen(HIP): Info2 [Measure] Db::StoreRecord time: 9.46558 ms
 INFO - MIOpen(HIP): Info2 [PrepareInvoker] Preparing kernel: MIOpenConvUni
 INFO - MIOpen(HIP): Info [EvaluateInvokers] ConvOclDirectFwd: MIOpenConvUni: 0.215039 < 3.40282e+38
 INFO - MIOpen(HIP): Info2 [Register] Invoker registered for algorithm 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF and solver ConvOclDirectFwd
 INFO - MIOpen(HIP): Info2 [SetAsFound1_0] Solver ConvOclDirectFwd registered as find 1.0 best for miopenConvolutionFwdAlgoDirect in 24x14x14x5x5x64x14x14x128xNCHWxFP16x2x2x1x1x1x1x1xF
 INFO - MIOpen(HIP): Info [EvaluateInvokers] Selected: ConvOclDirectFwd: MIOpenConvUni: 0.215039, workspce_sz = 0
 INFO - MIOpen(HIP): Info [SetValues] 24-14-14-5x5-64-14-14-128-2x2-1x1-1x1-0-NCHW-FP16-F, content inserted: miopenConvolutionFwdAlgoDirect:ConvOclDirectFwd,0.215039,0,miopenConvolutionFwdAlgoDirect,<unused>
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [FindSolutionImpl] ConvHipImplicitGemmForwardV4R4Xdlops
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info2 [Prepare] SELECT solver, params FROM perf_db INNER JOIN config ON perf_db.config = config.id WHERE ( (layout = ? ) AND (data_type = ? ) AND (direction = ? ) AND (spatial_dim = ? ) AND (in_channels = ? ) AND (in_h = ? ) AND (in_w = ? ) AND (in_d = ? ) AND (fil_h = ? ) AND (fil_w = ? ) AND (fil_d = ? ) AND (out_channels = ? ) AND (batchsize = ? ) AND (pad_h = ? ) AND (pad_w = ? ) AND (pad_d = ? ) AND (conv_stride_h = ? ) AND (conv_stride_w = ? ) AND (conv_stride_d = ? ) AND (dilation_h = ? ) AND (dilation_w = ? ) AND (dilation_d = ? ) AND (bias = ? ) AND (group_count = ? ) )AND (arch = 'gfx908' ) AND (num_cu = '120');
 INFO - MIOpen(HIP): Info2 [impl] [NCHW,FP16,F,2,24,14,14,1,5,5,1,64,128,2,2,0,1,1,0,1,1,0,0,1]
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwd:8,16,16,16,2,1,8,4,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvOclDirectFwdFused:8,8,16,16,2,2,8,2,2
 INFO - MIOpen(HIP): Info [SetValues] , content inserted: ConvBinWinogradRxSf2x3:98
 INFO - MIOpen(HIP): Info [GetValues] =ConvHipImplicitGemmForwardV4R4Xdlops:<values not found>
 INFO - MIOpen(HIP): Info2 [Measure] Db::Load time: 133.105 ms
 INFO - MIOpen(HIP): Info [FindSolutionImpl] Perf Db: record not found for: ConvHipImplicitGemmForwardV4R4Xdlops
 INFO - MIOpen(HIP): Info [FindSolutionImpl] Starting search: ConvHipImplicitGemmForwardV4R4Xdlops, enforce: SEARCH(3), ALL(1)
 INFO - MIOpen(HIP): Info [EuristicInit] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Info [GetPerformanceConfig] 64,256,1,64,64,8,0,1,8
 INFO - MIOpen(HIP): Warning [GenericSearch] ConvHipImplicitGemmForwardV4R4Xdlops: Searching the best solution among 0 (spare)...
 **INFO - MIOpen(HIP): Warning [GenericSearch] Done: 0/0/0, best #0 3.40282e+38 4,4,1,4,4,1,0,0,1
 INFO - MIOpen(HIP): Error [FindSolutionImpl] Search failed for: ConvHipImplicitGemmForwardV4R4Xdlops: /root/dMIOpen/src/include/miopen/generic_search.hpp:575: Search failed**
fpadmin@zigzag:/tmp/gfx908/120cu_10.216.64.100_30103p$

@atamazov
Copy link
Contributor

atamazov commented Sep 11, 2020 •
edited

Both failures relate to the ComputedContainer of PerformanceConfigs. In both cases the following occurs in the log:

MIOpen(HIP): Warning [GenericSearch] ConvHipImplicitGemmForwardV4R4Xdlops: Searching the best solution among 0 (spare)...

This means that the main ComputedContainer is empty. In other words, the Solver is unable to provide GenericSearch with a set of valid PerformanceConfigs for tuning.

Please note that #387 is not yet resolved.

@asroy asroy added this to Issue to do in hip-igemm Sep 28, 2020
@daniellowell daniellowell added this to Needs triage in BUG tracker via automation Oct 19, 2020
@daniellowell daniellowell moved this from Needs triage to High priority in BUG tracker Oct 19, 2020
@daniellowell
Copy link
Contributor

@asroy Please update this issue. Who is working it?

@asleepzzz
Copy link
Contributor

Hi Alex,
I reproduce this issue
with
./bin/MIOpenDriver conv -F 2 -n 128 -c 32 -H 7 -W 7 -k 128 -y 5 -x 5 -p 2 -q 2 -u 1 -v 1 -l 1 -j 1 -g 1 -t 1
&
./bin/MIOpenDriver convfp16 -F 1 -n 128 -c 24 -H 14 -W 14 -k 64 -y 5 -x 5 -p 2 -q 2 -u 1 -v 1 -l 1 -j 1 -g 1 -t 1

ConvHipImplicitGemmForwardV4R4Xdlops search failed
but ConvHipImplicitGemmBwdDataV1R1Xdlops can search if we don't disable it

I'll fix ConvHipImplicitGemmForwardV4R4Xdlops

@asleepzzz
Copy link
Contributor

ConvHipImplicitGemmForwardV4R4Xdlops failed due to
if(GemmKPerBlock * GemmKPack < 16)
return false;
in IsFastToBeUsedForTuning

@asleepzzz
Copy link
Contributor

asleepzzz commented Oct 21, 2020 •
edited by atamazov

will fix in PR #531

@atamazov atamazov mentioned this issue Oct 21, 2020
Merged
BUG tracker automation moved this from High priority to Closed Nov 16, 2020
bghimireamd pushed a commit that referenced this issue Mar 24, 2023
fix build (#427)
567f70f 
* fix build

* fix build
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@atamazov atamazov

@daniellowell daniellowell

@asroy asroy

@jane-zxy jane-zxy

Labels
bug
Projects
BUG tracker
  
Closed
hip-igemm
Issue to do
Milestone
No milestone
Development

No branches or pull requests
8 participants
@alexandraBara @asleepzzz @Kirpich30000 @atamazov @daniellowell @asroy @shurale-nkn @jane-zxy