AMDGPU-Pro issues

[claudioandre-br]

• SIGSEGV

Testing: md5crypt-opencl, crypt(3) $1$ [MD5 OpenCL]... Falha de segmentação

• If it works for someone else, a workaround can be:

	if (major == xxxx && minor == x) {
		fprintf(stderr,
		        "The OpenCL driver in use cannot run this kernel. Please, update your driver!\n");
		error();
	}

• Review max_constant_size usage

warning: unknown attribute 'max_constant_size' ignored

• Failing formats

Testing: TrueCrypt-opencl [RIPEMD160 AES256_XTS OpenCL]... FAILED (cmp_all(1))
Testing: ansible-opencl, Ansible Vault [PBKDF2-SHA256 HMAC-SHA256 OpenCL]... FAILED (cmp_all(1))
Testing: cloudkeychain-opencl, 1Password Cloud Keychain [PBKDF2-SHA512 AES OpenCL]... FAILED (cmp_all(1))
Testing: EncFS-opencl [PBKDF2-SHA1 AES OpenCL]... FAILED (cmp_all(1))
Testing: geli-opencl, FreeBSD GELI [PBKDF2-SHA512 AES OpenCL]... FAILED (cmp_all(1))
Testing: krb5pa-sha1-opencl, Kerberos 5 AS-REQ Pre-Auth etype 17/18 [PBKDF2-SHA1 OpenCL]... FAILED (cmp_all(1))
Testing: krb5asrep-aes-opencl, Kerberos 5 AS-REP etype 17/18 [PBKDF2-SHA1 OpenCL]... FAILED (cmp_all(1))
Testing: pgpdisk-opencl [SHA1 AES/TwoFish/CAST OpenCL]... FAILED (cmp_all(1))
Testing: raw-SHA512-opencl [SHA512 OpenCL]... FAILED (cmp_one(0))
Testing: XSHA512-opencl, Mac OS X 10.7 salted [SHA512 OpenCL]... FAILED (cmp_one(0))
Testing: ZIP-opencl, WinZip [PBKDF2-SHA1 OpenCL]... FAILED (cmp_all(1))

11 out of 86 tests have FAILED

[claudioandre-br]

I need to re-sync with sha512crypt (wait the PR and my post merge tests).

Testing: raw-SHA512-opencl [SHA512 OpenCL]... FAILED (cmp_one(0))
Testing: XSHA512-opencl, Mac OS X 10.7 salted [SHA512 OpenCL]... FAILED (cmp_one(0))

[solardiz]

Thanks, Claudio!

I think we should try getting md5crypt-opencl to work with AMDGPU-Pro, maybe in one of 2 ways:

1. Try to bisect it - comment out portions of the kernel until we've identified the smallest portion to comment out so that the segfault is gone (it is OK that the format will temporarily fail self-test because of incomplete implementation), then reimplement that portion in a way not triggering the driver bug.

2. Reimplement md5crypt-opencl without pre-filled buffers (or perhaps with fewer of them - just for the most frequently used combinations of strings), which would also speed it up (per Hashcat).

[solardiz]

I got md5crypt-opencl to work with AMDGPU-PRO with a change made in #3624.

[magnumripper]

The unknown attribute stuff is annoying, can we decide on some version numbers where we can use it, and where we can't? I'm not sure it's needed at all anywhere.

[solardiz]

According to @AlekseyCherepanov, only 5 out of 91 OpenCL formats currently fail with amdgpu-pro on R9 390:

https://www.openwall.com/lists/john-users/2021/11/28/3

Benchmarking: TrueCrypt-opencl [RIPEMD160 AES256_XTS OpenCL]... FAILED (cmp_all(1))

Benchmarking: axcrypt2-opencl, AxCrypt 2.x [PBKDF2-SHA512 AES OpenCL]... FAILED (cmp_all(1))

Benchmarking: pfx-opencl, (.pfx, .p12) [PKCS#12 PBE (SHA1/SHA-256/512) OpenCL]... FAILED (cmp_all(5))

Benchmarking: pgpdisk-opencl, PGP Disk / Virtual Disk [SHA1 AES/TwoFish/CAST OpenCL]... FAILED (cmp_all(1))

Benchmarking: sappse-opencl, SAP PSE [PKCS#12 PBE (SHA1) OpenCL]... FAILED (cmp_all(1))

The errors are the same for 2 versions of amdgpu-pro I tried:
amdgpu-pro-20.10-1084971-ubuntu-18.04
amdgpu-pro-20.40-1147286-ubuntu-20.04

OTOH, out of these only 2 (TrueCrypt-opencl and pgpdisk-opencl) were on @claudioandre-br's list of formats failing in 2019, and 3 are new (were not failing before).

[AlekseyCherepanov]

It looks like it is possible to trigger auto-tuning before TestDB using --gws=1.

Regular:

$ ./run/john --format=axcrypt2-opencl --test --verbosity=6
initUnicode(UNICODE, RAW/RAW)
RAW -> RAW -> RAW
Device 1: Hawaii [AMD Radeon (TM) R9 390 Series]
Benchmarking: axcrypt2-opencl, AxCrypt 2.x [PBKDF2-SHA512 AES OpenCL]... Loaded 2 hashes with 2 different salts to test db from test vectors
Options used: -I opencl -cl-mad-enable -D__GPU__ -DDEVICE_INFO=266 -D__SIZEOF_HOST_SIZE_T__=8 -DDEV_VER_MAJOR=3180 -DDEV_VER_MINOR=7 -D_OPENCL_COMPILER -DHASH_LOOPS=250 -DPLAINTEXT_LENGTH=110 -DPBKDF2_64_MAX_SALT_SIZE=107 $JOHN/opencl/axcrypt2_kernel.cl
binary size 4099224
TestDB LWS=7 GWS=49 (7 blocks) FAILED (cmp_all(1) $axcrypt$*2*28200*96fca6b5aff19cb540125e6452f1ab6e1cbf7097ad75326e7aa1726f687ed18e66a63295d735f458a138901be41d95d29ff760c7c4b4178320251f2ab0ecabd8*d1d90c425fd8e2d20b2aa7ef6f3ab9dcefad84917)

With --gws=1:

$ ./run/john --format=axcrypt2-opencl --test --verbosity=6 --gws=1
initUnicode(UNICODE, RAW/RAW)
RAW -> RAW -> RAW
Device 1: Hawaii [AMD Radeon (TM) R9 390 Series]
Benchmarking: axcrypt2-opencl, AxCrypt 2.x [PBKDF2-SHA512 AES OpenCL]... Loaded 2 hashes with 2 different salts to test db from test vectors
Calculating best GWS for LWS=7; max. 200 ms single kernel invocation.
Raw speed figures including buffer transfers:
Tuning for iteration count of 28200 and password length 7

At this point it hangs on amdgpu-pro. reboot helps. (So my comment on ability to reboot in #4108 is not specific to mesa.)

--lws=1 --gws=1 gives 1/1 values without auto-tuning.

Other failing formats do not hang, they complete auto-tuning for LWS=7 and then fail as usual.