Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux 4.13+: Warnings from objtool while compiling the 'icp' kernel module: "unsupported stack pointer realignment" (in SHA1/SHA2 functions) #6950

Open
jgottula opened this Issue Dec 13, 2017 · 6 comments

Comments

Projects
None yet
6 participants
@jgottula
Copy link
Contributor

jgottula commented Dec 13, 2017

I don't know how important these build-time warnings actually are, but I've typed up a full report here anyway.

The warnings

I was building the latest zfs master, 4e9b156, for Linux 4.14.5, on x86_64 today.

Here's what came up during the build process:

/home/build/zfs/zfs-20171212-latest/archzfs/packages/linux/zfs-linux-git/src/zfs/module/icp/asm-x86_64/sha1/sha1-x86_64.o: warning: objtool: sha1_block_data_order()+0x11: unsupported stack pointer realignment
/home/build/zfs/zfs-20171212-latest/archzfs/packages/linux/zfs-linux-git/src/zfs/module/icp/asm-x86_64/sha2/sha256_impl.o: warning: objtool: SHA256TransformBlocks()+0x19: unsupported stack pointer realignment
/home/build/zfs/zfs-20171212-latest/archzfs/packages/linux/zfs-linux-git/src/zfs/module/icp/asm-x86_64/sha2/sha512_impl.o: warning: objtool: SHA512TransformBlocks()+0x1c: unsupported stack pointer realignment

Those warnings correspond to these particular instructions from the objdump'd object files:

0000000000000000 <sha1_block_data_order>:
      ...
      11:       48 83 e4 c0             and    rsp,0xffffffffffffffc0
      ...
0000000000000000 <SHA256TransformBlocks>:
      ...
      19:       48 83 e4 c0             and    rsp,0xffffffffffffffc0
      ...
0000000000000000 <SHA512TransformBlocks>:
      ...
      1c:       48 83 e4 c0             and    rsp,0xffffffffffffffc0
      ...

The recent update to objtool

It looks like this warning will only appear when compiling for Linux 4.13 or later, since the objtool patchset that added new warnings (including the "unsupported stack pointer realignment" one) was merged into the mainline kernel in 4.13.

Here's some additional info on that objtool patchset:

Here's the specific part of the objtool code that's complaining.
And here's some internal documentation regarding what it's complaining about.

The ZoL source lines being complained about

Here are the relevant assembly code source file lines in ZoL:


and $-64,%rsp # align stack frame

and $-64,%rsp # align stack frame

32-bit

As far as I can tell, i386 builds of zfs won't emit this warning, as from what I can see, the asm implementations of the three SHA functions in question are only used for the kernel module on x86_64, and everything else just uses the C implementations.

But I don't have an i386 Linux box/VM ready-at-hand to do a quick build and actually confirm for sure that these warnings don't happen there.

@jgottula

This comment has been minimized.

Copy link
Contributor Author

jgottula commented Dec 13, 2017

Also, I thought it'd be worth mentioning that the asm implementations of SHA1/SHA2 in the ICP module come from Illumos, which in turn got them from OpenSSL; quite a long time ago, by the looks of it (2009 at the very latest).

It looks like the upstream OpenSSL implementation has advanced quite a bit since Illumos branched their code off; among other things, the OpenSSL version now has specializations for SSSE3, AVX, AVX2+BMI, and Intel SHA instruction set extensions now.

So that might be worth looking at sometime. (And/or #2351.)

@h1z1

This comment has been minimized.

Copy link

h1z1 commented Jan 31, 2018

Still outstanding as of latest git. Funny thing is this was the first hit on google for that error :)

behlendorf added a commit to behlendorf/zfs that referenced this issue Mar 5, 2018

OpenZFS 9188 - increase size of dbuf cache to reduce indirect block d…
…ecompression

With compressed ARC (bug zfsonlinux#6950) we use up to 25% of our CPU to decompress
indirect blocks, under a workload of random cached reads. To reduce this
decompression cost, we would like to increase the size of the dbuf cache so
that more indirect blocks can be stored uncompressed.

If we are caching entire large files of recordsize=8K, the indirect blocks
use 1/64th as much memory as the data blocks (assuming they have the same
compression ratio). We suggest making the dbuf cache be 1/32nd of all memory,
so that in this scenario we should be able to keep all the indirect blocks
decompressed in the dbuf cache. (We want it to be more than the 1/64th that
the indirect blocks would use because we need to cache other stuff in the dbuf
cache as well.)

In real world workloads, this won't help as dramatically as the example above,
but we think it's still worth it because the risk of decreasing performance is
low. The potential negative performance impact is that we will be slightly
reducing the size of the ARC (by ~3%).

Porting Notes:
* Added modules options to zfs-module-parameters.5 man page.
* Preserved scaling based on target ARC size rather than max ARC size.

Authored by: George Wilson <george.wilson@delphix.com>
Reviewed by: Dan Kimmel <dan.kimmel@delphix.com>
Reviewed by: Prashanth Sreenivasa <pks@delphix.com>
Reviewed by: Paul Dagnelie <pcd@delphix.com>
Ported-by: Brian Behlendorf <behlendorf1@llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/9188
OpenZFS-commit: openzfs/openzfs#564
Upstream bug: DLPX-46942
@c0d3z3r0

This comment has been minimized.

Copy link
Contributor

c0d3z3r0 commented Nov 11, 2018

Confirmed on Debian testing with ZFS 0.7.11, custom Kernel 4.19.1
(I know 0.7.11 officially supports 4.18 only... just for completeness)

[...]
/var/lib/dkms/zfs/0.7.11/build/module/icp/asm-x86_64/sha1/sha1-x86_64.o: warning: objtool: sha1_block_data_order()+0x11: unsupported stack pointer realignment
  AS [M]  /var/lib/dkms/zfs/0.7.11/build/module/icp/asm-x86_64/sha2/sha512_impl.o
/var/lib/dkms/zfs/0.7.11/build/module/icp/asm-x86_64/sha2/sha256_impl.o: warning: objtool: SHA256TransformBlocks()+0x19: unsupported stack pointer realignment
/var/lib/dkms/zfs/0.7.11/build/module/icp/asm-x86_64/sha2/sha512_impl.o: warning: objtool: SHA512TransformBlocks()+0x1c: unsupported stack pointer realignment
  LD [M]  /var/lib/dkms/zfs/0.7.11/build/module/icp/icp.o
make[3]: *** [Makefile:1517: _module_/var/lib/dkms/zfs/0.7.11/build/module] Error 2
make[3]: Leaving directory '/usr/src/linux-headers-4.19.1'
make[2]: *** [Makefile:27: modules] Error 2
make[2]: Leaving directory '/var/lib/dkms/zfs/0.7.11/build/module'
make[1]: *** [Makefile:701: all-recursive] Error 1
make[1]: Leaving directory '/var/lib/dkms/zfs/0.7.11/build'
make: *** [Makefile:572: all] Error 2
root@debian:/usr/src/kernel# dkms status
spl, 0.7.11, 4.18.0-2-amd64, x86_64: installed
spl, 0.7.11, 4.19.1, x86_64: installed
zfs, 0.7.11, 4.18.0-2-amd64, x86_64: installed
@darrenfreeman

This comment has been minimized.

Copy link

darrenfreeman commented Jan 31, 2019

What are the consequences of this? Is it safe to ignore?

@3f8wohoo

This comment has been minimized.

Copy link

3f8wohoo commented Feb 14, 2019

can confirm this for 0.7.12 and kernel 4.19.14 on debian stretch:

...
Making all in 02zfsexpandknowledge
Making all in 90zfs
Making all in initramfs
Making all in module
/root/zfs-0.7.12/module/icp/asm-x86_64/sha1/.tmp_sha1-x86_64.o: warning: objtool: sha1_block_data_order()+0x11: unsupported stack pointer realignment
/root/zfs-0.7.12/module/icp/asm-x86_64/sha2/.tmp_sha256_impl.o: warning: objtool: SHA256TransformBlocks()+0x19: unsupported stack pointer realignment
/root/zfs-0.7.12/module/icp/asm-x86_64/sha2/.tmp_sha512_impl.o: warning: objtool: SHA512TransformBlocks()+0x1c: unsupported stack pointer realignment
@behlendorf

This comment has been minimized.

Copy link
Member

behlendorf commented Feb 15, 2019

Yes, it's safe to ignore. Though of course we would like to resolve the warning.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.