Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[umbrella task] Track and improve code size #2867

Open
terrelln opened this issue Nov 16, 2021 · 1 comment
Open

[umbrella task] Track and improve code size #2867

terrelln opened this issue Nov 16, 2021 · 1 comment
Labels
long-term valid topics that are expected take a long time to make progress

Comments

@terrelln
Copy link
Contributor

Zstd has been getting larger over the years, as we've added features and improved performance. We should take a comprehensive look at code size, and try to optimize our code size while retaining performance.

  1. Explicitly track our code size in our tests, and monitor between releases.
  2. Look at the largest code size files and try to find wins.
    a. zstd_opt.c is a prime target: Issue [linux-kernel] Don't add -O3 to CFLAGS #2866.
    b. zstd_lazy.c was optimized in PR [lazy] Speed up compilation times #2828, but there is still more that can be won (as mentioned in the TODO).
  3. Enable modular builds. E.g. allow cutting out dictionary support, or cutting out high compression levels.
@terrelln terrelln added the long-term valid topics that are expected take a long time to make progress label Nov 16, 2021
torvalds added a commit to torvalds/linux that referenced this issue Nov 19, 2021
Pull zstd fixes from Nick Terrell:
 "Fix stack usage on parisc & improve code size bloat

  This contains three commits:

   1. Fixes a minor unused variable warning reported by Kernel test
      robot [0].

   2. Improves the reported code bloat (-88KB / 374KB) [1] by outlining
      some functions that are unlikely to be used in performance
      sensitive workloads.

   3. Fixes the reported excess stack usage on parisc [2] by removing
      -O3 from zstd's compilation flags. -O3 triggered bugs in the
      hppa-linux-gnu gcc-8 compiler. -O2 performance is acceptable:
      neutral compression, about -1% decompression speed. We also reduce
      code bloat (-105KB / 374KB).

  After this our code bloat is cut from 374KB to 105KB with gcc-11. If
  we wanted to cut the remaining 105KB we'd likely have to trade
  signicant performance, so I want to say that this is enough for now.

  We should be able to get further gains without sacrificing speed, but
  that will take some significant optimization effort, and isn't
  suitable for a quick fix. I've opened an upstream issue [3] to track
  the code size, and try to avoid future regressions, and improve it in
  the long term"

Link: https://lore.kernel.org/linux-mm/202111120312.833wII4i-lkp@intel.com/T/ [0]
Link: https://lkml.org/lkml/2021/11/15/710 [1]
Link: https://lkml.org/lkml/2021/11/14/189 [2]
Link: facebook/zstd#2867 [3]
Link: https://lore.kernel.org/r/20211117014949.1169186-1-nickrterrell@gmail.com/
Link: https://lore.kernel.org/r/20211117201459.1194876-1-nickrterrell@gmail.com/

* tag 'zstd-for-linus-5.16-rc1' of git://github.com/terrelln/linux:
  lib: zstd: Don't add -O3 to cflags
  lib: zstd: Don't inline functions in zstd_opt.c
  lib: zstd: Fix unused variable warning
@sav-ix
Copy link

sav-ix commented Nov 21, 2021

hello,

wondering, if dropping build flags because of buggy compiler is a standard way to solve the issues.
what will happen, if -O1 became buggy on that parisc system? if something different, than in e2d0186, why this something didn't happen for -O3 bug? thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
long-term valid topics that are expected take a long time to make progress
Projects
None yet
Development

No branches or pull requests

2 participants