Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use -O3 and link-time-optimization for published builds #1444

Merged
merged 1 commit into from
Oct 17, 2023

Conversation

ychin
Copy link
Member

@ychin ychin commented Oct 17, 2023

From testing and benchmarking, it appears that both result in a measurable improvement in performance, wtih some benchmarks showing 10% faster (when opening a large 400 MB binary file and searching-and-replacing within it). Use them when building a published build. Don't do it for legacy builds as I encountered some issues with it failing tests when testing for recursion limit and I suspect it's due to stack size issues. Since legacy builds are mostly kept for compatibility reasons, no need to optimize it for now.

From testing and benchmarking, it appears that both result in a
measurable improvement in performance, wtih some benchmarks showing 10%
faster (when opening a large 400 MB binary file and
searching-and-replacing within it). Use them when building a published
build. Don't do it for legacy builds as I encountered some issues with
it failing tests when testing for recursion limit and I suspect it's due
to stack size issues. Since legacy builds are mostly kept for
compatibility reasons, no need to optimize it for now.
@ychin ychin added the Infrastructure Non-app infrastructure issues, e.g. CI label Oct 17, 2023
@ychin ychin added this to the Release 179 milestone Oct 17, 2023
@github-actions github-actions bot added the CI Vim upstream label for CI issues label Oct 17, 2023
@ychin
Copy link
Member Author

ychin commented Oct 17, 2023

Just to document some of the benchmarks that I have done here in case other people are curious. I saw Neovim turning on -O3 and LTO and there were also questions about the concrete effects it brings as it wasn't benchmarked (neovim/neovim#23051). This is not a very scientific test but I made some benchmarks that I used to quickly see the results using different areas of Vim.

  1. open: Open Vim's README and quit. Do it 100 times.
  2. mkv: Open a ~400 MB binary video file, perform :%s/a/b in it and quit, and observe how long it takes.
  3. markdown: Install vim-markdown plugin (which has a lot of vimscript in it), then open a large (400 kB) Markdown file and quit and see how long it takes.
  4. vim9compile: Source a large file (just took the vim9 LSP plugin, duplicate the main file 670 times with renamed functions, resulting in ~23 MB worth on vim9script), and then add a defcompile at the end of file to force a compile.
  5. benchmark: Builtin make benchmark which from what I can tell tests regex. I'm using the last number (the one that takes the longest).

Testing was done on an M1 Max MacBook Pro / Xcode 15, and I tested -O2/-O3/-Os with/without -flto (link-time-optimization). "Perf %" is relative performance to base (-O2/no LTO), and higher is better (higher perf means it took less time to do). Just out of curiosity I ran similar tests on Linux/GCC and also Intel Mac with x86 and they all show similar results so not going to duplicate it here:

name size (bytes) size % open (s) open perf % mkv (s) mkv perf % markdown (s) markdown perf % vim9compile (s) vim9compile perf benchmark (s) benchmark perf%
vim_os 3645056 92% 11.07 94% 8.31 97% 4.88 91% 8.63 94% 6.41 83%
vim_o2 3956864 100% 10.36 100% 8.07 100% 4.43 100% 8.12 100% 5.34 100%
vim_o3 4114752 104% 10.22 101% 7.64 106% 4.38 101% 7.87 103% 5.02 107%
vim_os_lto 3416768 86% 10.65 97% 8.13 99% 4.60 96% 8.24 99% 5.80 92%
vim_o2_lto 4360864 110% 9.95 104% 7.67 105% 4.22 105% 7.93 102% 5.18 103%
vim_o3_lto 4491696 114% 9.86 105% 7.15 113% 4.22 105% 8.03 101% 5.26 102%

Generally -O3 and LTO both increase performance at a almost linear relationship with code size (probably due to better inlining). It's surprising in the "mkv" test case that we see 10+% performance increase when using both, which is a non-trivial improvement. Opening a small file and quit (a common use case for Vim) also shows 5% speed improvement. The other benchmarks show a more mixed results though, but usually it is still a minor improvement. -Os (optimize for binary size) is clearly the worst though (surprisingly -Os + LTO results in smaller code size and better performance). Given Vim is small program, there's really no need to optimize binary size. I did not benchmark memory use though, but I think it was good enough results to convince me to turn on both O3 and LTO.

@ychin ychin merged commit c368ebb into macvim-dev:master Oct 17, 2023
4 checks passed
@ychin ychin deleted the enable-ci-link-time-optimization-2 branch October 17, 2023 09:54
@ychin
Copy link
Member Author

ychin commented Oct 17, 2023

Previously #1314 tried to do this but I didn't have time to do a full benchmark to gauge the benefits. Also, I ran into Makefile dependency issues that resulted in really long build time, which vim/vim#13344 fixed.

ychin added a commit that referenced this pull request Jan 5, 2024
Updated to Vim 9.1.0

Vim 9.1 is now released! See
[announcement](https://www.vim.org/vim-9.1-released.php).

Features
====================

System monospace font (SF Mono)
--------------------

MacVim's `guifont` option now supports a new `-monospace-` value, which
instructs it to use the system monospace font, which is SF Mono in
recent macOS versions. As mentioned below, you can now use
tab-completion to see the available values in cmdline. See `:h
macvim-guifont` for more details on how to use it (including using
different font weights). #1463

Note: I'm contemplating changing the MacVim defaults to use
`-monospace-` in the future so MacVim will always use the native
monospace font instead of being hard-coded to Menlo. This makes it more
consistent with Apple Terminal and Xcode. Feel free to leave a comment
on #1277 if you have opinions on this.

New Vim features
--------------------

- Command-line tab completion improvements and bug fixes
    - Most string option values can now be completed. v9.0.1958
    - MacVim options (guifont, fuoptions) also support tab completion.
      #1436
    - ++opt (e.g. `:e ++`) and `:terminal ++` completion works as well.
      v9.0.2025
- New options:
    - `set jumpoptions=stack`. Ported from Neovim. v9.0.1921
- API changes
    - `getmousepos()` returns a new "coladd" for tab characters.
      v9.0.2032
- `:Man` now works properly when `gdefault` is set.
- A new small Vim script library that may expand in the future. See `:h
  vim-script-library`.
- Vim9 script improvements.
- Miscellaneous security fixes.

Misc
--------------------

New settings:

- "Scroll in one direction only" (Input). Prevents accidental horizontal
  scrolling when scrolling vertically using a trackpad. #1442

Clean mode (#1453):

- Vim can be opened in clean mode (does not use .vimrc or plugins) via
  the new menu item "New Clean Window". The new menu isn't localized in
  most languages. Please comment on the issue if you would like to help
  in localization.
- MacVim can be launched without loading user defaults for a clean
  experience via a command-line flag. See `:h macvim-settings`.

General
====================

- Sparkle (updater for MacVim) is now updated to 2.5.2. The updater can
  now show multiple release notes when updating MacVim across multiple
  versions. #1446 #1469
- Binary release is now built with more optimized compiler settings. Vim
  will now run slightly faster than before. #1444
- macOS 14 Sonoma:
    - Binary release is now built using the macOS 14 SDK (#1434, #1440,
      #1448). One small change is that very tall characters (e.g. "นี้")
      on the first line will now draw into the title bar instead of
      being clipped.
    - Fixed printing with `:hardcopy` under macOS 14. *NOTE:* Starting
      from macOS 14, you have to install `ps2pdf` (available from
      Ghostscript) yourself before you can print. See #1464
- Python 2 support: The default location for locating the Python 2 lib
  in the binary release is now under /Library/Frameworks rather than
  /usr/local. Note: Python 2 has long been obsolete. If you rely on
  Python 2 plugins, consider this a warning as it's only supported as
  long as it's feasible and could be removed in the future. #1434

Fixes
====================

- Fixed non-native full screen mode when using a MacBook with a notch
  and having the "Show menu bar in non-native mode" option set. Changing
  the screen resolution while using non-native full screen also works
  properly now. #1450
- Fixed Help menu's documentation search not working with tags with
  special characters like `<Down>`. #1455

Compatibility
====================

Requires macOS 10.9 or above. (10.9 - 10.12 requires downloading a
separate legacy build)

Script interfaces have compatibility with these versions:

- Lua 5.4
- Perl 5.30
- Python2 2.7
- Python3 3.9 or above
- Ruby 3.2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Vim upstream label for CI issues Infrastructure Non-app infrastructure issues, e.g. CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant