[RDY] Remove maxmem and maxmemtot options #4486

felipecrv · 2016-03-23T07:15:58Z

The option 'maxmem' ('mm') is used to set the maximum memory used for one
buffer (in kilobytes). 'maxmemtot' is used to set the maximum memory used for
all buffers (in kilobytes). The defaults depend on the system used. These
are not hard limits, but tell Vim when to move text into a swap file. If you
don't like Vim to swap to a file, set 'maxmem' and 'maxmemtot' to a very large
value. The swap file will then only be used for recovery. If you don't want
a swap file at all, set 'updatecount' to 0, or use the "-n" argument when
starting Vim.

On today's systems these values are huge (4GB in my machine with 8GB of RAM
since it's set as half the available memory by default) so the limits are
never reached in practice, but Vim wastes a lot of time checking if the limit
was reached.

If the limit is reached Vim starts saving pieces of the swap file that were in
memory to the disk. Said in a different way: Vim implements its own memory
paging mechanism. This is unnecessary and inefficient since the operating
system already has virtual memory and will swap to the disk if programs
start using too much memory.

This change does...

Reduce the number of config options and need for documentation.
Make the code more efficient as we don't have to keep track of memory usage
nor check if the memory limits were reached to start swapping to disk every
time we need memory for buffers.
Simplify the code. Once memfile.c is simple enough it could be replaced by
actual operating system memory mapping (mmap, MemoryViewOfFile...).

This change does not prevent Vim to recover changes from swap files since the
swapping code is never triggered with the huge limits set by default.

fmoralesc · 2016-03-23T10:49:48Z

runtime/doc/options.txt

@@ -4139,17 +4139,6 @@ A jump table for the options with a short description can be found at |Q_op|.
 	"\(.\)*" on a very long line.  ".*" works much better.
 	Vim may run out of memory before hitting the 'maxmempattern' limit.

-						*'maxmemtot'* *'mmt'*


Should also remove the documentation for 'maxmen', no?

Right! Removed it and updated the PR.

fmoralesc · 2016-03-24T07:13:37Z

It looks like clint.py will ask you to change the comment style for a bunch of lines in options_defs.h :(

cjoach · 2016-03-25T20:33:35Z

Why keep variables defined if you ignore it? It's not used after the change anyway.

justinmk · 2016-03-26T15:28:00Z

runtime/optwin.vim

 call append("$", " \tset mm=" . &mm)
-call append("$", "maxmemtot\tmaximum amount of memory in Kbyte used for all buffers")
+call append("$", "maxmemtot\tmaximum amount of memory in Kbyte used for all buffers (ignored in Neovim)")


These lines should be removed totally. That's what we have done for previous "hidden options".

justinmk · 2016-03-26T15:52:02Z

Nice changes @philix.

oni-link · 2016-04-04T17:01:56Z

src/nvim/option.c

-   * 'maxmemtot' and 'maxmem' may have to be adjusted for available memory
-   */
+  // 'maxmemtot' and 'maxmem' are ignored in Neovim, but we set them to the
+  // amount of available memory.
  opt_idx = findoption((char_u *)"maxmemtot");
  if (opt_idx >= 0) {


Unreachable code if option 'maxmemtot' is removed.

Thanks! Forget to remove this since I was planning to not completely remove the variables at first.

felipecrv · 2016-05-07T01:20:00Z

Rebased

jamessan · 2016-12-16T15:30:00Z

Was there anything left to be done on this? Can you rebase?

felipecrv · 2016-12-16T15:37:53Z

Was there anything left to be done on this?

No. I've clarified all the concerns.

Can you rebase?

I will do it this weekend.

felipecrv · 2017-03-03T23:03:32Z

@jamessan @justinmk I finally cleaned this up :)

It's much cleaner now that the formatting stuff has been taken care by someone else.

justinmk · 2017-03-03T23:30:42Z

Thanks! I didn't forget about this...

> The option 'maxmem' ('mm') is used to set the maximum memory used for one > buffer (in kilobytes). 'maxmemtot' is used to set the maximum memory used for > all buffers (in kilobytes). The defaults depend on the system used. These > are not hard limits, but tell Vim when to move text into a swap file. If you > don't like Vim to swap to a file, set 'maxmem' and 'maxmemtot' to a very large > value. The swap file will then only be used for recovery. If you don't want > a swap file at all, set 'updatecount' to 0, or use the "-n" argument when > starting Vim. On today's systems these values are huge (4GB in my machine with 8GB of RAM since it's set as half the available memory by default) so the limits are never reached in practice, but Vim wastes a lot of time checking if the limit was reached. If the limit is reached Vim starts saving pieces of the swap file that were in memory to the disk. Said in a different way: Vim implements its own memory swapping mechanism. This is unnecessary and inefficient since the operating system already virtualized the memory and will swap to the disk if programs start using too much memory. This change does... 1. Reduce the number of config options and need for documentation. 2. Make the code more efficient as we don't have to keep track of memory usage nor check if the memory limits were reached to start swapping to disk every time we need memory for buffers. 3. Simplify the code. Once `memfile.c` is simple enough it could be replaced by actual operating system memory mapping (`mmap`, `MemoryViewOfFile`...). This change does not prevent Vim to recover changes from swap files since the swapping code is never triggered with the huge limits set by default.

After this change we never release blocks from memory (in practice it never happened because the memory limits are never reached). Let the OS take care of that. --- On today's systems the 'maxmem' and 'maxmemtot' values are huge (4+ GB) so the limits are never reached in practice, but Vim wastes a lot of time checking if the limit was reached. If the limit is reached Vim starts saving pieces of the swap file that were in memory to the disk. Said in a different way: Vim implements its own memory-paging mechanism. This is unnecessary and inefficient since the operating system already has virtual memory and will swap to the disk if programs start using too much memory. This change does... 1. Reduce the number of config options and need for documentation. 2. Make the code more efficient as we don't have to keep track of memory usage nor check if the memory limits were reached to start swapping to disk every time we need memory for buffers. 3. Simplify the code. Once memfile.c is simple enough it could be replaced by actual operating system memory mapping (mmap, MemoryViewOfFile...). This change does not prevent Vim to recover changes from swap files since the swapping code is never triggered with the huge limits set by default.

felipecrv · 2018-05-02T10:28:28Z

Yay! :)

…

On Wed, 2 May 2018 at 05:18, Justin M. Keyes ***@***.***> wrote: Merged #4486 <#4486>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#4486 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMrs4uVj0zxbVxViUmjdoAcnJSdvvdQks5tuWvqgaJpZM4H20Ux> .

FEATURES: 3cc7ebf #7234 built-in VimL expression parser 6a7c904 #4419 implement <Cmd> key to invoke command in any mode b836328 #7679 'startup: treat stdin as text instead of commands' 58b210e :digraphs : highlight with hl-SpecialKey #2690 7a13611 #8276 'startup: Let `-s -` read from stdin' 1e71978 events: VimSuspend, VimResume #8280 1e7d5e8 #6272 'stdpath()' f96d99a #8247 server: introduce --listen e8c39f7 #8226 insert-mode: interpret unmapped META as ESC 98e7112 msg: do not scroll entire screen (#8088) f72630b #8055 let negative 'writedelay' show all redraws 5d2dd2e win: has("wsl") on Windows Subsystem for Linux #7330 a4f6cec cmdline: CmdlineEnter and CmdlineLeave autocommands (#7422) 207b7ca #6844 channels: support buffered output and bytes sockets/stdio API: f85cbea #7917 API: buffer updates 418abfc #6743 API: list information about all channels/jobs. 36b2e3f #8375 API: nvim_get_commands 273d2cd #8329 API: Make nvim_set_option() update `:verbose set …` 8d40b36 #8371 API: more reliable/descriptive VimL errors ebb1acb #8353 API: nvim_call_dict_function 9f994bb #8004 API: nvim_list_uis 3405704 #7520 API/UI: forward option updates to UIs 911b1e4 #7821 API: improve nvim_command_output WINDOWS OS: 9cefd83 #8084, #8516 build/win: support MSVC ee4e1fd win: Fix reading content from stdin (#8267) TUI: ffb8904 #8309 TUI: add support for mouse release events in urxvt 8d5a46e #8081 TUI: implement "standout" attribute 6071637 TUI: support TERM=konsole-256color 67848c0 #7653 TUI: report TUI info with -V3 ('verbose' >= 3) 3d0ee17 TUI/rxvt: enable focus-reporting d109f56 #7640 TUI: 'term' option: reflect effective terminal behavior FIXES: ed6a113 #8273 'job-control: avoid kill-timer race' 4e02f1a #8107 'jobs: separate process-group' 451c48a terminal: flush vterm output buffer on pty output #8486 5d6732f :checkhealth fixes #8335 53f11dc #8218 'Fix errors reported by PVS' d05712f inccommand: pause :terminal redraws (#8307) 51af911 inccommand: do not execute trailing commands #8256 84359a4 terminal: resize to the max dimensions (#8249) d49c1dd #8228 Make vim_fgets() return the same values as in Vim 60e96a4 screen: winhl=Normal:Background should not override syntax (#8093) 0c59ac1 #5908 'shada: Also save numbered marks' ba87a2c cscope: ignore EINTR while reading the prompt (#8079) b1412dc #7971 ':terminal Enter/Leave should not increment jumplist' 3a5721e TUI: libtermkey: force CSI driver for mouse input #7948 6ff13d7 #7720 TUI: faster startup 1c6e956 #7862 TUI: fix resize-related segfaults a58c909 #7676 TUI: always hide cursor when flushing, never flush buffers during unibilium output 303e1df #7624 TUI: disable BCE almost always 249bdb0 #7761 mark: Make sure that jumplist item will not have zero lnum 6f41ce0 #7704 macOS: Set $LANG based on the system locale a043899 #7633 'Retry fgets on EINTR' CHANGES: ad60927 #8304 default to 'nofsync' f3f1970 #8035 defaults: 'fillchars' a6052c7 #7984 defaults: sidescroll=1 b69fa86 #7888 defaults: enable cscopeverbose 7c4bb23 defaults: do :filetype stuff unless explicitly "off" 2aa308c #5658 'Apply :lmap in macros' 8ce6393 terminal: Leave 'relativenumber' alone (#8360) e46534b #4486 refactor: Remove maxmem, maxmemtot options 131aad9 win: defaults: 'shellcmdflag', 'shellxquote' #7343 c57d315 #8031 jobwait(): return -2 on interrupt also with timeout 6452831 clipboard: macOS: fallback to tmux if pbcopy is broken #7940 300d365 #7919 Make 'langnoremap' apply directly after a map ada1956 #7880 'lua/executor: Remove lightuserdata' INTERNAL: de0a954 #7806 internal statistics for list impl dee78a4 #7708 rewrite internal list impl

fmoralesc reviewed Mar 23, 2016
View reviewed changes

felipecrv force-pushed the rm-maxmem branch from 2497bde to 61e6bda Compare March 24, 2016 04:38

cjoach mentioned this pull request Mar 25, 2016

change numerical options types to NumOpt (alias intmax_t) #3914

Closed

justinmk reviewed Mar 26, 2016
View reviewed changes

felipecrv force-pushed the rm-maxmem branch 3 times, most recently from 315bce0 to bc90345 Compare March 29, 2016 20:39

oni-link reviewed Apr 4, 2016
View reviewed changes

felipecrv force-pushed the rm-maxmem branch from e6f022f to 1f54f85 Compare May 7, 2016 01:19

felipecrv force-pushed the rm-maxmem branch from 1f54f85 to bda62e2 Compare March 3, 2017 23:02

felipecrv added 2 commits April 11, 2017 00:33

Remove the mf_dont_release global after getting rid of maxmem[tot]

d0b08f3

felipecrv force-pushed the rm-maxmem branch from bda62e2 to d0b08f3 Compare April 10, 2017 22:33

felipecrv changed the title ~~Remove maxmem and maxmemtot options~~ [RDY] Remove maxmem and maxmemtot options Apr 10, 2017

marvim added the RDY label Apr 10, 2017

justinmk self-assigned this Jun 8, 2017

justinmk added this to the 0.3 milestone May 2, 2018

justinmk merged commit d0b08f3 into neovim:master May 2, 2018

justinmk removed the RDY label May 2, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[RDY] Remove maxmem and maxmemtot options #4486

[RDY] Remove maxmem and maxmemtot options #4486

felipecrv commented Mar 23, 2016 •

edited

Loading

fmoralesc Mar 23, 2016

felipecrv Mar 24, 2016

fmoralesc commented Mar 24, 2016

cjoach commented Mar 25, 2016

justinmk Mar 26, 2016

justinmk commented Mar 26, 2016

oni-link Apr 4, 2016

felipecrv Apr 4, 2016

felipecrv commented May 7, 2016

jamessan commented Dec 16, 2016

felipecrv commented Dec 16, 2016

felipecrv commented Mar 3, 2017

justinmk commented Mar 3, 2017

felipecrv commented May 2, 2018 via email

[RDY] Remove maxmem and maxmemtot options #4486

[RDY] Remove maxmem and maxmemtot options #4486

Conversation

felipecrv commented Mar 23, 2016 • edited Loading

fmoralesc Mar 23, 2016

Choose a reason for hiding this comment

felipecrv Mar 24, 2016

Choose a reason for hiding this comment

fmoralesc commented Mar 24, 2016

cjoach commented Mar 25, 2016

justinmk Mar 26, 2016

Choose a reason for hiding this comment

justinmk commented Mar 26, 2016

oni-link Apr 4, 2016

Choose a reason for hiding this comment

felipecrv Apr 4, 2016

Choose a reason for hiding this comment

felipecrv commented May 7, 2016

jamessan commented Dec 16, 2016

felipecrv commented Dec 16, 2016

felipecrv commented Mar 3, 2017

justinmk commented Mar 3, 2017

felipecrv commented May 2, 2018 via email

felipecrv commented Mar 23, 2016 •

edited

Loading