Use system memory allocator on Windows (removes support for

--max-mem-size, R_MAX_MEM_SIZE, memory.limit() and memory.size()).


git-svn-id: https://svn.r-project.org/R/trunk@80832 00db46b3-68df-0310-9c12-caf00c1e9a41

doc/NEWS.Rd

@@ -60,6 +60,16 @@
       \code{diff(ylim)} is infinite.
 
       \item \code{nclass.FD()} gains a \code{digits} option.
+
+      \item \R on Windows uses again the system memory allocator. 
+      Previously, \R shipped with Doug Lea's allocator as a replacement to
+      overcome performance limitations seen with system allocators on
+      earlier versions of Windows.  Like on Unix-alike, once using the
+      system allocator, \R on Windows no longer provides
+      \code{memory.limit()}, \code{memory.size()}, command-line option
+      \option{--max-mem-size} and environment variable \env{R_MAX_MEM_SIZE}. 
+      The allocation limit was important on 32-bit Windows, but \R no longer
+      supports 32-bit builds.
     }
   }
 

doc/manual/R-intro.texi

@@ -7152,15 +7152,6 @@ locations.  This defaults to 10000, but can be increased to allow
 large and complicated calculations to be done.  Currently the maximum
 value accepted is 100000.
 
-@item --max-mem-size=@var{N}
-(Windows only) Specify a limit for the amount of memory to be used both
-for @R{} objects and working areas.  This is set by default to the
-smaller of the amount of physical RAM in the machine and for 32-bit
-@R{}, 1.5Gb@footnote{2.5Gb on versions of Windows that support 3Gb per
-process and have the support enabled: see the @file{rw-FAQ} Q2.9; 3.5Gb
-on most 64-bit versions of Windows.}, and must be between 32Mb and the
-maximum allowed on that version of Windows.
-
 @item --quiet
 @itemx --silent
 @itemx -q

doc/manual/R-ints.texi

@@ -1574,22 +1574,7 @@ override for @code{c}.
 @section Memory allocators
 
 @R{}'s memory allocation is almost all done via routines in file
-@file{src/main/memory.c}.  It is important to keep track of where memory
-is allocated, as the Windows port (by default) makes use of a memory
-allocator that differs from @code{malloc} etc as provided by MinGW.
-Specifically, there are entry points @code{Rm_malloc}, @code{Rm_free},
-@code{Rm_calloc} and @code{Rm_free} provided by file
-@file{src/gnuwin32/malloc.c}.  This was done for two reasons.  The
-primary motivation was performance: the allocator provided by MSVCRT
-@emph{via} MinGW was far too slow at handling the many small allocations
-that the allocation system for @code{SEXPREC}s uses.  As a side benefit,
-we can set a limit on the amount of allocated memory: this is useful as
-whereas Windows does provide virtual memory it is relatively far slower
-than many other @R{} platforms and so limiting @R{}'s use of swapping is
-highly advantageous.  The high-performance allocator is only called from
-@file{src/main/memory.c}, @file{src/main/regex.c}, @file{src/extra/pcre}
-and @file{src/extra/xdr}: note that this means that it is not used in
-packages.
+@file{src/main/memory.c}. 
 
 The rest of @R{} should where possible make use of the allocators made
 available by file @file{src/main/memory.c}, which are also the methods

doc/manual/rw-FAQ.texi

@@ -97,7 +97,6 @@ It is biased towards users of 64-bit Windows.
 * Can I run R from a CD or USB drive?::  
 * How do I UNinstall R?::       
 * What's the best way to upgrade?::  
-* There seems to be a limit on the memory it uses!::  
 * How can I keep workspaces for different projects in different directories?::  
 * How do I print from R?::      
 * Can I use R CMD BATCH?::      
@@ -186,7 +185,7 @@ starting @code{R i386} and one for 64-bit starting @code{R x64}
 
 You may also want to add command-line arguments at the end of the
 Target field (@emph{after} any final double quote, and separated by a
-space), for example @code{--sdi --max-mem-size=1G}.  You can also set
+space), for example @code{--sdi --no-environ}.  You can also set
 environment variables at the end of the Target field, for example
 @code{R_LIBS=p:/myRlib}, and if you want to ensure that menus and
 messages are in (American) English, @code{LANGUAGE=en}.
@@ -291,7 +290,7 @@ If all else fails, you can just delete the whole directory in which R
 was installed.
 
 
-@node What's the best way to upgrade?, There seems to be a limit on the memory it uses!, How do I UNinstall R?, Installation and Usage
+@node What's the best way to upgrade?, How do I UNinstall R?, Installation and Usage
 @section What's the best way to upgrade?
 
 That's a matter of taste.  For most people the best thing to do is to
@@ -309,39 +308,7 @@ changes (e.g.@: from 4.0.2 to 4.1.0).  A simple way to do so is to
 copy (say) @file{R\win-library\4.0} to @file{R\win-library\4.1} before
 running @code{update.packages(checkBuilt=TRUE, ask=FALSE)}.
 
-
-@node There seems to be a limit on the memory it uses!, How can I keep workspaces for different projects in different directories?, What's the best way to upgrade?, Installation and Usage
-@section There seems to be a limit on the memory it uses!
-
-Indeed there is.  It is set by the command-line flag
-@option{--max-mem-size} (@pxref{How do I install R for Windows?}) or by
-environment variable @env{R_MAX_MEM_SIZE}. 
-
-For a 64-bit build of R it defaults to the amount of RAM.
-
-For a 32-bit build of R it defaults to the smaller of the amount of
-physical RAM in the machine and 0.5GB less than the limit on user
-virtual memory for a process (most often 2GB when using a 32-bit edition
-of Windows).
-
-Use @code{?Memory} and @code{?memory.size} for information about memory
-usage.  The limit can be raised (if possible) by calling
-@code{memory.limit} within a running R session.
-
-The 32-bit executables support up to 3GB of user address space per
-process under suitably enabled versions of 32-bit Windows (see
-@uref{https://docs.microsoft.com/@/en-gb/@/windows/@/desktop/@/Memory/@/physical-address-extension},
-@uref{https://docs.microsoft.com/@/en-gb/@/windows/@/desktop/@/Memory/@/4-gigabyte-tuning};
-this is not enabled by default).  On such systems, the default for
-@code{--max-mem-size} is the smaller of the amount of RAM and 2.5GB.  On
-all but the earliest 64-bit versions of Windows the user address space
-for a 32-bit process is 4GB, and there the default for
-@code{--max-mem-size} is the smaller of the amount of RAM and 3.5GB.
-
-
-@c limits are documented at http://msdn.microsoft.com/en-us/library/aa366778%28VS.85%29.aspx
-
-@node How can I keep workspaces for different projects in different directories?, How do I print from R?, There seems to be a limit on the memory it uses!, Installation and Usage
+@node How can I keep workspaces for different projects in different directories?, How do I print from R?, Installation and Usage
 @section How can I keep workspaces for different projects in different directories?
 
 Create a separate shortcut for each project: see Q2.5.  All the paths to

src/extra/xdr/rpc/types.h

@@ -59,19 +59,11 @@
 #	define NULL 0
 #endif
 
-#if !defined(_WIN32) && !defined(macintosh) && !defined(__CYGWIN__)
+#if !defined(macintosh) && !defined(__CYGWIN__)
 #include <stdlib.h> // for malloc
 #endif
-#if defined(_WIN32)
-#include <stddef.h>
-extern void *Rm_malloc(size_t n);
-extern void Rm_free(void * p);
-#define mem_alloc	Rm_malloc
-#define mem_free(ptr, bsize)	Rm_free(ptr)
-#else
 #define mem_alloc	malloc
 #define mem_free(ptr, bsize)	free(ptr)
-#endif
 
 #ifndef makedev /* ie, we haven't already included it */
 #ifndef macintosh

src/gnuwin32/CHANGES2

@@ -215,16 +215,18 @@ Internal changes
 The fast bmp/png/jpeg code introduced in R 1.8.0 is used even for
 256-color displays (as we have now been able to test it on such).
 
-R's internal malloc etc are now remapped to Rm_malloc etc and only
-used in allocating memory for R objects, the Wilcoxon tests and a few
+Until R 4.2, R's internal malloc etc were remapped to Rm_malloc etc and
+(only) used in allocating memory for R objects, the Wilcoxon tests and a few
 other memory-intensive applications.
 
 Improved malloc routines from the current version of Doug Lea's malloc
-(as suggested by David Teller) should enable large memory areas to be
+(as suggested by David Teller) have enabled large memory areas to be
 used more effectively, in particular those over 2Gb where OS support
-has been enabled.  The initially requested memory is no longer
-reserved, but as this malloc is able to work with non-contiguous
-memory chunks that should not matter.
+were enabled. 
+
+The initially requested memory is no longer reserved, but as that and
+today's malloc is able to work with non-contiguous memory chunks that should
+not matter.
 
 The installer uses LZMA compression, so Inno Setup >= 4.1.5 is required.
 

src/gnuwin32/Makefile

@@ -77,7 +77,7 @@ endif
 
 dllversion-RESFLAGS = -I../include
 CSOURCES = \
-  console.c dynload.c editor.c embeddedR.c extra.c malloc.c \
+  console.c dynload.c editor.c embeddedR.c extra.c \
   opt.c pager.c preferences.c psignal.c rhome.c rt_complete.c \
   rui.c run.c shext.c sys-win32.c system.c dos_wglob.c
 OBJS = $(CSOURCES:.c=.o) dllversion.o
@@ -428,7 +428,6 @@ distribution:
 
 ## ============= End of maintainer targets ========================
 
-# Dependencies: malloc.c doesn't have any
 DEPS = $(CSOURCES:.c=.d)
 
 makeMakedeps: $(DEPS)

src/gnuwin32/Rdll.hide

@@ -225,7 +225,6 @@
  R_lazyLoadDBinsertValue
  R_localtime
  R_localtime_r
- R_max_memory
  R_max_num_math_threads
  R_mktime
  R_moduleCdynload
@@ -415,10 +414,6 @@
  RguiPackageMenu
  Rgui_Edit
  Rgui_configure
- Rm_calloc
- Rm_free
- Rm_malloc
- Rm_realloc
  Rsockselect
  Rstrlen
  Rstrwid
@@ -829,7 +824,6 @@
  do_memDecompress
  do_memlimits
  do_memoryprofile
- do_memsize
  do_merge
  do_mget
  do_missing
@@ -1310,7 +1304,6 @@
  lzma_vli_decode
  lzma_vli_encode
  lzma_vli_size
- mallinfo
  mb_char_len
  mbcsValid
  menuclear

src/gnuwin32/extra.c

@@ -284,60 +284,6 @@ void Rsleep(double timeint)
 
 }
 
-
-#define MALLINFO_FIELD_TYPE size_t
-struct mallinfo {
-    MALLINFO_FIELD_TYPE arena;    /* non-mmapped space allocated from system */
-    MALLINFO_FIELD_TYPE ordblks;  /* number of free chunks */
-    MALLINFO_FIELD_TYPE smblks;   /* number of fastbin blocks */
-    MALLINFO_FIELD_TYPE hblks;    /* number of mmapped regions */
-    MALLINFO_FIELD_TYPE hblkhd;   /* space in mmapped regions */
-    MALLINFO_FIELD_TYPE usmblks;  /* maximum total allocated space */
-    MALLINFO_FIELD_TYPE fsmblks;  /* space available in freed fastbin blocks */
-    MALLINFO_FIELD_TYPE uordblks; /* total allocated space */
-    MALLINFO_FIELD_TYPE fordblks; /* total free space */
-    MALLINFO_FIELD_TYPE keepcost; /* top-most, releasable (via malloc_trim) space */
-};
-extern R_size_t R_max_memory;
-
-struct mallinfo mallinfo(void);
-
-SEXP in_memsize(SEXP ssize)
-{
-    SEXP ans;
-    int maxmem = NA_LOGICAL;
-
-    if(isLogical(ssize)) 
-	maxmem = asLogical(ssize);
-    else if(isReal(ssize)) {
-	R_size_t newmax;
-	double mem = asReal(ssize);
-	if (!R_FINITE(mem))
-	    error(_("incorrect argument"));
-#ifndef _WIN64
-	if(mem >= 4096)
-	    error(_("don't be silly!: your machine has a 4Gb address limit"));
-#endif
-	newmax = mem * 1048576.0;
-	if (newmax < R_max_memory)
-	    warning(_("cannot decrease memory limit: ignored"));
-	else
-	    R_max_memory = newmax;
-    } else
-	error(_("incorrect argument"));
-
-    PROTECT(ans = allocVector(REALSXP, 1));
-    if(maxmem == NA_LOGICAL)
-	REAL(ans)[0] = R_max_memory;
-    else if(maxmem)
-	REAL(ans)[0] = mallinfo().usmblks;
-    else
-	REAL(ans)[0] = mallinfo().uordblks;
-    REAL(ans)[0] /= 1048576.0;
-    UNPROTECT(1);
-    return ans;
-}
-
 SEXP do_dllversion(SEXP call, SEXP op, SEXP args, SEXP rho)
 {
     SEXP path = R_NilValue, ans;