action upon failure of new() isn't specified

[gopherbot]

by JQBalter:

The language specification doesn't say what happens if new() fails to allocate
memory. Here are three ways to fix this:

1) Make new() return nil upon failure. This will introduce a bug into every current
call of new() (except those that checked new() for nil for no documented reason).
It also requires a coding style that is verbose and known to be error-prone. And it
wouldn't work well if nonnullable pointer types were ever added to the language.

2) Make new() return res, ok. Calls that don't check the second return value could
result in a fatal error if new() fails. This avoids introducing a bug into every current
call, but turns local failures into global ones -- not a good design for systems
programming or servers. This approach also requires a verbose coding style in 
code that checks for allocation failure (which rarely happens, discouraging
programmers from checking).

3) Introduce exceptions into the language and issue an exception if new() fails.

[rsc]

Comment 1:

4) Let new crash the program if it runs out of memory.

Owner changed to g...@golang.org.

Status changed to Thinking.

[gopherbot]

Comment 2 by JQBalter:

Yes, that's a fourth "fix". It turns local failures into global ones -- not a good
design for systems programming or servers. And, without some sort of hook to be
called upon failure, it provides no recourse at all, and leads to designers avoiding
large caches, sparse arrays, and other address-space intensive techniques. This is
not the sort of decision that language designers should make for their users, not if
they want their language to be taken seriously for systems programming.

[gopherbot]

Comment 3:

In many environments, malloc() will never fail -- you just eventually fault if the
system can't satisfy the memory requirements.  Linux with overcommit (default setup
for many (most?) distros) behaves like this.  Go's practice of zero-filling freshly
allocated objects may or may not end up hitting an immediate fault in new(),
depending how exactly the memory is allocated (if the heap sits on top of an
anonymous mmap'd zero'd region the fault may be deferred quite a ways).
In nontrivial applications and environments you may need more memory just to have the
resources to deal with fielding out of memory situations and somehow trying to
recover or inform the user -- which even if you reserve memory upfront you may not
end up with enough.  You quite possibly need to allocate to create the stack trace
for your exception or to field your exception, and if unwinding your stack doesn't
release enough memory, hm, throw the exception again?
Perhaps a way to register an observer with the heap/gc subsystem would allow one to
start taking corrective action once memory consumption passes a threshold.

[gopherbot]

Comment 4 by JQBalter:

Even with Linux overcommit, malloc() can fail if you exhaust your virtual address
space, which could be done with an erroneous size request. But we shouldn't be
talking about what Linux does because this is about a *language* feature, and
languages should not be dependent on a specific operating system, and certainly not a
"default setup" for that operating system (I myself have, on occasion, changed that
setting).
As for the point about needing memory to handle out-of-memory conditions, serious
systems programmers have been known to handle such things by taking care to static
allocate whatever is needed to handle the error. Again, at the language level one
should not make assumptions as to what the users of the language will do in such
circumstances.
As for registering an observer with the heap/gc subsystem, that's the sort of hook I
referred to, which would be fix number 5. It does however mean a global response to
what may be a local logic error or excessive allocation.

[rsc]

Comment 5:

Labels changed: added languagechange.

[robpike]

Comment 6:

This topic will likely be revisited at some point but until the memory management
implementation in Go is much 
more robust, there's little to do here since any fix requires significant changes to a
piece of the system that is 
going away soon.

Status changed to WontFix.

[gopherbot]

Comment 7 by JQBalter:

new() is going away? If not, that comment makes no sense -- or rather, it indicates
that its author doesn't understand the most important distinction in software
engineering -- that between specification and implementation.