NAMED-CLOSURE - introspectable and redefinable closures

Why?

Hot take: closures in Common Lisp and most lisps are broken for long-running images. (Maybe more so in non-lisps, but who cares.)

Comparing to function symbols, closures are nearly unusable in the following aspects:

1. They’re hard to introspect, both from a user aspect and (even more so) programmatically. SBCL for example allows you to retrieve values in the flat closure, but do not save variable information.
2. As a consequence, they in general don’t have readable print syntax, and it might be impossible to write one. Function symbols on the other hand can be print and then read to get the exactly same object.
3. They’re near impossible to redefine. For a function symbol, setting its function cell causes all call site to consistently call the new definition. This is impossible for closures.

   Why redefining closure? It allows you to fix buggy code for closure (nobody always write correct code the first time!). Without redefinability you’ll have closure with wrong code floating around in the image that is almost impossible to fix, unless you remember all the location the closure is used and fix it manually – I find it much less pleasant and isn’t always possible.

Closures are still useful because:

1. Concise syntax.
2. They’re the lingua franca for a whole bunch of “functional programs”, which expect objects that are funcallable.

What

NAMED-CLOSURE provides DEFNCLO and NCLO.

• (nclo NAME LAMBDA-LIST . BODY) is similar to (lambda LAMBDA-LIST . BODY), but returns a funcallable object with slots corresponding to free variable in BODY, has readable print syntax, and if nclo with the same NAME is encountered (for example, if re-evaluated from REPL), the function definition of all such funcallable objects is updated. Closed variables with the same names are carried over across update.
• defnclo

  (defnclo something (lambda-list-1...)
      (lambda-list-2...)
    body...)
      

  is similar to

  (defun make-something (lambda-list-1...)
    (lambda (lambda-list-2...)
      body...))
      

  except that make-something now returns a funcallable object with slots corresponding to variables declared in lambda-list-1, has readable print syntax, and re-evaluating the defnclo updates the function definition of all such funcallable objects. Closed variables with the same names are carried over across update.

Note: newly introduced variables are unbound for updated old closures! This will likely cause an unbound-slot condition when such closure is called. You’re free to use the store-value restart your implementation (usually) provides to fix up the closure if possible. There isn’t anything more we can help :/

!!!Caveat!!!

Saying nclo is similar to lambda is a lie. Currently, nclo effectively copies the captured environment instead of directly link to it. See #1 to understand the important behavioral difference between nclo and lambda.

Excuses

It is possible to simulate the “correct” environment sharing behavior identical to lambda. I’m currently not doing it because

• It complicates introspection and proper readable printing
• It nukes upgradability. While it’s not unreasonable to ask user to fix up old closures using store-value, it sounds pretty impratical to expect user to fix the sharing structure between different closures.
• Closures are convenient. But remember we have real objects! If you need to rely on multiple closures sharing one environments, maybe it’s better to just use CLOS.

Example

(use-package :named-closure)
(defun make-inc-1 (x) (nclo inc (y) (setf x (+ x y))))
(defparameter *test-instance* (make-inc-1 5))
*test-instance* ; => #.(MAKE-INC :X 5)
(funcall *test-instance* 6) ; => 11
(funcall *test-instance* 6) ; => 17
(defun make-inc-1 (x) (nclo inc (y) (setf x (- x y)))) ; changed our mind!!!
(funcall *test-instance* 6) ; => 11
(funcall *test-instance* 6) ; => 5

p.s. I will probably ensure NAME-CLOSURE only ever exports obscure names, so it should be quite safe to use-package it!

How

Under the hood, defnclo defines a funcallable class named something, which in turn indirect calls through its class-allocated ~’named-closure::code~ slot so that it is redefinable.

nclo is implemented by walking BODY and collecting its free variables, then calling defnclo with the free variable list (with &key prepended) passed as LAMBDA-LIST-1.

Note: Because free variables are converted to keyword argument, their symbol-name must be distinct. Is this good enough?

There’s one subtlety involved with nclo: nclo usually appears as a non-top-level form, but it needs to ensure creating a top-level function definition for NAME in the runtime environment. We do this by abusing load-time-value.