Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Provides DSSSL support in syntax-case #69

Open
wants to merge 6 commits into from

1 participant

@matthastie

I think there are three interesting implementation details worth highlighting, as I suspect that they might apply to the general implementation of DSSSL in the context of any syntax expander. Points 2 & 3 certainly weren't obvious to my untrained eye when I commenced the work, and so I hope this discussion may enlighten those who may want DSSSL in other syntax expander implementations.

  1. Implementation of formals that include specifically #!optional and #!rest are simple - the variables are alpha-converted, and the DSSSL directives are passed-through to Gambit, which provides native DSSSL support. For example,
> ((lambda (a #!optional (b 3) #!rest r) `(,a ,b ,r)) 1 4 5 6)
((lambda (%%a0 #!optional (%%b1 '3) #!rest %%r2) (list %%a0 %%b1 %%r2)) 1 4 5 6)
(1 4 (5 6))
> 

In this case, a b and r are alpha-converted to %%a0 %%b1 and %%r2 respectively. The main alteration psyntax needed to support this use case was the filtering of #!optional and #!rest from variable lists to avoid triggering lambda syntax errors. While this would appear a simple win, further complexity lurked in the implementation of default and keyword parameters.

  1. Default parameters are expressions! Thus it is possible for the mad Schemer to do things like this:
(let ((a 1))
  ((lambda (#!optional (b (begin 'gratuitous-side-effect-ftw (+ 1 a)))) b)))

In this case, the default parameter of b is expressed in terms of a inside a begin statement. In psyntax, a is alpha-converted in the let binding, and so each default expression also requires alpha-conversion. The output for this example thus looks like this:

((lambda (%%a5)
   ((lambda (#!optional (%%b6 (begin 'gratuitous-side-effect-ftw (+ 1 %%a5))))
      %%b6))) 1)
2
  1. Keyword parameters expose environment to the application. When one writes, for example:
(define foo
 (lambda (#!keyword a) a))

The binding a becomes exposed in the environment of the application as a keyword, so that one may write

(foo a: 1)

This 'injection of environment' is problematic for any syntax expander that alpha-converts symbols; the keyword arguments also need alpha-conversion for correctness. This means that all applications of the lambda definition must be cognizant of the converted symbol names. For foo above, alpha-conversion may yield:

(define %%foo23
 (lambda (#!keyword %%a24) %%a24))

(foo a: 1) ;; is now wrong, and now needs to be
(foo %%a24: 1)

When choosing the implementation provided, I contemplated three possible solutions:

a) Avoid alpha-conversion of keyword parameters. For example, emit something like:

(define %%foo23
 (lambda (#!keyword a) a))

While this solution is simple, I did fear that it would break syntax hygiene when keywords were used. I thus avoided it.

b) Extend the environment of applications to include the keyword variable alpha-conversions. I personally think this is the optimal choice for performance and correctness purposes. However, I confess that reading the psyntax codebase to make this change made my eyes bleed, I couldn't identify simple strategic edits that could achieve such a solution. Furthermore, I identified that unit compilation of any such solution would be difficult: it necessitates publishing the keyword environment for all keyworded lambda in a visit file, so that separate compilation units can be made aware of the alpha-conversions ... quite a nasty problem that I did not wish to solve. One additional detail to contemplate is the need to avoid alpha-conversion of keyword parameters in applications of native Gambit functions. With all this to consider, I opted for solution C.

c) Place the application environment within a non-alpha-converted closure within the emitted lambda construct. This yields a simple solution that preserves hygiene at the cost of performance. The emitted code looks like this (speed freaks should overt eyes to prevent foaming at mouth):

> ((lambda (#!key a) a) a: 1)
((lambda %%dsssl-args10
   (receive (%%a9) (apply (lambda (#!key a) (values a)) %%dsssl-args10)
      %%a9)) a: 1)
1
> 

Thus, in this solution all keyword parameters are alpha converted, and the original lambda expression is placed in the body of the receive block. I believe hygiene is preserved due to the evaluation of keywords in the apply, and client code to the syntax expander cannot dereference the generated %%dsssl-args10.

I've included some unit tests for this work in the pull request, with intent to provide reasonable coverage for DSSSL argument variations up to this level of complexity, with both #!rest and (<formals> . rest) variants covered:

> ((lambda (x #!optional o #!key a . r) `(,x ,o ,a ,r)) 4 1 a: 2 3 5 6)
((lambda %%dssl-args15
   (receive (%%x14 %%o13 %%a12 %%r11)
            (apply (lambda (x #!optional o #!key a . r) (values x o a r))
                   %%dssl-args15)
            (list %%x14 %%o13 %%a12 %%r11)))
 4 1 a: 2 3 5 6)
(4 1 2 (3 5 6))
> 

Kind regards,
Matt.

@matthastie

I've just discovered the pull request fails on this srfi-1 reference definition ... I'll add this to the tests, debug the issue, and update the pull request.

(define (foo a . b)
    #f)
*** ERROR IN ##main -- (Argument 1) PAIR expected
(car
 '#(syntax-object (#(#(source1) a (console) 852010) . #(#(source1) b (consol...
)

Actually, looks like all (define ...) forms have problems ... revisiting.

@matthastie

I've added a third commit that corrects the aforementioned problem with DSSSL and define. Unit tests are now hardened to include both coverage for both lambda and define use cases.

bash-3.2$ pwd
/Volumes/Data/Users/matthastie/scratch/scheme/gambit-mdh/lib
bash-3.2$ ../gsi/gsi
Gambit v4.7.1

> (load "boot")
"/Volumes/Data/scratch/scheme/gambit-mdh/lib/boot.scm"
> (load "test-syntax-case-dsssl")
#(((lambda () 1)) -> 1)
#(((lambda l l) 1 2 3) -> (1 2 3))
#(((lambda (a) a) 1) -> 1)
#(((lambda (a . r) `(,a ,r)) 1 2 3) -> (1 (2 3)))
#(((lambda (a b) `(,a ,b)) 1 2) -> (1 2))
#(((lambda (#!optional a) a)) -> #f)
#(((lambda (#!optional a) a) 1) -> 1)
#(((lambda (a #!optional b) `(,a ,b)) 1) -> (1 #f))
#(((lambda (a #!optional b) `(,a ,b)) 1 2) -> (1 2))
#(((lambda (a #!optional (b 3)) `(,a ,b)) 1) -> (1 3))
#(((lambda (a #!optional (b 3)) `(,a ,b)) 1 4) -> (1 4))
#(((lambda (a #!optional b #!rest r) `(,a ,b ,r)) 1) -> (1 #f ()))
#(((lambda (a #!optional b . r) `(,a ,b ,r)) 1) -> (1 #f ()))
#(((lambda (a #!optional (b 3) #!rest r) `(,a ,b ,r)) 1) -> (1 3 ()))
#(((lambda (a #!optional (b 3) . r) `(,a ,b ,r)) 1) -> (1 3 ()))
#(((lambda (a #!optional (b 3) #!rest r) `(,a ,b ,r)) 1 5) -> (1 5 ()))
#(((lambda (a #!optional (b 3) . r) `(,a ,b ,r)) 1 5) -> (1 5 ()))
#(((lambda (a #!optional (b 3) #!rest r) `(,a ,b ,r)) 1 5 7) -> (1 5 (7)))
#(((lambda (a #!optional (b 3) . r) `(,a ,b ,r)) 1 5 7) -> (1 5 (7)))
#(((lambda (#!key a) a)) -> #f)
#(((lambda (#!key (a 4)) a)) -> 4)
#(((lambda (#!key a) a) a: 1) -> 1)
#(((lambda (#!key (a 5)) a) a: 1) -> 1)
#(((lambda (x #!key a) `(,x ,a)) 1) -> (1 #f))
#(((lambda (x #!key a) `(,x ,a)) 1 a: 3) -> (1 3))
#(((lambda (x #!key (a 4)) `(,x ,a)) 1) -> (1 4))
#(((lambda (x #!key (a 4)) `(,x ,a)) 1 a: 2) -> (1 2))
#(((lambda (x #!optional o #!key a) `(,x ,o ,a)) 1) -> (1 #f #f))
#(((lambda (x #!optional o #!key a #!rest r) `(,x ,o ,a ,r)) 1 2 3 4)
  ->
  (1 2 #f (3 4)))
#(((lambda (x #!optional o #!key a . r) `(,x ,o ,a ,r)) 1 2 3 4)
  ->
  (1 2 #f (3 4)))
#(((lambda (x #!optional o #!key a #!rest r) `(,x ,o ,a ,r)) 1 2 a: 3 4)
  ->
  (1 2 3 (4)))
#(((lambda (x #!optional o #!key a . r) `(,x ,o ,a ,r)) 1 2 a: 3 4)
  ->
  (1 2 3 (4)))
#(((lambda (x #!optional o #!key (a 3) #!rest r) `(,x ,o ,a ,r)) 1 2 4)
  ->
  (1 2 3 (4)))
#(((lambda (x #!optional o #!key (a 3) . r) `(,x ,o ,a ,r)) 1 2 4)
  ->
  (1 2 3 (4)))
#(((lambda (x #!optional o #!key (a 3) #!rest r) `(,x ,o ,a ,r)) 1 2 a: 4 4)
  ->
  (1 2 4 (4)))
#(((lambda (x #!optional o #!key (a 3) . r) `(,x ,o ,a ,r)) 1 2 a: 4 4)
  ->
  (1 2 4 (4)))
pass
"/Volumes/Data/scratch/scheme/gambit-mdh/lib/test-syntax-case-dsssl.scm"
> ,q
bash-3.2$ 

This additional commit also alters the psyntax implementation to use Gambit ##structure instead of vector for all structures. Testing for this change at the moment is somewhat light, but it does bootstrap and run the provided DSSSL tests correctly. It is also capable of loading the module.scm file in my scsc github repository, but I'm finding issues with the import of srfi-1.

> (import srfi/1)
*** ERROR -- (Argument 1) VECTOR expected
(vector-ref
 '#<syntax-object #2 expression: #(#(source1) (#(#(source1) include "/Volume...
 1)
> 

I'll try to track down this failure next, and get this corrected before proposing application of this push request.

I am, however, somewhat excited by the move to gambit ##structure and use of 'serialize in the output of syntax-case.scm - there's a substantial reduction in file size, and the loading of syntax-case is also faster:

bash-3.2$ # before
bash-3.2$ ls -l syntax-case.scm 
-rw-r--r--  1 matthastie  staff  630901 Mar  5 22:33 syntax-case.scm
bash-3.2$ # after
bash-3.2$ ../gsi/gsi boot
bash-3.2$ ls -l syntax-case.scm 
-rw-r--r--  1 matthastie  staff  234533 Mar  5 22:38 syntax-case.scm
bash-3.2$ 

A downside to using 'serialize on structure sharing is that syntax-case.scm needs to be loaded with the reader state altered, I parameterize current-readtable in boot.scm accordingly:

(parameterize ((current-readtable (readtable-sharing-allowed?-set (current-readtable) 'serialize)))
    (load "syntax-case"))

Additionally, I'm not certain of the prevalence of quoted syntax objects in generated source at this time ... I'll post more detail as I find out more.

@matthastie

The VECTOR expected error message found earlier was tracked to a defective implementation of import. A fourth commit is provided herein that corrects this, as shown:

bash-3.2$  ../gsi/gsi
Gambit v4.7.1

> (parameterize ((current-readtable (readtable-sharing-allowed?-set (current-readtable) 'serialize)))
    (load "syntax-case"))
"/Volumes/Data/scratch/scheme/gambit-mdh/lib/syntax-case.scm"
> (include "hello.scm")
*** ERROR -- (Argument 1) VECTOR expected
(vector-ref
 '#<syntax-object #2 expression: #(#(source1) (#(#(source1) include (console...
 1)
> ,q
bash-3.2$ ../gsi/gsi
Gambit v4.7.1

> (load "boot")
"/Volumes/Data/scratch/scheme/gambit-mdh/lib/boot.scm"
> (include "hello.scm")
hello, world.
> ,q
bash-3.2$ 

Further integration work is still needed to make gsi -:s load syntax-case with readtable-sharing-allowed?. I'll get this corrected next.

bash-3.2$ bin/gsi -:s
*** ERROR IN "lib/syntax-case.scm"@1.47098 -- Incomplete form
bash-3.2$ 
matthastie added some commits
@matthastie matthastie provide syntax-case support for dsssl d686850
@matthastie matthastie fix dsssl misspellings 5eb7122
@matthastie matthastie repair broken dsssl / use native gambit structs / new bootloader 1497d0c
@matthastie matthastie corrects defective (include <file>) implementation af8c4b4
@matthastie matthastie correct -:s behavior
parameterize reader to read syntax-case with shared structure
aa3ab83
@matthastie matthastie improved syntax errors
e4bff50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Apr 23, 2015
  1. @matthastie
  2. @matthastie

    fix dsssl misspellings

    matthastie authored
  3. @matthastie
  4. @matthastie
  5. @matthastie

    correct -:s behavior

    matthastie authored
    parameterize reader to read syntax-case with shared structure
  6. @matthastie

    improved syntax errors

    matthastie authored
Something went wrong with that request. Please try again.