Skip to content
Newer
Older
100644 174 lines (109 sloc) 5.75 KB
02949ea @dmiller Added some files with analysis of Compiler
dmiller authored Sep 7, 2009
1 Somewhere (far) below are some questions on internals of the Clojure compiler,
2 all related to one particular aspect of the compiler, but one that is ubiquitous.
3
4 I've been trying to analyze the context parameter used in parsing and emitting code.
5 The one of type C, where
6
7 public enum C{
8 STATEMENT, //value ignored
9 EXPRESSION, //value required
10 RETURN, //tail position relative to enclosing recur frame
11 EVAL
12 }
13
14 This parameter is passed to IParser.parse() and Expr.emit() and some others,
15 so it is everywhere.
16
17 The first three values are fairly obvious. The last is a bit more mysterious.
18 And the root call for parsing, to analyze, for both eval and compile pass C.EVAL.
19
20
21 I stepped through all the Compiler code, looking for how the context parameter is used.
22
23
24 Parsing
25 =======
26
27 Parsing subforms
28 ----------------
29
30 There are several idioms for how the context is passed on when parsing subforms:
31
32 (a) pass the current value through to the subform
33 (b) pass C.EXPRESSION to the subform -- i.e., the subform's value is required
34 (c) pass a value to the subform computed by: context == C.EVAL ? context ? C.EXPRESSION
35 in other words, pass along EVAL if that's the current context,
36 otherwise, we need a value (do not pass along STATEMENT or RETURN)
37 (d) ignore -- not used
38
39 There are two special cases:
40
41 (a) in BodyExpr, all subforms but the last are parsed with C.STATEMENT unless the
42 current context is C.EVAL (in which case use C.EVAL).
43 The last subform is parsed with the current context.
44 (b) The finally clause of a TryExpr is parsed with C.STATEMENT.
45
46
47 Parsing escape
48 --------------
49
50 There are several expression types where the form to be parsed wraps itself in
51 a no-arg function -- (fn [] form) -- and parses the resulting form instead.
52
53 Those are:
54
55 LetExpr: if ( context == C.EVAL || ( context == C.EXPR && isLoop ) ) ...
56 LetFnExpr: if ( context == C.EVAL ) ...
57 ThrowExpr: if ( context == c.EVAL ) ...
58 TryExpr: if ( context != c.RETURN ) ...
59
60
61 recur context
62 -------------
63
64 RecurExpr's parser throws an exception if the context is not C.RETURN.
65
66
67
68 Emitting code
69 =============
70
71 Emitting subforms
72 -----------------
73
74 There are several idioms for how the context is passed on to subforms when they are
75 asked to emit code.
76
77 (a) pass through the current context to subform emit calls
78 (c) pass c.EXPRESSION to subform emit calls
79 (a) ignored
80
81 Again, BodyExpr and TryExpr are the exceptions, just as in parsing:
82
83 (a) BodyExpr: all subforms but last are emitted with context C.STATEMENT,
84 the last form is emitted using the current context
85 (b) TryExpr: the finally clause is emitted with C.STATEMENT. Also, if the
86 current context is not C.STATEMENT, code is emitted to save and restore
87 the value from the body's evaluation, giving a value to be returned to the caller.
88
89
90 Popping the stack
91 -----------------
92
93 The context is used to determine when a value needs to be popped from the runtime
94 evaluation stack:
95
96 if(context == C.STATEMENT) gen.pop();
97
98
99
100 Clearing locals
101 ---------------
102
103 Finally the context is used to force locals to be cleared before a tail call:
104
105 if(context == C.RETURN)
106 {
107 FnMethod method = (FnMethod) METHOD.deref();
108 method.emitClearLocals(gen);
109 }
110
111
112 Getting started
113 ===============
114
115 When #eval is handed a form to evaluate, it does of several things (after macroexpansion):
116
117 (a) If the form is (do form-1 form-2 ... ), eval each form-i in turn and return
118 the value of the last one.
119
120 (b) If the form is an IPersistentCollection that does not begin with Symbol whose
121 name starts with "def", wrap the form in a no-arg function -- (fn [] form),
122 parse it starting with context C.EXPRESSION, then instantiate and eval the function.
123
124 (c) otherwise, parse it starting with context C.EVAL.
125
126
127 When #compile1 is handed a form to compile, it is similar, except case (b) is eliminated.
128
129
130 Questions
131 =========
132
133 The reason I'm so interested in this is that as currently written, the compiler
134 in ClojureCLR does not do _any_ of this. However, I have to add a little more
135 machinery to make sure that I catch recur calls occurring in the wrong places,
136 so I found analyzing the machinery in ClojureJVM that handles that.
137 And I've learned that I ignore things that Rich has done at my own peril.
138 So I'm trying to make sure I'm not missing some crucial aspect of the use of
139 Context in parsing forms and emitting code.
140
141
142 (0) What am I missing? (read that question any way you want)
143
144 (1) Is there a succinct description for the meaning of C.EVAL?
145
146 (2) Why are 'forms that of the form (defXXX ... )' singled out for special treatment
147 in eval?
148
149
150 (3) (T or F) The forms that wrap themselves as (fn [] form) do so because they need
151 to make sure they have a context for local variable mangling, closing over, etc.,
152 that is provided only by a FnExpr. (Those forms are let, letfn, throw, try.)
153
154
155 (4) Do we need an interpreted eval? Is there a problem with just wrapping any form
156 in an (fn [] form) and processing that?
157
158
159 (5) If there is no need for the three separate cases in eval, is there any need for the
160 information carried by a context of C.Eval?
161
162
163 (6) If we don't have to worry about popping values from the runtime stack where values
164 are being ignored, is there any need for the information carried by C.STATEMENT?
165 (In ClojureCLR, the DLR handles this automatically.)
166
167 And while I'm at it: C.RETURN is used for tail-call detection. Recur is obvious.
168 Not so obvious?
169
170
171 (7) Are there any succinct examples of the problem being solved by emitClearLocals?
172
173 (8) Why is emitClearLocals not called in more places? (It is called for InstanceMethodExpr,
174 InvokeExpr, NewExpr, and StaticMethodExpr.)
Something went wrong with that request. Please try again.