Skip to content
Newer
Older
100644 233 lines (194 sloc) 6.98 KB
a8ae56e @gregsh first cases
gregsh authored May 5, 2012
1 ----------------
2 I. General Notes
3 ================
31d54dc @gregsh HOWTO initial
gregsh authored May 5, 2012
4
a8ae56e @gregsh first cases
gregsh authored May 5, 2012
5 1. Writing a grammar doesn't mean the generated parser will work and produce nice AST.
6 The tricky part is to *tune* some raw grammar that *looks correct* into a *working* grammar i.e. the grammar that produces working parser.
7 But once you've mastered some basics the rest is as easy as combining different blocks into a working solution.
31d54dc @gregsh HOWTO initial
gregsh authored May 5, 2012
8
a8ae56e @gregsh first cases
gregsh authored May 5, 2012
9 2. While editing grammar it is better to think that you manipulate generated code on a higher level of abstraction.
10
11 3. Handwritten classes and generated classes should be in different source roots.
12
13
14 --------------------------
15 II. HOWTO: Generated Parser
16 ==========================
17
18 Parser Basics
19 -------------
20
21 Each rule is either matched or not so every BNF expression is a boolean expression.
22 **True** means some part of an input sequence is matched, **false** *always* means nothing is matched even if some part of the input was matched.
23 Here are some of the grammar-to-code mappings:
24
25 Sequence:
26 ````
27 // rule ::= part1 part2 part3
28 public boolean rule() {
29 <header>
30 boolean result = false;
31 result = part1();
32 result = result && part2();
33 result = result && part3();
34 if (!result) <rollback any state changes>
35 <footer>
36 return result;
37 }
38 ````
39
40 Ordered choice:
41 ````
42 // rule ::= part1 | part2 | part3
43 public boolean rule() {
44 <header>
45 boolean result = false;
46 result = part1();
47 if (!result) result = part2();
48 if (!result) result = part3();
49 if (!result) <rollback any state changes>
50 <footer>
51 return result;
52 }
53 ````
54
55 Zero-or-more construct:
56 ````
57 // rule ::= part *
58 public boolean rule() {
59 <header>
60 while (true) {
61 if (!part()) break;
62 }
63 <footer>
64 return true;
65 }
66 ````
67
68 *One-or-more*, *optional*, *and-predicate* and *not-predicate* constructs are implemented accordingly.
69 As you can see the generated code can be easily debugged as any handwritten code.
70 Attributes like *pin* and *recoverUntil*, rule modifiers add some lines to this general structure.
71
72
73 Making *recoverUntil* actually work
74 -----------------------------------
75
76 1. This attribute in most cases should be specified on a rule that is inside a loop.
77 2. That rule should always have *pin* attribute somewhere as well.
78 3. Predicate rule should look like *!( token_to_stop_at | rule_to_stop_at | ....) *
79
80 ````
81 script ::= statement *
82 private statement ::= select_statement | delete_statement | ... {recoverUntil="statement_recover"}
83 private statement_recover ::= !(';' | SELECT | DELETE | ...)
84 select_statement ::= SELECT ... {pin=1} // something has to be pinned!
85 // pin="SELECT" is a valid alternative
86 ````
87
88 When nothing helps: *external* rules
89 ------------------------------------
90
91 1. Sometimes it's easier to do something right in code.
92 2. Sometimes there's no way around some external dependency.
93 3. Sometimes we already have some logic implemented elsewhere.
94
95 ````
96 {
97 stubParserClass="com.sample.SampleParserUtil"
98 }
99 external my_external_rule ::= parseMyExternalRule false 10 // we can pass some extra parameters
100 // .. even other rules!
101 rule ::= part1 my_external_rule part3
102 // rule ::= part1 <<parseMyExternalRule true 5>> part3 // is a valid alternative
103 ````
104
105 ````
106 public class SampleParserUtil {
107 public static boolean parseMyExternalRule(PsiBuilder builder, int level, // required arguments
108 boolean extraArg1, int extraArg2) { // extra arguments
109 // do the work, ask the audience, phone a friend
110 }
111 }
112 ````
113
114
115 ----------------------------------------
116 III. HOWTO: Generated PSI Classes Hierarchy
117 ========================================
118
119 PSI Basics
120 ----------
121
122 1. Specify *private* attribute on any rule if you don't want it to be present in AST as early as possible. The first rule is implicitly *private*.
123 2. Make use of *extends* attribute to achieve two goals at once: make PSI hierarchy look nice and make AST shallow.
124
125 ````
126 {
127 extends(".*_expr")=expr // make AST for literal one level deep: FileNode/LiteralExpr
128 // otherwise it will look like: FileNode/Expr/LiteralExpr
129 // tokens
130 PLUS='+'
131 MINUS='-'
132 ...
133 }
134 expr ::= factor add_expr *
135 private factor ::= primary mul_expr * // we don't need this node in AST
136 private primary ::= literal_expr // .. and this as well
137 left add_expr ::= ('+'|'-') factor // if classic recursive descent is used without "left" rules
138 left mul_expr ::= ('*'|'/') primary // then the AST without "extends" will look even worse:
139 literal_expr ::= ... // FileNode/Expr/AddExpr/MulExpr/LiteralExpr
140 ````
141
142 Organize PSI using *fake* rules and user methods
143 ------------------------------------------------
144 ````
145 {
146 extends("(add|mul)_expr")=binary_expr // this attributes can be placed directly after rule
147 extends(".*_expr")=expr // .. but I prefer grammars less cluttered with alien gibberish
148 }
149
150 // won't be taken into account by parser
151 fake binary_expr ::= expr + {
152 methods=[
153 left="/expr[0]" // will be @NotNull as far as we have "+" in the expression
154 right="/expr[1]" // "expr" is the name of the auto-calculated child property (singular or list)
155 ]
156 }
157
158 expr ::= factor add_expr *
159 ... and the rest of "PSI Basics" example
160 ````
161
162 Will produce among other code:
163 ````
164 public interface BinaryExpr {
165 List<Expr> getExprList();
166 @NotNull
167 Expr getLeft();
168 @Nullable
169 Expr getRight();
170 }
171
172 public interface AddExpr extends BinaryExpr { ... }
173
174 public interface MulExpr extends BinaryExpr { ... }
175
176 // and PsiElementVisitor implementation
177 public class Visitor extends PsiElementVisitor {
178 ...
179 public visitAddExpr(AddExpr o) {
180 visitBinaryExpr(o);
181 }
182 public visitMulExpr(MulExpr o) {
183 visitBinaryExpr(o);
184 }
185 public visitBinaryExpr(BinaryExpr o) {
186 visitExpr(o);
187 }
188 ...
189 }
190 ````
191
192
193 Implement interface via implementation *mixin*
194 ---------------------------------------------
195 ````
196 {
197 mixin("my_named")="com.sample.psi.impl.MyNamedImplMixin"
198 }
199 my_named ::= part1 part2 part3
200 // my_named ::= part1 part2 part3 {mixin="com.sample.psi.impl.MyNamedImplMixin"} // is a valid alternative
201
202 ````
203
204 ````
205 public class MyNamedImplMixin extends MyNamed implements PsiNamedElement {
206 // methods from PsiNamedElement interface
207 @Nullable @NonNls String getName() { ... }
208 PsiElement setName(@NonNls @NotNull String name) throws IncorrectOperationException { ... }
209 }
210 ````
211
212 Implement interface via method injection
213 ----------------------------------------
214 ````
215 {
216 psiImplUtilClass="com.sample.SamplePsiImplUtil"
217 implements("my_named")="com.intellij.psi.PsiNamedElement"
218 }
219 my_named ::= part1 part2 part3 {
220 methods=[getName setName] // no need to specify arguments or return type
221 }
222 ````
223
224 ````
225 public class SamplePsiImplUtil {
226 // methods from PsiNamedElement interface with an extra MyName parameter
227 @Nullable @NonNls String getName(MyNamed o) { ... }
228 PsiElement setName(MyNamed o, @NonNls @NotNull String name) throws IncorrectOperationException { ... }
229 }
230 ````
231
232
233 ... to be continued
Something went wrong with that request. Please try again.