/
CHANGES.txt
3572 lines (2418 loc) · 119 KB
/
CHANGES.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
****************************************************************************
As of ANTLR 3.5.2, March 25 2014, we are no longer updating this file. Instead,
we are using the github release mechanism. For example, here is
3.5.2 release notes:
https://github.com/antlr/antlr3/releases/tag/3.5.2
****************************************************************************
ANTLR 3.5 Release
January 4, 2012
Terence Parr, parrt at cs usfca edu
ANTLR project lead and supreme dictator for life
University of San Francisco
CHANGES
January 4 2012 -- release 3.5
January 3, 2012
* Improve error reporting and recovery for STRAY_BRACKET, fixes antlr/antlr3#42
* Do not write output files if errors were reported, fixes antlr/antlr3#61
* Fix AST operator on labeled set of terminals e.g. x=(A|B)^
* Fix labeled set of terminals with rewrite e.g. x=(A|B) -> $x
December 1, 2012
* Added error msg for .. in parser
September 17, 2012
* Add Gokulakannan Somasundaram's C++ target based upon C target.
Use language=Cpp in options. It's a header-only library, runtime/Cpp/include,
so installation is not required.
September 16, 2012
* Python 3.3 target added by Benjamin Wolf based upon Python 2 target
https://github.com/antlr/antlr3/pull/23
September 15, 2012
* LookaheadStream bug fixes;
https://github.com/antlr/antlr3/pull/21
* Pulled "Fix Python handling of syntactic predicates"
https://github.com/antlr/antlr3/pull/33
July 15, 2012
* GUnit improvements
https://github.com/antlr/antlr3/pull/27
May 2012:
* ANTLR3 update of ObjC runtime to go with latest ST4-ObjC
https://github.com/antlr/antlr3/pull/17
August 9, 2012
* Provide Lexer get end of file method so people can override it.
November 25, 2011
* stderr not test correctly in gunit examineExecResult
October 27, 2011
* Dieter Habelitz reported bug in java code gen with synpreds. labels were
being carried from grammar into synpreds but weren't typed properly (they
were "void x=null;" for x=ruleref labels)
October 25, 2011
* (Sam) Rule.setOption didn't do memoize option right.
* (Sam) Fix labels in synpreds
* (Sam) Fix input index for NoViableAltException during inline prediction
* (Sam) Fix label aliasing errors in cases like (x=y|x=z)
August 10, 2011
* (Sam) fix stack overflow in semantic context analysis
July 30, 2011
* added range check to BaseTree.insertChild()
July 18, 2011 -- release 3.4
* Added tree method insertChild(int i, Object t).
July 14, 2011
* Added BaesTree.freshenParentAndChildIndexesDeeply() to recursively
walk tree and set ptrs.
July 6, 2011
* reset() for token stream didn't skip initial off-channel tokens.
July 5, 2011
* Sam fixes rare infinite loop upon error in nextToken().
* rewrites weren't pulled from syntactic predicates.
June 29, 2011
* Sam noticed CommonToken.getText() cached substring pulled from input, which
defeated purpose of pointing into input char array. Altered to not cache.
Should reduce memory footprint.
June 24, 2011
* empty alts with actions didn't have EPSILON dummy alt node.
June 19, 2011
* Udo noticed that Parser.getMissingSymbol() didn't set invented token's input
June 8, 2011
* Added inContext(String context) back into TreeParser.
April 21, 2011
* Updated for ST v4.0.2 (setting iterateAcrossValues = true as instance var)
* Needed throws condition for delegatedRules.
April 20, 2011 (Sam Harwell)
* Implement the 'throwsSpec' feature of parser rules for the Java target
* Improve algorithm for SemanticContext Boolean predicate reduction
April 13, 2011
* Unmangled region names in STViz hiearchy tree display.
* Removed conversion timeout thing again
April 11, 2011
* Added option -Xconversiontimeout back in. Turns out we hit NFA conversion
time landmine occasionally with huge grammars; fails over to backtracking
(if turned on) if it can't make DFA.
March 29 - April 10, 2011
* Uses ST v4 now!!! Massive change. Only updated Java target so far.
Ripped out ST v3 usage to remove indirect dependency on ANTLR v2.
March 28, 2011
* Sam Harwell ported all v2 grammars to v3!
March 3, 2011
* left-recursion pattern off until all targets catch up
* ANTLRCore.sti no longer used; removed from all targets.
* Adding type, text terminal options
* Replaced hetero arg with terminalOptions arg in all templates that reference hetero
(this is the class name / node type used for TOKEN<NODETYPE> references
in grammar). Widespread but trivial changes to all targets. hetero is
now terminalOptions.node. Can also get terminalOptions.type and
terminalOptions.text from dictionary of options.
* Fixed mispelling of license in BSD license headers
March 3, 2011
* Add tree, getTree() to ParserRuleReturnScope to do away with specific ones like:
public static class rewrite_template_args_return extends ParserRuleReturnScope {
CommonTree tree;
public Object getTree() { return tree; }
};
Removed these special classes if it's just AST; keep if they have defined "returns"
values
February 26, 2011
* All finally {} have comment now to suppress warning.
* removed ; from {;} blank method
* Added @SuppressWarnings({"all"}) to front of each
generated class.
* -print wasn't always showing ^ and ! in grammar
* Added java-left-recur/Java.g example.
* left-recursion pattern detection handles backtracking mode now
February 25, 2011
* -Xmaxinlinedfastates went to 60 from 10 for forcing prediction in left-
recursive expression rules to stay in rule; preds use a parameter.
* trees know where they came from now start..stop tokens; todo: use for better err handling.
* Got immediate left-recursion working for rules. Added TestLeftRecursion.java
February 21, 2011
* Fixed http://www.antlr.org/jira/browse/ANTLR-437 thanks to Vladislav Kuzkokov.
added unit test.
February 17, 2011
* Add -language L option to override language=L option in grammar. Same
grammar can yield multiple parsers in different languages now.
February 10, 2011
* Added method to generated Java code to return the array of delegates; e.g.,
import Sub1, Sub2;
yields:
public Parser[] getDelegates() {
return new Parser[] {gSub1, gSub2};
}
January 25, 2011
* Improve error messages for no viable alt parse exceptions
January 20, 2011
* TokenRewriteStream had dead code; also updated insertBefore followed by
replace or delete. If input is abc and I did insertBefore(2,"y"), where
'c' is index 2, then did delete of 2 previously defined functionality
was to ignore the insert. that's weird; fixed to keep insert. Also
Delete special case of replace (text==null):
D.i-j.u D.x-y.v | boundaries overlap => combine to max(min)..max(right)
December 12, 2010
* Send couldBacktrack now to enterDecision in debug protocol
December 4, 2010
* TreeWizard ctor needed a default tree adapator.
November 29, 2010 -- ANTLR v3.3
November 23, 2010
* CodeGenerator.loadLanguageTarget is now static and available to load
targets so we can ask them questions during analysis.
* Fixed and added unit test
http://www.antlr.org/jira/browse/ANTLR-370
http://www.antlr.org/jira/browse/ANTLR-375
November 23, 2010
* Added source name to syntax error msgs
October 20, 2010
Added boolean couldBacktrack to enterDecision in dbg interface. Breaks AW
interface and other tools! [BREAKS BACKWARD COMPATIBILITY]
October 17, 2010
* Missing -trace in help msg
November 22, 2010
* Added GrammarAST: public int getCharPositionInLine() { return getColumn()-1; }
and Grammar.getHasDelegates() for C# guys
October 16, 2010
* Doesn't write profile data to file anymore; emits decision data to stderr
October 14, 2010
* Make OrderedHashSet have deterministic iteration
July 20, 2010
* greedy=true option shuts off nondeterminism warning.
* code gen for AST and -profile didn't compile. had useless line:
proxy.setTreeAdaptor(adap);
July 17, 2010
* Removed conversion timeout failsafe; no longer needed.
* Stats updated to be correct for -report.
June 10, 2010
* added toArray in OrderedHashSet to make addAll calls get same order for DFA edges and possibly code gen in some areas.
June 5, 2010
* Added -Xsavelexer
May 24, 2010
* lexerStringRef was missing elementIndex attribute. i='import' didn't work
in lexer. Altered all target stg files. Set in codegen.g
* output=AST, rewrite=true for tree rewriters broken. nextNode for subtree
streams didn't dup node, it gave whole tree back.
March 17, 2010
* Added MachineProbe class to make it easier to highlight ambig paths in
grammar. More accurate than DecisionProbe; retrofitted from v4.
February 20, 2010
* added range to TokenStream and implementors:
/** How far ahead has the stream been asked to look? The return
* value is a valid index from 0..n-1.
*/
int range();
* added new method to get subset of tokens to buffered token streams:
public List get(int start, int stop);
February 15, 2010
* Refs to other tokens in a lexer rule didn't get its line/charpos right.
altered Java.stg.
January 31, 2010
* Creating token from another token didn't copy input stream in CommonToken.
makes sense to copy too; i don't think anybody relies on it being null after
a copy. We might want to know where token came from.
January 26, 2009
* TreeParser.getMissingSymbol() use CommonTree instead of using
adaptor.create()
December 8, 2009
* Instead of sharing Token.EOF_TOKEN, I'm now creating EOF tokens so I can set the char position for better error messages.
December 5, 2009
* Fixed bug in TreeVisitor when rewrites altered number of children. Thanks to Chris DiGiano.
* added new buffered on-demand streams: BufferedTokenStream. Renamed CommonTokenStream to LegacyCommonTokenStream and made new one as subclass of BufferedTokenStream.
November 3, 2009
* Added org.antlr.runtime.UnbufferedTokenStream. Was trivial and works!
November 1, 2009
* Couldn't properly reuse parser state; ctor reset the state; fixed.
Parser(TokenStream input, RecognizerSharedState state)
* LookaheadStream<T> used some hardcoded Object return types for LT, etc...
uses T now.
September 23, 2009 -- ANTLR v3.2
September 21, 2009 [Jim Idle]
* Added new options for tool invocation to control the points at which the code
generator tells the target code to use its equivalent of switch() instead of
inline ifs.
-Xmaxswitchcaselabels m don't generate switch() statements for dfas
bigger than m [300]
-Xminswitchalts m don't generate switch() statements for dfas smaller
than m [3]
* Upgraded -X help output to include new optins and provide the default
settings, as well as provide units for those settings that need them.
* Change the C Target to overide the deafults for the new settings to
generate the most optimizable C code from the modern C compiler point of
view. This is essentially to always use swtich statements unless there
is absolutely no other option. C defaults are to use 1 for minimum and
3000 for maximum number of alts that trigger switch(). This results in
object code that is 30% smaller and up to 20% faster.
April 23, 2009
* Added reset to TreeNodeStream interface.
April 22, 2009
* Fixed ANTLR-374. Was caused by moved of grammars. %foo() stuff didn't work
April 9, 2009
* .g and .g3 file extensions work again.
* introduced bug in 3.1.3: gives exception not error msg upon
missing .g file
March 26, 2009
* Made ctor in TreeRewriter and TreeFilter call this not super.
March 21, 2009
* Added ctor to RecognizerSharedState to allow cloning it.
March 17, 2009 -- ANTLR v3.1.3
* improved ANTLRv3.g to handle <...> element options
March 15, 2009
* Fixed ANTLR-389. Strip didn't ignore options in subrules; also seemed
to demand stdin.
March 15, 2009
* ANTLR always sorts incoming grammar list by dependency. For example,
If W.g depends on tokens from P.g then P.g is done first even if
W.g mentioned first on command line. It does not ignore any files you
specify the commandline. If you do *.g and that includes some
imported grammars, it will run antlr on them.
* -make option prevents ANTLR from running on P.g if P older than
generated files.
* Added org.antlr.tool.GrammarSpelunker to build a faster dependency
checker (what grammars depend on etc...). Totally independent of any
ANTLR code; easy to pull into other tools.
* Added org.antlr.misc.Graph, a general graph with nodes
containing an Object payload. It knows how to do a topological sort
on the nodes.
March 10, 2009
* Added associativity token option to support upcoming special expression
parsing. Added rule option strategy=precedence also
March 1, 2009
* Changed ANTLRWorks debug port from 49153 to 49100. Apparently we change the port in
ANTLRWorks to 49100 in 1.2 but forgot to do so in the ANTLR targets.
START CHANGES FOR TREE FILTER MODE (pulled from dev branch)
This feature will be announced in 3.2, but I am integrating from my development branch now into the mainline so target developers have a chance to implement. We might release 3.1.3 bug fix release before 3.2.
* CommonTreeNodeStream -> BufferedTreeNodeStream. Now,
CommonTreeNodeStream is completely unbuffered unless you are
backtracking. No longer making a list of all nodes before tree parsing.
* Added tree grammar filter=true mode.
Altered templates:
Java.stg: added filterMode to genericParser and treeParser.
This required a change to ANTLRCore.sti
Defined a default parameter in treeParser to set the superclass
to TreeFilter for tree grammar with filter=true. It sets
superclass to TreeRewriter if filter=true and output=AST.
Other them that, I only had to change ANTLR itself a little bit.
Made filter mode valid for tree grammars and have it automatically set
the necessary elements: @synpredgate, backtrack=true, rewrite=true
(if output=AST). Added error message for detecting conflicting
options.
* Added misc.FastQueue and TestFastQueue:
A queue that can dequeue and get(i) in O(1) and grow arbitrarily large.
A linked list is fast at dequeue but slow at get(i). An array is
the reverse. This is O(1) for both operations.
* Added tree.TreeIterator, a generator that walks a doubly linked tree.
The nodes must know what index they are. It's an Iterator but
remove() is not supported. Returns navigation nodes always:
UP, DOWN, EOF.
* Added misc.LookaheadStream: A lookahead queue that knows how
to mark/release locations in the buffer for backtracking purposes.
I hope to use for both tree nodes and tokens. Just implement
nextElement() to say how to get next node or token.
END CHANGES FOR TREE FILTER MODE
February 23, 2009 -- ANTLR v3.1.2
February 18, 2009
* Added org.antlr.tool.Strip (reads from file arg or stdin, emits to stdout)
to strip actions from a grammar.
February 4, 2009
* Added CommonTree.setUnknownTokenBoundaries(). Sometimes we build trees
in a grammar and some of the token boundaries are not set properly.
This only matters if you want to print out the original text associated
with a subtree. Check this out rule:
postfixExpression
: primary ('.'^ ID)*
;
For a.b.c, we get a '.' that does not have the token boundaries set.
ANTLR only sets token boundaries for subtrees returned from a rule.
SO, the overall '.' operator has the token boundaries set from 'a'
to 'c' tokens, but the lower '.' subtree does not get the boundaries
set (they are -1,-1). Calling setUnknownTokenBoundaries() on
the returned tree sets the boundaries appropriately according to the
children's token boundaries.
January 22, 2009
* fixed to be listeners.add(listener); in addListener() of DebugEventHub.java
January 20, 2009
* Removed runtime method: mismatch in BaseRecognizer and TreeParser. Seems
to be unused. Had to override method recoverFromMismatchedToken() in
TreeParser to get rid of single token insertion and deletion for
tree parsing because it makes no sense with all of the up-and-down nodes.
* Changed JIRA port number from 8888 to no port spec (aka port 80) and all
refs to it in this file.
* Changed BaseTree to Tree typecase in getChild and toStringTree() and
deleteChild() to make more generic.
December 16, 2008
* Added -verbose cmd-line option and turned off standard header
and list of read files. Silent now without -verbose.
November 24, 2008
* null-ptr protected getParent and a few others.
* Added new ctor to CommonTreeNodeStream for walking subtrees. Avoids
having to make new serialized stream as it can reuse overall node stream
buffer.
November 20, 2008
* Updated BaseTest to isolate tests better.
November 17, 2008
* BaseTreeAdaptor.getType() was hosed; always gave 0. Thanks to Sam Harwell.
November 8, 2008
* Added methods to BaseRecognizer:
public void setBacktrackingLevel(int n) { state.backtracking = n; }
/** Return whether or not a backtracking attempt failed. */
public boolean failed() { return state.failed; }
November 5, 2008
* Tweaked traceIn/Out to say "fail/succeeded"
* Bug in code gen for tree grammar wildcard list label x+=.
* Use of backtrack=true anywhere in grammar causes backtracking sensitive
code to be generated. Actions are gated etc... Previously, that only
happened when a syntactic predicate appeared in a DFA. But, we need
to gate actions when backtracking option is set even if no decision
is generated to support filtering of trees.
October 25, 2008
* Fixed debug event socket protocol to allow spaces in filenames.
* Added TreeVisitor and TreeVisitorAction to org.antlr.runtime.tree.
October 22, 2008
* Added inContext() to TreeParser. Very useful for predicating
tree grammar productions according to context (their parent list).
Added new TestTreeContext unit tests (15).
/** Check if current node in input has a context. Context means sequence
* of nodes towards root of tree. For example, you might say context
* is "MULT" which means my parent must be MULT. "CLASS VARDEF" says
* current node must be child of a VARDEF and whose parent is a CLASS node.
* You can use "..." to mean zero-or-more nodes. "METHOD ... VARDEF"
* means my parent is VARDEF and somewhere above that is a METHOD node.
* The first node in the context is not necessarily the root. The context
* matcher stops matching and returns true when it runs out of context.
* There is no way to force the first node to be the root.
*/
public boolean inContext(String context) {...}
* Added 3 methods to Tree interface [BREAKS BACKWARD COMPATIBILITY]
/** Is there is a node above with token type ttype? */
public boolean hasAncestor(int ttype);
/** Walk upwards and get first ancestor with this token type. */
public Tree getAncestor(int ttype);
/** Return a list of all ancestors of this node. The first node of
* list is the root and the last is the parent of this node.
*/
public List getAncestors();
October 21, 2008
* Updated unit tests to be correct for \uFFFE->\uFFFF change
* Made . in tree grammar look like ^(. .*) to analysis, though ^(. foo)
is illegal (can't have . at root). Wildcard is subtree or node.
Fixed bugs:
http://www.antlr.org/browse/ANTLR-248
http://www.antlr.org/browse/ANTLR-344
October 1, 2008 -- ANTLR v3.1.1
September 8, 2008
* Labels on tokens, rules carry into synpreds now so semantic predicates work.
This didn't work since labels were stripped in the synpred and they weren't
defined in the generated method.
a : x=A z=a {$x.text.equals($z.text)}? A
| y=A a A A
;
September 3, 2008
* Made a REV static variable in Tool so that we can change the rev for
daily builds.
* Made \uFFFF a valid character. Token types are 32-bit clean using -1
not 0x0000FFFF as -1 so it should be okay. Label.java:
public static final int MIN_CHAR_VALUE = '\u0000';
public static final int MAX_CHAR_VALUE = '\uFFFF';
August 30, 2008
* Changed messages in en.stg so that TOKEN_NONDETERMINISM correctly
indicates when actions hid semantic predicates.
August 15, 2008
* Tweaked build properties and build.xml
August 13, 2008
* Fixed ANTLR-314; 3.1 introduced a problem with list labels +=
August 12, 2008 -- ANTLR v3.1
* Added JavaScript target
August 7, 2008
* an NFA target of EOF predicate transition in DFA cause an exception in
getPredicatesPerNonDeterministicAlt().
* Kay Roepke found a nasty bug when debugging AST-constructing
composite recognizers. If the input state was null to the constructor,
super class constructor created a new parser state object.
Later, though we passed the argument state not this.state
to the delegate constructors, forcing them to share a different
state objects! Changed state to this.state in Dbg.stg constructors.
* Ack. messed up debug/AST. Have to set proxy's tree adaptor; it's
a circular ref. Just an ASTDbg.stg change.
August 4, 2008
* superClass works now for lexers
* Made Grammar.defineNamedAction propogate header actions down to all
delegates if root grammar; regardless of lexer/parser scope.
* Rejiggered AST templates to propogate changes to tree adaptor
for delegate grammars. Fixes ANTLR-302
August 4, 2008
* FOLLOW set computations altered constant FOLLOW bit sets.
* Added (...) are all predicate evaluations.
* Extra init code for tree parser nonrewrite mode removed.
* Added empty child list check in becomeRoot
August 3, 2008
* Was using RuleReturnScope not Rulename_return for list labels in tree
parser.
* Didn't set _last in tree parser for rule ref track stuff (rewrite=true)
August 2, 2008
* Benjamin found another rewrite engine bug.
July 30, 2008
* CommonTreeNodeStream / CommonTokenStream did not reset properly.
July 29, 2008
* Fixed another bug in TokenRewriteStream; didn't like inserts after end.
July 28, 2008
* Fixed bug in TokenRewriteStream.toString(start,stop); it ignored
parameters. ;)
July 17, 2008
* allow qualified type names in hetero <...> options like T<a.b.c.Node>
July 5, 2008
* treeLevel not set for setBlock alts; added unit test
July 3, 2008
* Fixed ANTLR-267. parse tree added nodes during backtracking and
cyclic DFAs. tracks hidden tokens too now. Added toInputString() to
get text back including hidden tokens. Shows <epsilon> for rules
that match nothing.
June 26, 2008
* Added gParent ptr that points to immediate parent grammar. E.g.,
// delegators
public MParser gM;
public M_S gS;
public M_S gParent = gS; // NEW
* Grammar imports didn't set all the delegate pointers...lots of imported
grammars would cause a null ptr exception. Fixes ANTLR-292.
June 25, 2008
* List labels in tree construction didn't always track the tree; sometimes
had a rule result structure.
June 4, 2008
* Improved unit testing so that each test suite executes and builds grammars
in a separate temporary directory. This means they can execute concurrently.
Also seem to be a problem with my class path during execution. Moved
tmpdir for ahead of standard CLASSPATH.
* By virtue of an improvement to StringTemplate, output newlines
in generated files should be normalized to whatever your host uses.
June 3, 2008
* Restrict legality of grammar options; for example you cannot use output option
in lexer anymore.
June 2, 2008
* Throw illegal arg exception upon invalid TokenRewriteStream ops. Rewrote
core of engine. Slightly different operation. Added many more unit tests.
3.1b1 - May 20, 2008
May 11, 2008
* rewrite=true, output=AST for tree grammar was not working. Altered trees were not
propagated back up the rule reference chain. Required a number of mods to
ASTTreeParser.stg. Added unit tests.
May 10, 2008
* [BACKWARD INCOMPATIBLE if you override match()]
I had turned off single token insertion and deletion because I could not figure
out how to work with trees and actions. Figure that out and so I turned it back on.
match() returns Object matched now (parser, tree parser) so we can set labels
on token refs properly after single token ins/del error recovery. Allows actions
and tree construction to proceed normally even though we recover in the middle of
an alternative. Added methods for conjuring up missing symbols: getMissingSymbol().
* refactored BaseRecognizer error handling routines
* Single token error recovery was not properly taking into consideration EOF.
* ANTLR no longer tries to recover in tree parsers inline using single node deletion or insertion; throw exception. Trees should be well formed as they are not created by users.
* Added empty constructors to the exception classes that did not have them so that ANTLRWorks can create the exceptions.
* Made debug tree adaptor deal with tokens conjured up during error recovery.
* Removed extra location() debug element that was emitted.
May 8, 2008
* ANTLR didn't update line/col to DFA map for AW.
May 6-7, 2008
* Insufficiently covered (with semantic predicates) alt warnings are now emitted before
nondeterminisms so it's clear the nondeterminism is a result of insufficient preds.
* Improved insufficiently covered alt warnings from:
warning(203): T.g:2:3: The following alternatives are insufficiently covered with predicates: 1
to:
warning(203): T.g:2:3: Input B is insufficiently covered with predicates at loca
tions: alt 1: line 3:15, alt 2: line 2:9
* Improved nondeterminism warning to have:
Semantic predicates were present but were hidden by actions.
parser grammar U;
a : (A B)? ;
b : X a {p1}? A B | Y a {a1} {p2}? A B | Z a ;
To create the prediction DFA for the optional sub rule in 'a', ANTLR must find all references to 'a' to determine what can follow. A B can follow 'a' in the first two alts rule 'b'. To resolve the conflict between matching A B immediately in the sub rule and exiting rule 'a' to match it in 'b', ANTLR looks for predicates. In this case, there are two predicates that indicate the semantic context in which the surrounding alternatives are valid. The problem is that one of the predicates is hidden by an action. It took me 1.5 days, but I've finally have gotten ANTLR to properly track the insufficiently covered alternatives. Further, I have gotten it to tell you precisely where the uncovered predicates are even if they are simply hidden by actions. I have also updated all of the nondeterminism warnings so that it tells you if there was a predicate but one hidden by an action (this could be a separate condition from insufficiently covered predicates). here are your messages from ANTLR:
ANTLR Parser Generator Version 3.1b1 (??) 1989-2007
warning(203): U.g:2:5: Input such as "A B" is insufficiently covered with predicates at locations: alt 2: line 3:38 at B
Semantic predicates were present but were hidden by actions.
warning(200): U.g:2:5: Decision can match input such as "A B" using multiple alternatives: 1, 2
As a result, alternative(s) 2 were disabled for that input
Semantic predicates were present but were hidden by actions.
* Fixed issue where
r41
: (INT -> INT) ( ('+' i=INT) -> ^($i $r41) )* ';'
;
still warned about $r41 being ambig.
* actions are now added to the NFA.
* Fixed ANTLR-222. ANTLR now ignores preds after actions.
May 5, 2008
* Fixed ANTLR-235 by backing out a change from 12-31-07.
* Fixed ANTLR-249; I include semantic context again in closure busy signal.
May 3, 2008
* Fixed ANTLR-208. Looks in library or in -o output path. antlr -o foo T.g U.g where U needs T.tokens won't work unless we look in foo too. fixed.
* Refactored assign.types.g to move methods to a class called AssignTokenTypesBehavior.
* Fixed ANTLR-207. Lexers importing vocabs didn't see ';'=4 type aliases in .tokens.
* Fixed ANTLR-228. Couldn't use wildcard in alts with AST rewrites.
May 2, 2008
* Fixed ANTLR-230; can use \' now in action.
* Scope attributes no longer have a stack depth check on front. If you ref $r::a when r has not invoked you, then you get an exception not a default value. Back to the way 3.0.1 worked.
* $channel was a global variable in 3.0.1 unlike $type which did not affect an invoking lexer rule. Now it's local too. Only $type and $channel are ever set with regularity. Setting those should not affect an invoking lexer rule as in the following should work:
X : ID WS? '=' ID ; // result is X on normal channel
WS : ' '+ {$channel = HIDDEN; } ;
STRING : '"' (ESC|.)* '"' ; // result is STRING not ESC
FLOAT : INT '.' INT? ; // should be FLOAT
INT : Digit+ ;
fragment
Digit : '0'..'9' ;
* Fixed bug in interpreter regarding (...)* loops
May 1, 2008
* Fixed ANTLR-202. These now give warnings about ambig ref to $a.
a : ID a -> $a | INT ;
and
a : A a {$a.text} | B ;
April 30, 2008
* Fixed ANTLR-237. updated -depend to know about imported grammars.
$ java org.antlr.Tool -depend -lib foo T.g
ANTLR Parser Generator Version 3.1b1 (??) 1989-2007
T.g: foo/Java.g
TParser.java : T.g
T.tokens : T.g
TLexer.java : T.g
T_Java : T.g
April 29, 2008
* Fixed ANTLR-217; scope A,B,C; didn't work
* Fixed ANTLR-224; ! or ^ on item in alt with rewrite gave exception
* Added token options to terminals: ID<node=V; foo="Big bob"> etc...
node is default so you can do ID<V> for hetero tree types. most common.
April 17, 2008
* Use default msg if unknown recog type in getErrorMessage():
String msg = e.getMessage();
April 14, 2008
* %x.y = foo; was not working in @members section
March 29, 2008
* Import couldn't handle A imports B imports C.
March 27, 2008
* Added get/setInputStream to Token interface and affected classes.
February 26, 2008
* made fillBuffer public in CommonTreeNodeStream so we can add trees
to stream for interactive interpreters.
February 14, 2008
* Fixed a bug in the code generation where tree level 0 was used
no matter what to rewrite trees in tree grammars. added unit test
* Fixed ANTLR-221. exceptions were generated when using
AST construction operators and no output=AST option.
February 13, 2008
* Improved error msgs for unreachable alts and tokens.
February 11-12, 2008
* Fixed ANTLR-219.
It looks like the AST construction code for sets was totally messed up.
This was for not only the new tree parser AST construction, but also
the regular tree construction for parsers. I had to introduce templates
in the ASTTreeParser.stg file to deal with this. added unit tests:
TestTreeGrammarRewriteAST.testSetMatchNoRewrite(),
testSetMatchNoRewriteLevel2(), testSetMatchNoRewriteLevel2Root().
Had to add template matchRuleBlockSet()
to differentiate between a regular set in one that is an entire rule.
If it is an entire rule, it has to set the return value, retval.tree.
* Fixed ANTLR-220.
Made TreeAdaptor dupNode and dupTree events emit debugging events
so AW could see tree node duplications.
February 4, 2008
* BACKWARD INCOMPATIBILITY
Added getSourceName to IntStream and TokenSource interfaces and also the
BaseRecognizer. Have to know where char come from for error messages.
Widespread change, but a trivial one.
January 17, 2008
* Interpreter throws FailedPredicateException now when it sees a predicate;
before it was silently failing. I'll make it work one of these days. ;)
January 12, 2008
* Copy ctor not copying start and stop for common token. Fixes ANTLR-212
* Removed single token insertion and deletion for tokens, sets.
Required a change to the code generation for matchSet() template
and a tweak inside the BaseRecognizer. To engage this again is easy,
just override mismatch() to call mismatchRecover(). I changed it to simply
throw an exception.
* Added syntaxError recognizer state var so you can easily tell if
a recognizer failed. Added getNumberOfSyntaxErrors() to recognizers.
* Added doc for the error node stuff:
http://www.antlr.org/wiki/display/ANTLR3/Tree+construction
* Fixed ANTLR-193
* Added recognizer methods to answer questions about current mismatched
token error. Useful now since i don't automatically recover inline
to such errors (I throw exception):
mismatchIsUnwantedToken(IntStream input, int ttype)
mismatchIsMissingToken(IntStream input, BitSet follow)
* Added UnwantedTokenException and MissingTokenException to make
match() problems more precise in case you want to catch differently.
Updated getErrorMessage() to be more precise. Says:
line 2:9 missing EQ at '0'
now instead of
line 2:9 mismatched input '0' expecting EQ
Input "x=9 9;" gives
line 3:8 extraneous input '9' expecting ';'