The {@link RecognitionException} is non-null for all syntax errors except - * when we discover mismatched token errors that we can recover from - * in-line, without returning from the surrounding rule (via the single - * token insertion and deletion mechanism).
- * - * @param recognizer - * What parser got the error. From this - * object, you can access the context as well - * as the input stream. - * @param offendingSymbol - * The offending token in the input token - * stream, unless recognizer is a lexer (then it's null). If - * no viable alternative error, {@code e} has token at which we - * started production for the decision. - * @param line - * The line number in the input where the error occurred. - * @param charPositionInLine - * The character position within that line where the error occurred. - * @param msg - * The message to emit. - * @param e - * The exception generated by the parser that led to - * the reporting of an error. It is null in the case where - * the parser was able to recover in line without exiting the - * surrounding rule. - */ - public void syntaxError(Recognizer recognizer, - Object offendingSymbol, - int line, - int charPositionInLine, - String msg, - RecognitionException e); - - /** - * This method is called by the parser when a full-context prediction - * results in an ambiguity. - * - *Each full-context prediction which does not result in a syntax error - * will call either {@link #reportContextSensitivity} or - * {@link #reportAmbiguity}.
- * - *When {@code ambigAlts} is not null, it contains the set of potentially - * viable alternatives identified by the prediction algorithm. When - * {@code ambigAlts} is null, use {@link ATNConfigSet#getAlts} to obtain the - * represented alternatives from the {@code configs} argument.
- * - *When {@code exact} is {@code true}, all of the potentially - * viable alternatives are truly viable, i.e. this is reporting an exact - * ambiguity. When {@code exact} is {@code false}, at least two of - * the potentially viable alternatives are viable for the current input, but - * the prediction algorithm terminated as soon as it determined that at - * least the minimum potentially viable alternative is truly - * viable.
- * - *When the {@link PredictionMode#LL_EXACT_AMBIG_DETECTION} prediction - * mode is used, the parser is required to identify exact ambiguities so - * {@code exact} will always be {@code true}.
- * - *This method is not used by lexers.
- * - * @param recognizer the parser instance - * @param dfa the DFA for the current decision - * @param startIndex the input index where the decision started - * @param stopIndex the input input where the ambiguity was identified - * @param exact {@code true} if the ambiguity is exactly known, otherwise - * {@code false}. This is always {@code true} when - * {@link PredictionMode#LL_EXACT_AMBIG_DETECTION} is used. - * @param ambigAlts the potentially ambiguous alternatives, or {@code null} - * to indicate that the potentially ambiguous alternatives are the complete - * set of represented alternatives in {@code configs} - * @param configs the ATN configuration set where the ambiguity was - * identified - */ - void reportAmbiguity(Parser recognizer, - DFA dfa, - int startIndex, - int stopIndex, - boolean exact, - BitSet ambigAlts, - ATNConfigSet configs); - - /** - * This method is called when an SLL conflict occurs and the parser is about - * to use the full context information to make an LL decision. - * - *If one or more configurations in {@code configs} contains a semantic - * predicate, the predicates are evaluated before this method is called. The - * subset of alternatives which are still viable after predicates are - * evaluated is reported in {@code conflictingAlts}.
- * - *This method is not used by lexers.
- * - * @param recognizer the parser instance - * @param dfa the DFA for the current decision - * @param startIndex the input index where the decision started - * @param stopIndex the input index where the SLL conflict occurred - * @param conflictingAlts The specific conflicting alternatives. If this is - * {@code null}, the conflicting alternatives are all alternatives - * represented in {@code configs}. At the moment, conflictingAlts is non-null - * (for the reference implementation, but Sam's optimized version can see this - * as null). - * @param configs the ATN configuration set where the SLL conflict was - * detected - */ - void reportAttemptingFullContext(Parser recognizer, - DFA dfa, - int startIndex, - int stopIndex, - BitSet conflictingAlts, - ATNConfigSet configs); - - /** - * This method is called by the parser when a full-context prediction has a - * unique result. - * - *Each full-context prediction which does not result in a syntax error - * will call either {@link #reportContextSensitivity} or - * {@link #reportAmbiguity}.
- * - *For prediction implementations that only evaluate full-context - * predictions when an SLL conflict is found (including the default - * {@link ParserATNSimulator} implementation), this method reports cases - * where SLL conflicts were resolved to unique full-context predictions, - * i.e. the decision was context-sensitive. This report does not necessarily - * indicate a problem, and it may appear even in completely unambiguous - * grammars.
- * - *{@code configs} may have more than one represented alternative if the - * full-context prediction algorithm does not evaluate predicates before - * beginning the full-context prediction. In all cases, the final prediction - * is passed as the {@code prediction} argument.
- * - *Note that the definition of "context sensitivity" in this method - * differs from the concept in {@link DecisionInfo#contextSensitivities}. - * This method reports all instances where an SLL conflict occurred but LL - * parsing produced a unique result, whether or not that unique result - * matches the minimum alternative in the SLL conflicting set.
- * - *This method is not used by lexers.
- * - * @param recognizer the parser instance - * @param dfa the DFA for the current decision - * @param startIndex the input index where the decision started - * @param stopIndex the input index where the context sensitivity was - * finally determined - * @param prediction the unambiguous result of the full-context prediction - * @param configs the ATN configuration set where the unambiguous prediction - * was determined - */ - void reportContextSensitivity(Parser recognizer, - DFA dfa, - int startIndex, - int stopIndex, - int prediction, - ATNConfigSet configs); -} diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/ANTLRErrorStrategy.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/ANTLRErrorStrategy.java deleted file mode 100644 index 5f5321556..000000000 --- a/runtime/Java/src/main/java/org/antlr/v5/runtime/ANTLRErrorStrategy.java +++ /dev/null @@ -1,120 +0,0 @@ -/* - * Copyright (c) 2012-present The ANTLR Project. All rights reserved. - * Use of this file is governed by the BSD 3-clause license that - * can be found in the LICENSE.txt file in the project root. - */ - -package org.antlr.v5.runtime; - -import org.antlr.v5.runtime.tree.ErrorNode; - -/** - * The interface for defining strategies to deal with syntax errors encountered - * during a parse by ANTLR-generated parsers. We distinguish between three - * different kinds of errors: - * - *TODO: what to do about lexers
- */ -public interface ANTLRErrorStrategy { - /** - * Reset the error handler state for the specified {@code recognizer}. - * @param recognizer the parser instance - */ - void reset(Parser recognizer); - - /** - * This method is called when an unexpected symbol is encountered during an - * inline match operation, such as {@link Parser#match}. If the error - * strategy successfully recovers from the match failure, this method - * returns the {@link Token} instance which should be treated as the - * successful result of the match. - * - *This method handles the consumption of any tokens - the caller should - * not call {@link Parser#consume} after a successful recovery.
- * - *Note that the calling code will not report an error if this method - * returns successfully. The error strategy implementation is responsible - * for calling {@link Parser#notifyErrorListeners} as appropriate.
- * - * @param recognizer the parser instance - * @throws RecognitionException if the error strategy was not able to - * recover from the unexpected input symbol - */ - Token recoverInline(Parser recognizer) throws RecognitionException; - - /** - * This method is called to recover from exception {@code e}. This method is - * called after {@link #reportError} by the default exception handler - * generated for a rule method. - * - * @see #reportError - * - * @param recognizer the parser instance - * @param e the recognition exception to recover from - * @throws RecognitionException if the error strategy could not recover from - * the recognition exception - */ - void recover(Parser recognizer, RecognitionException e) throws RecognitionException; - - /** - * This method provides the error handler with an opportunity to handle - * syntactic or semantic errors in the input stream before they result in a - * {@link RecognitionException}. - * - *The generated code currently contains calls to {@link #sync} after - * entering the decision state of a closure block ({@code (...)*} or - * {@code (...)+}).
- * - *For an implementation based on Jim Idle's "magic sync" mechanism, see - * {@link DefaultErrorStrategy#sync}.
- * - * @see DefaultErrorStrategy#sync - * - * @param recognizer the parser instance - * @throws RecognitionException if an error is detected by the error - * strategy but cannot be automatically recovered at the current state in - * the parsing process - */ - void sync(Parser recognizer) throws RecognitionException; - - /** - * Tests whether or not {@code recognizer} is in the process of recovering - * from an error. In error recovery mode, {@link Parser#consume} adds - * symbols to the parse tree by calling - * {@link Parser#createErrorNode(ParserRuleContext, Token)} then - * {@link ParserRuleContext#addErrorNode(ErrorNode)} instead of - * {@link Parser#createTerminalNode(ParserRuleContext, Token)}. - * - * @param recognizer the parser instance - * @return {@code true} if the parser is currently recovering from a parse - * error, otherwise {@code false} - */ - boolean inErrorRecoveryMode(Parser recognizer); - - /** - * This method is called by when the parser successfully matches an input - * symbol. - * - * @param recognizer the parser instance - */ - void reportMatch(Parser recognizer); - - /** - * Report any kind of {@link RecognitionException}. This method is called by - * the default exception handler generated for a rule method. - * - * @param recognizer the parser instance - * @param e the recognition exception to report - */ - void reportError(Parser recognizer, RecognitionException e); -} diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/BailErrorStrategy.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/BailErrorStrategy.java index e977e19b1..9cf1fca87 100644 --- a/runtime/Java/src/main/java/org/antlr/v5/runtime/BailErrorStrategy.java +++ b/runtime/Java/src/main/java/org/antlr/v5/runtime/BailErrorStrategy.java @@ -6,13 +6,20 @@ package org.antlr.v5.runtime; -import org.antlr.v5.runtime.misc.ParseCancellationException; +import org.antlr.v5.runtime.core.Parser; +import org.antlr.v5.runtime.core.Token; +import org.antlr.v5.runtime.core.context.ParserRuleContext; +import org.antlr.v5.runtime.core.error.ANTLRErrorStrategy; +import org.antlr.v5.runtime.core.error.DefaultErrorStrategy; +import org.antlr.v5.runtime.core.error.InputMismatchException; +import org.antlr.v5.runtime.core.error.RecognitionException; +import org.antlr.v5.runtime.core.misc.ParseCancellationException; /** * This implementation of {@link ANTLRErrorStrategy} responds to syntax errors * by immediately canceling the parse operation with a * {@link ParseCancellationException}. The implementation ensures that the - * {@link ParserRuleContext#exception} field is set for all parse tree nodes + * {@link ParserRuleContext#getException} field is set for all parse tree nodes * that were not completed prior to encountering the error. * *@@ -44,8 +51,8 @@ public class BailErrorStrategy extends DefaultErrorStrategy { */ @Override public void recover(Parser recognizer, RecognitionException e) { - for (ParserRuleContext context = recognizer.getContext(); context != null; context = context.getParent()) { - context.exception = e; + for (ParserRuleContext context = recognizer.getContext(); context != null; context = (ParserRuleContext) context.getParent()) { + context.setException(e); } throw new ParseCancellationException(e); @@ -59,8 +66,8 @@ public Token recoverInline(Parser recognizer) throws RecognitionException { InputMismatchException e = new InputMismatchException(recognizer); - for (ParserRuleContext context = recognizer.getContext(); context != null; context = context.getParent()) { - context.exception = e; + for (ParserRuleContext context = recognizer.getContext(); context != null; context = (ParserRuleContext) context.getParent()) { + context.setException(e); } throw new ParseCancellationException(e); diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/BaseErrorListener.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/BaseErrorListener.java index 23fb70b3a..85babb76a 100644 --- a/runtime/Java/src/main/java/org/antlr/v5/runtime/BaseErrorListener.java +++ b/runtime/Java/src/main/java/org/antlr/v5/runtime/BaseErrorListener.java @@ -5,8 +5,13 @@ */ package org.antlr.v5.runtime; -import org.antlr.v5.runtime.atn.ATNConfigSet; -import org.antlr.v5.runtime.dfa.DFA; +import org.antlr.v5.runtime.core.atn.ATNConfigSet; +import org.antlr.v5.runtime.core.Parser; +import org.antlr.v5.runtime.core.Recognizer; +import org.antlr.v5.runtime.core.error.ANTLRErrorListener; +import org.antlr.v5.runtime.core.error.RecognitionException; +import org.antlr.v5.runtime.core.dfa.DFA; +import org.jetbrains.annotations.NotNull; import java.util.BitSet; @@ -19,7 +24,7 @@ */ public class BaseErrorListener implements ANTLRErrorListener { @Override - public void syntaxError(Recognizer recognizer, + public void syntaxError(@NotNull Recognizer recognizer, Object offendingSymbol, int line, int charPositionInLine, diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/BufferedTokenStream.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/BufferedTokenStream.java deleted file mode 100644 index 4e066605b..000000000 --- a/runtime/Java/src/main/java/org/antlr/v5/runtime/BufferedTokenStream.java +++ /dev/null @@ -1,491 +0,0 @@ -/* - * Copyright (c) 2012-present The ANTLR Project. All rights reserved. - * Use of this file is governed by the BSD 3-clause license that - * can be found in the LICENSE.txt file in the project root. - */ - -package org.antlr.v5.runtime; - -import org.antlr.v5.runtime.misc.Interval; - -import java.util.ArrayList; -import java.util.HashSet; -import java.util.List; -import java.util.Set; - -/** - * This implementation of {@link TokenStream} loads tokens from a - * {@link TokenSource} on-demand, and places the tokens in a buffer to provide - * access to any previous token by index. - * - *
- * This token stream ignores the value of {@link Token#getChannel}. If your - * parser requires the token stream filter tokens to only those on a particular - * channel, such as {@link Token#DEFAULT_CHANNEL} or - * {@link Token#HIDDEN_CHANNEL}, use a filtering token stream such a - * {@link CommonTokenStream}.
- */ -public class BufferedTokenStream implements TokenStream { - /** - * The {@link TokenSource} from which tokens for this stream are fetched. - */ - protected TokenSource tokenSource; - - /** - * A collection of all tokens fetched from the token source. The list is - * considered a complete view of the input once {@link #fetchedEOF} is set - * to {@code true}. - */ - protected ListThis field is set to -1 when the stream is first constructed or when - * {@link #setTokenSource} is called, indicating that the first token has - * not yet been fetched from the token source. For additional information, - * see the documentation of {@link IntStream} for a description of - * Initializing Methods.
- */ - protected int p = -1; - - /** - * Indicates whether the {@link Token#EOF} token has been fetched from - * {@link #tokenSource} and added to {@link #tokens}. This field improves - * performance for the following cases: - * - *For example, {@link CommonTokenStream} overrides this method to ensure that - * the seek target is always an on-channel token.
- * - * @param i The target token index. - * @return The adjusted target token index. - */ - protected int adjustSeekIndex(int i) { - return i; - } - - protected final void lazyInit() { - if (p == -1) { - setup(); - } - } - - protected void setup() { - sync(0); - p = adjustSeekIndex(0); - } - - /** Reset this token stream by setting its token source. */ - public void setTokenSource(TokenSource tokenSource) { - this.tokenSource = tokenSource; - tokens.clear(); - p = -1; - fetchedEOF = false; - } - - public List- * If {@code i} specifies an index at or after the EOF token, the EOF token - * index is returned. This is due to the fact that the EOF token is treated - * as though it were on every channel.
- */ - protected int previousTokenOnChannel(int i, int channel) { - sync(i); - if (i >= size()) { - // the EOF token is on every channel - return size() - 1; - } - - while (i >= 0) { - Token token = tokens.get(i); - if (token.getType() == Token.EOF || token.getChannel() == channel) { - return i; - } - - i--; - } - - return i; - } - - /** Collect all tokens on specified channel to the right of - * the current token up until we see a token on DEFAULT_TOKEN_CHANNEL or - * EOF. If channel is -1, find any non default channel token. - */ - public List- * These properties share a field to reduce the memory footprint of - * {@link CommonToken}. Tokens created by a {@link CommonTokenFactory} from - * the same source and input stream share a reference to the same - * {@link Pair} containing these values.
- */ - - protected Pair- * If {@code oldToken} is also a {@link CommonToken} instance, the newly - * constructed token will share a reference to the {@link #text} field and - * the {@link Pair} stored in {@link #source}. Otherwise, {@link #text} will - * be assigned the result of calling {@link #getText}, and {@link #source} - * will be constructed from the result of {@link Token#getTokenSource} and - * {@link Token#getInputStream}.
- * - * @param oldToken The token to copy. - */ - public CommonToken(Token oldToken) { - type = oldToken.getType(); - line = oldToken.getLine(); - index = oldToken.getTokenIndex(); - charPositionInLine = oldToken.getCharPositionInLine(); - channel = oldToken.getChannel(); - start = oldToken.getStartIndex(); - stop = oldToken.getStopIndex(); - - if (oldToken instanceof CommonToken) { - text = ((CommonToken)oldToken).text; - source = ((CommonToken)oldToken).source; - } - else { - text = oldToken.getText(); - source = new Pair- * This token factory does not explicitly copy token text when constructing - * tokens.
- */ - public static final TokenFactory- * The default value is {@code false} to avoid the performance and memory - * overhead of copying text for every token unless explicitly requested.
- */ - protected final boolean copyText; - - /** - * Constructs a {@link CommonTokenFactory} with the specified value for - * {@link #copyText}. - * - *- * When {@code copyText} is {@code false}, the {@link #DEFAULT} instance - * should be used instead of constructing a new instance.
- * - * @param copyText The value for {@link #copyText}. - */ - public CommonTokenFactory(boolean copyText) { this.copyText = copyText; } - - /** - * Constructs a {@link CommonTokenFactory} with {@link #copyText} set to - * {@code false}. - * - *- * The {@link #DEFAULT} instance should be used instead of calling this - * directly.
- */ - public CommonTokenFactory() { this(false); } - - @Override - public CommonToken create(Pair- * This token stream provides access to all tokens by index or when calling - * methods like {@link #getText}. The channel filtering is only used for code - * accessing tokens via the lookahead methods {@link #LA}, {@link #LT}, and - * {@link #LB}.
- * - *- * By default, tokens are placed on the default channel - * ({@link Token#DEFAULT_CHANNEL}), but may be reassigned by using the - * {@code ->channel(HIDDEN)} lexer command, or by using an embedded action to - * call {@link Lexer#setChannel}. - *
- * - *- * Note: lexer rules which use the {@code ->skip} lexer command or call - * {@link Lexer#skip} do not produce tokens at all, so input text matched by - * such a rule will not be available as part of the token stream, regardless of - * channel.
we - */ -public class CommonTokenStream extends BufferedTokenStream { - /** - * Specifies the channel to use for filtering tokens. - * - *- * The default value is {@link Token#DEFAULT_CHANNEL}, which matches the - * default channel assigned to tokens created by the lexer.
- */ - protected int channel = Token.DEFAULT_CHANNEL; - - /** - * Constructs a new {@link CommonTokenStream} using the specified token - * source and the default token channel ({@link Token#DEFAULT_CHANNEL}). - * - * @param tokenSource The token source. - */ - public CommonTokenStream(TokenSource tokenSource) { - super(tokenSource); - } - - /** - * Constructs a new {@link CommonTokenStream} using the specified token - * source and filtering tokens to the specified channel. Only tokens whose - * {@link Token#getChannel} matches {@code channel} or have the - * {@link Token#getType} equal to {@link Token#EOF} will be returned by the - * token stream lookahead methods. - * - * @param tokenSource The token source. - * @param channel The channel to use for filtering tokens. - */ - public CommonTokenStream(TokenSource tokenSource, int channel) { - this(tokenSource); - this.channel = channel; - } - - @Override - protected int adjustSeekIndex(int i) { - return nextTokenOnChannel(i, channel); - } - - @Override - protected Token LB(int k) { - if ( k==0 || (p-k)<0 ) return null; - - int i = p; - int n = 1; - // find k good tokens looking backwards - while ( n<=k && i>0 ) { - // skip off-channel tokens - i = previousTokenOnChannel(i - 1, channel); - n++; - } - if ( i<0 ) return null; - return tokens.get(i); - } - - @Override - public Token LT(int k) { - //System.out.println("enter LT("+k+")"); - lazyInit(); - if ( k == 0 ) return null; - if ( k < 0 ) return LB(-k); - int i = p; - int n = 1; // we know tokens[p] is a good one - // find k good tokens - while ( nThe default implementation simply calls {@link #endErrorCondition} to - * ensure that the handler is not in error recovery mode.
- */ - @Override - public void reset(Parser recognizer) { - endErrorCondition(recognizer); - } - - /** - * This method is called to enter error recovery mode when a recognition - * exception is reported. - * - * @param recognizer the parser instance - */ - protected void beginErrorCondition(Parser recognizer) { - errorRecoveryMode = true; - } - - /** - * {@inheritDoc} - */ - @Override - public boolean inErrorRecoveryMode(Parser recognizer) { - return errorRecoveryMode; - } - - /** - * This method is called to leave error recovery mode after recovering from - * a recognition exception. - * - * @param recognizer - */ - protected void endErrorCondition(Parser recognizer) { - errorRecoveryMode = false; - lastErrorStates = null; - lastErrorIndex = -1; - } - - /** - * {@inheritDoc} - * - *The default implementation simply calls {@link #endErrorCondition}.
- */ - @Override - public void reportMatch(Parser recognizer) { - endErrorCondition(recognizer); - } - - /** - * {@inheritDoc} - * - *The default implementation returns immediately if the handler is already - * in error recovery mode. Otherwise, it calls {@link #beginErrorCondition} - * and dispatches the reporting task based on the runtime type of {@code e} - * according to the following table.
- * - *The default implementation resynchronizes the parser by consuming tokens - * until we find one in the resynchronization set--loosely the set of tokens - * that can follow the current rule.
- */ - @Override - public void recover(Parser recognizer, RecognitionException e) { -// System.out.println("recover in "+recognizer.getRuleInvocationStack()+ -// " index="+recognizer.getInputStream().index()+ -// ", lastErrorIndex="+ -// lastErrorIndex+ -// ", states="+lastErrorStates); - if ( lastErrorIndex==recognizer.getInputStream().index() && - lastErrorStates != null && - lastErrorStates.contains(recognizer.getState()) ) { - // uh oh, another error at same token index and previously-visited - // state in ATN; must be a case where LT(1) is in the recovery - // token set so nothing got consumed. Consume a single token - // at least to prevent an infinite loop; this is a failsafe. -// System.err.println("seen error condition before index="+ -// lastErrorIndex+", states="+lastErrorStates); -// System.err.println("FAILSAFE consumes "+recognizer.getTokenNames()[recognizer.getInputStream().LA(1)]); - recognizer.consume(); - } - lastErrorIndex = recognizer.getInputStream().index(); - if ( lastErrorStates==null ) lastErrorStates = new IntervalSet(); - lastErrorStates.add(recognizer.getState()); - IntervalSet followSet = getErrorRecoverySet(recognizer); - consumeUntil(recognizer, followSet); - } - - /** - * The default implementation of {@link ANTLRErrorStrategy#sync} makes sure - * that the current lookahead symbol is consistent with what were expecting - * at this point in the ATN. You can call this anytime but ANTLR only - * generates code to check before subrules/loops and each iteration. - * - *Implements Jim Idle's magic sync mechanism in closures and optional - * subrules. E.g.,
- * - *
- * a : sync ( stuff sync )* ;
- * sync : {consume to what can follow sync} ;
- *
- *
- * At the start of a sub rule upon error, {@link #sync} performs single
- * token deletion, if possible. If it can't do that, it bails on the current
- * rule and uses the default error recovery, which consumes until the
- * resynchronization set of the current rule.
- *
- * If the sub rule is optional ({@code (...)?}, {@code (...)*}, or block - * with an empty alternative), then the expected set includes what follows - * the subrule.
- * - *During loop iteration, it consumes until it sees a token that can start a - * sub rule or what follows loop. Yes, that is pretty aggressive. We opt to - * stay in the loop as long as possible.
- * - *ORIGINS
- * - *Previous versions of ANTLR did a poor job of their recovery within loops. - * A single mismatch token or missing token would force the parser to bail - * out of the entire rules surrounding the loop. So, for rule
- * - *
- * classDef : 'class' ID '{' member* '}'
- *
- *
- * input with an extra token between members would force the parser to
- * consume until it found the next class definition rather than the next
- * member definition of the current class.
- *
- * This functionality cost a little bit of effort because the parser has to - * compare token set at the start of the loop and at each iteration. If for - * some reason speed is suffering for you, you can turn off this - * functionality by simply overriding this method as a blank { }.
- */ - @Override - public void sync(Parser recognizer) throws RecognitionException { - ATNState s = recognizer.getInterpreter().atn.states.get(recognizer.getState()); -// System.err.println("sync @ "+s.stateNumber+"="+s.getClass().getSimpleName()); - // If already recovering, don't try to sync - if (inErrorRecoveryMode(recognizer)) { - return; - } - - TokenStream tokens = recognizer.getInputStream(); - int la = tokens.LA(1); - - // try cheaper subset first; might get lucky. seems to shave a wee bit off - IntervalSet nextTokens = recognizer.getATN().nextTokens(s); - if (nextTokens.contains(la)) { - // We are sure the token matches - nextTokensContext = null; - nextTokensState = ATNState.INVALID_STATE_NUMBER; - return; - } - - if (nextTokens.contains(Token.EPSILON)) { - if (nextTokensContext == null) { - // It's possible the next token won't match; information tracked - // by sync is restricted for performance. - nextTokensContext = recognizer.getContext(); - nextTokensState = recognizer.getState(); - } - return; - } - - switch (s.getStateType()) { - case ATNState.BLOCK_START: - case ATNState.STAR_BLOCK_START: - case ATNState.PLUS_BLOCK_START: - case ATNState.STAR_LOOP_ENTRY: - // report error and recover if possible - if ( singleTokenDeletion(recognizer)!=null ) { - return; - } - - throw new InputMismatchException(recognizer); - - case ATNState.PLUS_LOOP_BACK: - case ATNState.STAR_LOOP_BACK: -// System.err.println("at loop back: "+s.getClass().getSimpleName()); - reportUnwantedToken(recognizer); - IntervalSet expecting = recognizer.getExpectedTokens(); - IntervalSet whatFollowsLoopIterationOrRule = - expecting.or(getErrorRecoverySet(recognizer)); - consumeUntil(recognizer, whatFollowsLoopIterationOrRule); - break; - - default: - // do nothing if we can't identify the exact kind of ATN state - break; - } - } - - /** - * This is called by {@link #reportError} when the exception is a - * {@link NoViableAltException}. - * - * @see #reportError - * - * @param recognizer the parser instance - * @param e the recognition exception - */ - protected void reportNoViableAlternative(Parser recognizer, - NoViableAltException e) - { - TokenStream tokens = recognizer.getInputStream(); - String input; - if ( tokens!=null ) { - if ( e.getStartToken().getType()==Token.EOF ) input = "This method is called when {@link #singleTokenDeletion} identifies - * single-token deletion as a viable recovery strategy for a mismatched - * input error.
- * - *The default implementation simply returns if the handler is already in - * error recovery mode. Otherwise, it calls {@link #beginErrorCondition} to - * enter error recovery mode, followed by calling - * {@link Parser#notifyErrorListeners}.
- * - * @param recognizer the parser instance - */ - protected void reportUnwantedToken(Parser recognizer) { - if (inErrorRecoveryMode(recognizer)) { - return; - } - - beginErrorCondition(recognizer); - - Token t = recognizer.getCurrentToken(); - String tokenName = getTokenErrorDisplay(t); - IntervalSet expecting = getExpectedTokens(recognizer); - String msg = "extraneous input "+tokenName+" expecting "+ - expecting.toString(recognizer.getVocabulary()); - recognizer.notifyErrorListeners(t, msg, null); - } - - /** - * This method is called to report a syntax error which requires the - * insertion of a missing token into the input stream. At the time this - * method is called, the missing token has not yet been inserted. When this - * method returns, {@code recognizer} is in error recovery mode. - * - *This method is called when {@link #singleTokenInsertion} identifies - * single-token insertion as a viable recovery strategy for a mismatched - * input error.
- * - *The default implementation simply returns if the handler is already in - * error recovery mode. Otherwise, it calls {@link #beginErrorCondition} to - * enter error recovery mode, followed by calling - * {@link Parser#notifyErrorListeners}.
- * - * @param recognizer the parser instance - */ - protected void reportMissingToken(Parser recognizer) { - if (inErrorRecoveryMode(recognizer)) { - return; - } - - beginErrorCondition(recognizer); - - Token t = recognizer.getCurrentToken(); - IntervalSet expecting = getExpectedTokens(recognizer); - String msg = "missing "+expecting.toString(recognizer.getVocabulary())+ - " at "+getTokenErrorDisplay(t); - - recognizer.notifyErrorListeners(t, msg, null); - } - - /** - * {@inheritDoc} - * - *The default implementation attempts to recover from the mismatched input - * by using single token insertion and deletion as described below. If the - * recovery attempt fails, this method throws an - * {@link InputMismatchException}.
- * - *EXTRA TOKEN (single token deletion)
- * - *{@code LA(1)} is not what we are looking for. If {@code LA(2)} has the - * right token, however, then assume {@code LA(1)} is some extra spurious - * token and delete it. Then consume and return the next token (which was - * the {@code LA(2)} token) as the successful result of the match operation.
- * - *This recovery strategy is implemented by {@link #singleTokenDeletion}.
- * - *MISSING TOKEN (single token insertion)
- * - *If current token (at {@code LA(1)}) is consistent with what could come - * after the expected {@code LA(1)} token, then assume the token is missing - * and use the parser's {@link TokenFactory} to create it on the fly. The - * "insertion" is performed by returning the created token as the successful - * result of the match operation.
- * - *This recovery strategy is implemented by {@link #singleTokenInsertion}.
- * - *EXAMPLE
- * - *For example, Input {@code i=(3;} is clearly missing the {@code ')'}. When - * the parser returns from the nested call to {@code expr}, it will have - * call chain:
- * - *- * stat → expr → atom - *- * - * and it will be trying to match the {@code ')'} at this point in the - * derivation: - * - *
- * => ID '=' '(' INT ')' ('+' atom)* ';'
- * ^
- *
- *
- * The attempt to match {@code ')'} will fail when it sees {@code ';'} and
- * call {@link #recoverInline}. To recover, it sees that {@code LA(1)==';'}
- * is in the set of tokens that can follow the {@code ')'} token reference
- * in rule {@code atom}. It can assume that you forgot the {@code ')'}.
- */
- @Override
- public Token recoverInline(Parser recognizer)
- throws RecognitionException
- {
- // SINGLE TOKEN DELETION
- Token matchedSymbol = singleTokenDeletion(recognizer);
- if ( matchedSymbol!=null ) {
- // we have deleted the extra token.
- // now, move past ttype token as if all were ok
- recognizer.consume();
- return matchedSymbol;
- }
-
- // SINGLE TOKEN INSERTION
- if ( singleTokenInsertion(recognizer) ) {
- return getMissingSymbol(recognizer);
- }
-
- // even that didn't work; must throw the exception
- InputMismatchException e;
- if (nextTokensContext == null) {
- e = new InputMismatchException(recognizer);
- } else {
- e = new InputMismatchException(recognizer, nextTokensState, nextTokensContext);
- }
-
- throw e;
- }
-
- /**
- * This method implements the single-token insertion inline error recovery
- * strategy. It is called by {@link #recoverInline} if the single-token
- * deletion strategy fails to recover from the mismatched input. If this
- * method returns {@code true}, {@code recognizer} will be in error recovery
- * mode.
- *
- * This method determines whether or not single-token insertion is viable by - * checking if the {@code LA(1)} input symbol could be successfully matched - * if it were instead the {@code LA(2)} symbol. If this method returns - * {@code true}, the caller is responsible for creating and inserting a - * token with the correct type to produce this behavior.
- * - * @param recognizer the parser instance - * @return {@code true} if single-token insertion is a viable recovery - * strategy for the current mismatched input, otherwise {@code false} - */ - protected boolean singleTokenInsertion(Parser recognizer) { - int currentSymbolType = recognizer.getInputStream().LA(1); - // if current token is consistent with what could come after current - // ATN state, then we know we're missing a token; error recovery - // is free to conjure up and insert the missing token - ATNState currentState = recognizer.getInterpreter().atn.states.get(recognizer.getState()); - ATNState next = currentState.transition(0).target; - ATN atn = recognizer.getInterpreter().atn; - IntervalSet expectingAtLL2 = atn.nextTokens(next, recognizer._ctx); -// System.out.println("LT(2) set="+expectingAtLL2.toString(recognizer.getTokenNames())); - if ( expectingAtLL2.contains(currentSymbolType) ) { - reportMissingToken(recognizer); - return true; - } - return false; - } - - /** - * This method implements the single-token deletion inline error recovery - * strategy. It is called by {@link #recoverInline} to attempt to recover - * from mismatched input. If this method returns null, the parser and error - * handler state will not have changed. If this method returns non-null, - * {@code recognizer} will not be in error recovery mode since the - * returned token was a successful match. - * - *If the single-token deletion is successful, this method calls - * {@link #reportUnwantedToken} to report the error, followed by - * {@link Parser#consume} to actually "delete" the extraneous token. Then, - * before returning {@link #reportMatch} is called to signal a successful - * match.
- * - * @param recognizer the parser instance - * @return the successfully matched {@link Token} instance if single-token - * deletion successfully recovers from the mismatched input, otherwise - * {@code null} - */ - protected Token singleTokenDeletion(Parser recognizer) { - int nextTokenType = recognizer.getInputStream().LA(2); - IntervalSet expecting = getExpectedTokens(recognizer); - if ( expecting.contains(nextTokenType) ) { - reportUnwantedToken(recognizer); - /* - System.err.println("recoverFromMismatchedToken deleting "+ - ((TokenStream)recognizer.getInputStream()).LT(1)+ - " since "+((TokenStream)recognizer.getInputStream()).LT(2)+ - " is what we want"); - */ - recognizer.consume(); // simply delete extra token - // we want to return the token we're actually matching - Token matchedSymbol = recognizer.getCurrentToken(); - reportMatch(recognizer); // we know current token is correct - return matchedSymbol; - } - return null; - } - - /** Conjure up a missing token during error recovery. - * - * The recognizer attempts to recover from single missing - * symbols. But, actions might refer to that missing symbol. - * For example, x=ID {f($x);}. The action clearly assumes - * that there has been an identifier matched previously and that - * $x points at that token. If that token is missing, but - * the next token in the stream is what we want we assume that - * this token is missing and we keep going. Because we - * have to return some token to replace the missing token, - * we have to conjure one up. This method gives the user control - * over the tokens returned for missing tokens. Mostly, - * you will want to create something special for identifier - * tokens. For literals such as '{' and ',', the default - * action in the parser or tree parser works. It simply creates - * a CommonToken of the appropriate type. The text will be the token. - * If you change what tokens must be created by the lexer, - * override this method to create the appropriate tokens. - */ - protected Token getMissingSymbol(Parser recognizer) { - Token currentSymbol = recognizer.getCurrentToken(); - IntervalSet expecting = getExpectedTokens(recognizer); - int expectedTokenType = Token.INVALID_TYPE; - if ( !expecting.isNil() ) { - expectedTokenType = expecting.getMinElement(); // get any element - } - String tokenText; - if ( expectedTokenType== Token.EOF ) tokenText = "Initializing Methods: Some methods in this interface have - * unspecified behavior if no call to an initializing method has occurred after - * the stream was constructed. The following is a list of initializing methods:
- * - *This method is guaranteed to succeed if any of the following are true:
- * - *If {@code i} represents a position at or beyond the end of the stream, - * this method returns {@link #EOF}.
- * - *The return value is unspecified if {@code i<0} and fewer than {@code -i} - * calls to {@link #consume consume()} have occurred from the beginning of - * the stream before calling this method.
- * - * @throws UnsupportedOperationException if the stream does not support - * retrieving the value of the specified symbol - */ - int LA(int i); - - /** - * A mark provides a guarantee that {@link #seek seek()} operations will be - * valid over a "marked range" extending from the index where {@code mark()} - * was called to the current {@link #index index()}. This allows the use of - * streaming input sources by specifying the minimum buffering requirements - * to support arbitrary lookahead during prediction. - * - *The returned mark is an opaque handle (type {@code int}) which is passed - * to {@link #release release()} when the guarantees provided by the marked - * range are no longer necessary. When calls to - * {@code mark()}/{@code release()} are nested, the marks must be released - * in reverse order of which they were obtained. Since marked regions are - * used during performance-critical sections of prediction, the specific - * behavior of invalid usage is unspecified (i.e. a mark is not released, or - * a mark is released twice, or marks are not released in reverse order from - * which they were created).
- * - *The behavior of this method is unspecified if no call to an - * {@link IntStream initializing method} has occurred after this stream was - * constructed.
- * - *This method does not change the current position in the input stream.
- * - *The following example shows the use of {@link #mark mark()}, - * {@link #release release(mark)}, {@link #index index()}, and - * {@link #seek seek(index)} as part of an operation to safely work within a - * marked region, then restore the stream position to its original value and - * release the mark.
- *
- * IntStream stream = ...;
- * int index = -1;
- * int mark = stream.mark();
- * try {
- * index = stream.index();
- * // perform work here...
- * } finally {
- * if (index != -1) {
- * stream.seek(index);
- * }
- * stream.release(mark);
- * }
- *
- *
- * @return An opaque marker which should be passed to
- * {@link #release release()} when the marked range is no longer required.
- */
- int mark();
-
- /**
- * This method releases a marked range created by a call to
- * {@link #mark mark()}. Calls to {@code release()} must appear in the
- * reverse order of the corresponding calls to {@code mark()}. If a mark is
- * released twice, or if marks are not released in reverse order of the
- * corresponding calls to {@code mark()}, the behavior is unspecified.
- *
- * For more information and an example, see {@link #mark}.
- * - * @param marker A marker returned by a call to {@code mark()}. - * @see #mark - */ - void release(int marker); - - /** - * Return the index into the stream of the input symbol referred to by - * {@code LA(1)}. - * - *The behavior of this method is unspecified if no call to an - * {@link IntStream initializing method} has occurred after this stream was - * constructed.
- */ - int index(); - - /** - * Set the input cursor to the position indicated by {@code index}. If the - * specified index lies past the end of the stream, the operation behaves as - * though {@code index} was the index of the EOF symbol. After this method - * returns without throwing an exception, then at least one of the following - * will be true. - * - *- * {@link ParserRuleContext} does not include field storage for the rule index - * since the context classes created by the code generator override the - * {@link #getRuleIndex} method to return the correct value for that context. - * Since the parser interpreter does not use the context classes generated for a - * parser, this class (with slightly more memory overhead per node) is used to - * provide equivalent functionality.
- */ -public class InterpreterRuleContext extends ParserRuleContext { - /** This is the backing field for {@link #getRuleIndex}. */ - protected int ruleIndex = -1; - - public InterpreterRuleContext() { } - - /** - * Constructs a new {@link InterpreterRuleContext} with the specified - * parent, invoking state, and rule index. - * - * @param parent The parent context. - * @param invokingStateNumber The invoking state number. - * @param ruleIndex The rule index for the current context. - */ - public InterpreterRuleContext(ParserRuleContext parent, - int invokingStateNumber, - int ruleIndex) - { - super(parent, invokingStateNumber); - this.ruleIndex = ruleIndex; - } - - @Override - public int getRuleIndex() { - return ruleIndex; - } -} diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/Lexer.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/Lexer.java deleted file mode 100644 index 9100975b2..000000000 --- a/runtime/Java/src/main/java/org/antlr/v5/runtime/Lexer.java +++ /dev/null @@ -1,410 +0,0 @@ -/* - * Copyright (c) 2012-present The ANTLR Project. All rights reserved. - * Use of this file is governed by the BSD 3-clause license that - * can be found in the LICENSE.txt file in the project root. - */ -package org.antlr.v5.runtime; - -import org.antlr.v5.runtime.atn.LexerATNSimulator; -import org.antlr.v5.runtime.misc.IntegerStack; -import org.antlr.v5.runtime.misc.Interval; -import org.antlr.v5.runtime.misc.Pair; - -import java.util.ArrayList; -import java.util.EmptyStackException; -import java.util.List; - -/** A lexer is recognizer that draws input symbols from a character stream. - * lexer grammars result in a subclass of this object. A Lexer object - * uses simplified match() and error recovery mechanisms in the interest - * of speed. - */ -public abstract class Lexer extends RecognizerIf the symbol type does not match, - * {@link ANTLRErrorStrategy#recoverInline} is called on the current error - * strategy to attempt recovery. If {@link #getBuildParseTree} is - * {@code true} and the token index of the symbol returned by - * {@link ANTLRErrorStrategy#recoverInline} is -1, the symbol is added to - * the parse tree by calling {@link #createErrorNode(ParserRuleContext, Token)} then - * {@link ParserRuleContext#addErrorNode(ErrorNode)}.
- * - * @param ttype the token type to match - * @return the matched symbol - * @throws RecognitionException if the current input symbol did not match - * {@code ttype} and the error strategy could not recover from the - * mismatched symbol - */ - public Token match(int ttype) throws RecognitionException { - Token t = getCurrentToken(); - if ( t.getType()==ttype ) { - if ( ttype==Token.EOF ) { - matchedEOF = true; - } - _errHandler.reportMatch(this); - consume(); - } - else { - t = _errHandler.recoverInline(this); - if ( _buildParseTrees && t.getTokenIndex()==-1 ) { - // we must have conjured up a new token during single token insertion - // if it's not the current symbol - _ctx.addErrorNode(createErrorNode(_ctx,t)); - } - } - return t; - } - - /** - * Match current input symbol as a wildcard. If the symbol type matches - * (i.e. has a value greater than 0), {@link ANTLRErrorStrategy#reportMatch} - * and {@link #consume} are called to complete the match process. - * - *If the symbol type does not match, - * {@link ANTLRErrorStrategy#recoverInline} is called on the current error - * strategy to attempt recovery. If {@link #getBuildParseTree} is - * {@code true} and the token index of the symbol returned by - * {@link ANTLRErrorStrategy#recoverInline} is -1, the symbol is added to - * the parse tree by calling {@link Parser#createErrorNode(ParserRuleContext, Token)}. then - * {@link ParserRuleContext#addErrorNode(ErrorNode)}
- * - * @return the matched symbol - * @throws RecognitionException if the current input symbol did not match - * a wildcard and the error strategy could not recover from the mismatched - * symbol - */ - public Token matchWildcard() throws RecognitionException { - Token t = getCurrentToken(); - if (t.getType() > 0) { - _errHandler.reportMatch(this); - consume(); - } - else { - t = _errHandler.recoverInline(this); - if (_buildParseTrees && t.getTokenIndex() == -1) { - // we must have conjured up a new token during single token insertion - // if it's not the current symbol - _ctx.addErrorNode(createErrorNode(_ctx,t)); - } - } - - return t; - } - - /** - * Track the {@link ParserRuleContext} objects during the parse and hook - * them up using the {@link ParserRuleContext#children} list so that it - * forms a parse tree. The {@link ParserRuleContext} returned from the start - * rule represents the root of the parse tree. - * - *Note that if we are not building parse trees, rule contexts only point - * upwards. When a rule exits, it returns the context but that gets garbage - * collected if nobody holds a reference. It points upwards but nobody - * points at it.
- * - *When we build parse trees, we are adding all of these contexts to - * {@link ParserRuleContext#children} list. Contexts are then not candidates - * for garbage collection.
- */ - public void setBuildParseTree(boolean buildParseTrees) { - this._buildParseTrees = buildParseTrees; - } - - /** - * Gets whether or not a complete parse tree will be constructed while - * parsing. This property is {@code true} for a newly constructed parser. - * - * @return {@code true} if a complete parse tree will be constructed while - * parsing, otherwise {@code false} - */ - public boolean getBuildParseTree() { - return _buildParseTrees; - } - - /** - * Trim the internal lists of the parse tree during parsing to conserve memory. - * This property is set to {@code false} by default for a newly constructed parser. - * - * @param trimParseTrees {@code true} to trim the capacity of the {@link ParserRuleContext#children} - * list to its size after a rule is parsed. - */ - public void setTrimParseTree(boolean trimParseTrees) { - if (trimParseTrees) { - if (getTrimParseTree()) return; - addParseListener(TrimToSizeListener.INSTANCE); - } - else { - removeParseListener(TrimToSizeListener.INSTANCE); - } - } - - /** - * @return {@code true} if the {@link ParserRuleContext#children} list is trimmed - * using the default {@link Parser.TrimToSizeListener} during the parse process. - */ - public boolean getTrimParseTree() { - return getParseListeners().contains(TrimToSizeListener.INSTANCE); - } - - - public ListTo support output-preserving grammar transformations (including but not - * limited to left-recursion removal, automated left-factoring, and - * optimized code generation), calls to listener methods during the parse - * may differ substantially from calls made by - * {@link ParseTreeWalker#DEFAULT} used after the parse is complete. In - * particular, rule entry and exit events may occur in a different order - * during the parse than after the parser. In addition, calls to certain - * rule entry methods may be omitted.
- * - *With the following specific exceptions, calls to listener events are - * deterministic, i.e. for identical input the calls to listener - * methods will be the same.
- * - *If {@code listener} is {@code null} or has not been added as a parse - * listener, this method does nothing.
- * - * @see #addParseListener - * - * @param listener the listener to remove - */ - public void removeParseListener(ParseTreeListener listener) { - if (_parseListeners != null) { - if (_parseListeners.remove(listener)) { - if (_parseListeners.isEmpty()) { - _parseListeners = null; - } - } - } - } - - /** - * Remove all parse listeners. - * - * @see #addParseListener - */ - public void removeParseListeners() { - _parseListeners = null; - } - - /** - * Notify any parse listeners of an enter rule event. - * - * @see #addParseListener - */ - protected void triggerEnterRuleEvent() { - for (ParseTreeListener listener : _parseListeners) { - listener.enterEveryRule(_ctx); - _ctx.enterRule(listener); - } - } - - /** - * Notify any parse listeners of an exit rule event. - * - * @see #addParseListener - */ - protected void triggerExitRuleEvent() { - // reverse order walk of listeners - for (int i = _parseListeners.size()-1; i >= 0; i--) { - ParseTreeListener listener = _parseListeners.get(i); - _ctx.exitRule(listener); - listener.exitEveryRule(_ctx); - } - } - - /** - * Gets the number of syntax errors reported during parsing. This value is - * incremented each time {@link #notifyErrorListeners} is called. - * - * @see #notifyErrorListeners - */ - public int getNumberOfSyntaxErrors() { - return _syntaxErrors; - } - - @Override - public TokenFactory getTokenFactory() { - return _input.getTokenSource().getTokenFactory(); - } - - /** Tell our token source and error strategy about a new way to create tokens. */ - @Override - public void setTokenFactory(TokenFactory factory) { - _input.getTokenSource().setTokenFactory(factory); - } - - /** - * The ATN with bypass alternatives is expensive to create so we create it - * lazily. - * - * @throws UnsupportedOperationException if the current parser does not - * implement the {@link #getSerializedATN()} method. - */ - - public ATN getATNWithBypassAlts() { - String serializedAtn = getSerializedATN(); - if (serializedAtn == null) { - throw new UnsupportedOperationException("The current parser does not support an ATN with bypass alternatives."); - } - - synchronized (this) { - if ( bypassAltsAtnCache!=null ) { - return bypassAltsAtnCache; - } - ATNDeserializationOptions deserializationOptions = new ATNDeserializationOptions(); - deserializationOptions.setGenerateRuleBypassTransitions(true); - bypassAltsAtnCache = new ATNDeserializer(deserializationOptions).deserialize(serializedAtn.toCharArray()); - return bypassAltsAtnCache; - } - } - - /** - * The preferred method of getting a tree pattern. For example, here's a - * sample use: - * - *
- * ParseTree t = parser.expr();
- * ParseTreePattern p = parser.compileParseTreePattern("<ID>+0", MyParser.RULE_expr);
- * ParseTreeMatch m = p.match(t);
- * String id = m.get("ID");
- *
- */
- public ParseTreePattern compileParseTreePattern(String pattern, int patternRuleIndex) {
- if ( getTokenStream()!=null ) {
- TokenSource tokenSource = getTokenStream().getTokenSource();
- if ( tokenSource instanceof Lexer ) {
- Lexer lexer = (Lexer)tokenSource;
- return compileParseTreePattern(pattern, patternRuleIndex, lexer);
- }
- }
- throw new UnsupportedOperationException("Parser can't discover a lexer to use");
- }
-
- /**
- * The same as {@link #compileParseTreePattern(String, int)} but specify a
- * {@link Lexer} rather than trying to deduce it from this parser.
- */
- public ParseTreePattern compileParseTreePattern(String pattern, int patternRuleIndex,
- Lexer lexer)
- {
- ParseTreePatternMatcher m = new ParseTreePatternMatcher(lexer, this);
- return m.compile(pattern, patternRuleIndex);
- }
-
-
- public ANTLRErrorStrategy getErrorHandler() {
- return _errHandler;
- }
-
- public void setErrorHandler(ANTLRErrorStrategy handler) {
- this._errHandler = handler;
- }
-
- @Override
- public TokenStream getInputStream() { return getTokenStream(); }
-
- @Override
- public final void setInputStream(IntStream input) {
- setTokenStream((TokenStream)input);
- }
-
- public TokenStream getTokenStream() {
- return _input;
- }
-
- /** Set the token stream and reset the parser. */
- public void setTokenStream(TokenStream input) {
- this._input = null;
- reset();
- this._input = input;
- }
-
- /** Match needs to return the current input symbol, which gets put
- * into the label for the associated token ref; e.g., x=ID.
- */
-
- public Token getCurrentToken() {
- return _input.LT(1);
- }
-
- public final void notifyErrorListeners(String msg) {
- notifyErrorListeners(getCurrentToken(), msg, null);
- }
-
- public void notifyErrorListeners(Token offendingToken, String msg,
- RecognitionException e)
- {
- _syntaxErrors++;
- int line = -1;
- int charPositionInLine = -1;
- line = offendingToken.getLine();
- charPositionInLine = offendingToken.getCharPositionInLine();
-
- ANTLRErrorListener listener = getErrorListenerDispatch();
- listener.syntaxError(this, offendingToken, line, charPositionInLine, msg, e);
- }
-
- /**
- * Consume and return the {@linkplain #getCurrentToken current symbol}.
- *
- * E.g., given the following input with {@code A} being the current - * lookahead symbol, this function moves the cursor to {@code B} and returns - * {@code A}.
- * - *- * A B - * ^ - *- * - * If the parser is not in error recovery mode, the consumed symbol is added - * to the parse tree using {@link ParserRuleContext#addChild(TerminalNode)}, and - * {@link ParseTreeListener#visitTerminal} is called on any parse listeners. - * If the parser is in error recovery mode, the consumed symbol is - * added to the parse tree using {@link #createErrorNode(ParserRuleContext, Token)} then - * {@link ParserRuleContext#addErrorNode(ErrorNode)} and - * {@link ParseTreeListener#visitErrorNode} is called on any parse - * listeners. - */ - public Token consume() { - Token o = getCurrentToken(); - if (o.getType() != EOF) { - getInputStream().consume(); - } - boolean hasListener = _parseListeners != null && !_parseListeners.isEmpty(); - if (_buildParseTrees || hasListener) { - if ( _errHandler.inErrorRecoveryMode(this) ) { - ErrorNode node = _ctx.addErrorNode(createErrorNode(_ctx,o)); - if (_parseListeners != null) { - for (ParseTreeListener listener : _parseListeners) { - listener.visitErrorNode(node); - } - } - } - else { - TerminalNode node = _ctx.addChild(createTerminalNode(_ctx,o)); - if (_parseListeners != null) { - for (ParseTreeListener listener : _parseListeners) { - listener.visitTerminal(node); - } - } - } - } - return o; - } - - /** How to create a token leaf node associated with a parent. - * Typically, the terminal node to create is not a function of the parent. - * - * @since 4.7 - */ - public TerminalNode createTerminalNode(ParserRuleContext parent, Token t) { - return new TerminalNodeImpl(t); - } - - /** How to create an error node, given a token, associated with a parent. - * Typically, the error node to create is not a function of the parent. - * - * @since 4.7 - */ - public ErrorNode createErrorNode(ParserRuleContext parent, Token t) { - return new ErrorNodeImpl(t); - } - - protected void addContextToParseTree() { - ParserRuleContext parent = (ParserRuleContext)_ctx.parent; - // add current context to parent if we have a parent - if ( parent!=null ) { - parent.addChild(_ctx); - } - } - - /** - * Always called by generated parsers upon entry to a rule. Access field - * {@link #_ctx} get the current context. - */ - public void enterRule(ParserRuleContext localctx, int state, int ruleIndex) { - setState(state); - _ctx = localctx; - _ctx.start = _input.LT(1); - if (_buildParseTrees) addContextToParseTree(); - if ( _parseListeners != null) triggerEnterRuleEvent(); - } - - public void exitRule() { - if ( matchedEOF ) { - // if we have matched EOF, it cannot consume past EOF so we use LT(1) here - _ctx.stop = _input.LT(1); // LT(1) will be end of file - } - else { - _ctx.stop = _input.LT(-1); // stop node is what we just matched - } - // trigger event on _ctx, before it reverts to parent - if ( _parseListeners != null) triggerExitRuleEvent(); - setState(_ctx.invokingState); - _ctx = (ParserRuleContext)_ctx.parent; - } - - public void enterOuterAlt(ParserRuleContext localctx, int altNum) { - localctx.setAltNumber(altNum); - // if we have new localctx, make sure we replace existing ctx - // that is previous child of parse tree - if ( _buildParseTrees && _ctx != localctx ) { - ParserRuleContext parent = (ParserRuleContext)_ctx.parent; - if ( parent!=null ) { - parent.removeLastChild(); - parent.addChild(localctx); - } - } - _ctx = localctx; - } - - /** - * Get the precedence level for the top-most precedence rule. - * - * @return The precedence level for the top-most precedence rule, or -1 if - * the parser context is not nested within a precedence rule. - */ - public final int getPrecedence() { - if (_precedenceStack.isEmpty()) { - return -1; - } - - return _precedenceStack.peek(); - } - - /** - * @deprecated Use - * {@link #enterRecursionRule(ParserRuleContext, int, int, int)} instead. - */ - @Deprecated - public void enterRecursionRule(ParserRuleContext localctx, int ruleIndex) { - enterRecursionRule(localctx, getATN().ruleToStartState[ruleIndex].stateNumber, ruleIndex, 0); - } - - public void enterRecursionRule(ParserRuleContext localctx, int state, int ruleIndex, int precedence) { - setState(state); - _precedenceStack.push(precedence); - _ctx = localctx; - _ctx.start = _input.LT(1); - if (_parseListeners != null) { - triggerEnterRuleEvent(); // simulates rule entry for left-recursive rules - } - } - - /** Like {@link #enterRule} but for recursive rules. - * Make the current context the child of the incoming localctx. - */ - public void pushNewRecursionContext(ParserRuleContext localctx, int state, int ruleIndex) { - ParserRuleContext previous = _ctx; - previous.parent = localctx; - previous.invokingState = state; - previous.stop = _input.LT(-1); - - _ctx = localctx; - _ctx.start = previous.start; - if (_buildParseTrees) { - _ctx.addChild(previous); - } - - if ( _parseListeners != null ) { - triggerEnterRuleEvent(); // simulates rule entry for left-recursive rules - } - } - - public void unrollRecursionContexts(ParserRuleContext _parentctx) { - _precedenceStack.pop(); - _ctx.stop = _input.LT(-1); - ParserRuleContext retctx = _ctx; // save current ctx (return value) - - // unroll so _ctx is as it was before call to recursive method - if ( _parseListeners != null ) { - while ( _ctx != _parentctx ) { - triggerExitRuleEvent(); - _ctx = (ParserRuleContext)_ctx.parent; - } - } - else { - _ctx = _parentctx; - } - - // hook into tree - retctx.parent = _parentctx; - - if (_buildParseTrees && _parentctx != null) { - // add return ctx into invoking rule's tree - _parentctx.addChild(retctx); - } - } - - public ParserRuleContext getInvokingContext(int ruleIndex) { - ParserRuleContext p = _ctx; - while ( p!=null ) { - if ( p.getRuleIndex() == ruleIndex ) return p; - p = (ParserRuleContext)p.parent; - } - return null; - } - - public ParserRuleContext getContext() { - return _ctx; - } - - public void setContext(ParserRuleContext ctx) { - _ctx = ctx; - } - - @Override - public boolean precpred(RuleContext localctx, int precedence) { - return precedence >= _precedenceStack.peek(); - } - - public boolean inContext(String context) { - // TODO: useful in parser? - return false; - } - - /** - * Checks whether or not {@code symbol} can follow the current state in the - * ATN. The behavior of this method is equivalent to the following, but is - * implemented such that the complete context-sensitive follow set does not - * need to be explicitly constructed. - * - *
- * return getExpectedTokens().contains(symbol); - *- * - * @param symbol the symbol type to check - * @return {@code true} if {@code symbol} can follow the current state in - * the ATN, otherwise {@code false}. - */ - public boolean isExpectedToken(int symbol) { -// return getInterpreter().atn.nextTokens(_ctx); - ATN atn = getInterpreter().atn; - ParserRuleContext ctx = _ctx; - ATNState s = atn.states.get(getState()); - IntervalSet following = atn.nextTokens(s); - if (following.contains(symbol)) { - return true; - } -// System.out.println("following "+s+"="+following); - if ( !following.contains(Token.EPSILON) ) return false; - - while ( ctx!=null && ctx.invokingState>=0 && following.contains(Token.EPSILON) ) { - ATNState invokingState = atn.states.get(ctx.invokingState); - RuleTransition rt = (RuleTransition)invokingState.transition(0); - following = atn.nextTokens(rt.followState); - if (following.contains(symbol)) { - return true; - } - - ctx = (ParserRuleContext)ctx.parent; - } - - if ( following.contains(Token.EPSILON) && symbol == Token.EOF ) { - return true; - } - - return false; - } - - public boolean isMatchedEOF() { - return matchedEOF; - } - - /** - * Computes the set of input symbols which could follow the current parser - * state and context, as given by {@link #getState} and {@link #getContext}, - * respectively. - * - * @see ATN#getExpectedTokens(int, RuleContext) - */ - public IntervalSet getExpectedTokens() { - return getATN().getExpectedTokens(getState(), getContext()); - } - - - public IntervalSet getExpectedTokensWithinCurrentRule() { - ATN atn = getInterpreter().atn; - ATNState s = atn.states.get(getState()); - return atn.nextTokens(s); - } - - /** Get a rule's index (i.e., {@code RULE_ruleName} field) or -1 if not found. */ - public int getRuleIndex(String ruleName) { - Integer ruleIndex = getRuleIndexMap().get(ruleName); - if ( ruleIndex!=null ) return ruleIndex; - return -1; - } - - public ParserRuleContext getRuleContext() { return _ctx; } - - /** Return List<String> of the rule names in your parser instance - * leading up to a call to the current rule. You could override if - * you want more details such as the file/line info of where - * in the ATN a rule is invoked. - * - * This is very useful for error messages. - */ - public List
If the state number is not known, this method returns -1.
- */ - public int getOffendingState() { - return offendingState; - } - - protected final void setOffendingState(int offendingState) { - this.offendingState = offendingState; - } - - /** - * Gets the set of input symbols which could potentially follow the - * previously matched symbol at the time this exception was thrown. - * - *If the set of expected tokens is not known and could not be computed, - * this method returns {@code null}.
- * - * @return The set of token types that could potentially follow the current - * state in the ATN, or {@code null} if the information is not available. - */ - public IntervalSet getExpectedTokens() { - if (recognizer != null) { - return recognizer.getATN().getExpectedTokens(offendingState, ctx); - } - - return null; - } - - /** - * Gets the {@link RuleContext} at the time this exception was thrown. - * - *If the context is not available, this method returns {@code null}.
- * - * @return The {@link RuleContext} at the time this exception was thrown. - * If the context is not available, this method returns {@code null}. - */ - public RuleContext getCtx() { - return ctx; - } - - /** - * Gets the input stream which is the symbol source for the recognizer where - * this exception was thrown. - * - *If the input stream is not available, this method returns {@code null}.
- * - * @return The input stream which is the symbol source for the recognizer - * where this exception was thrown, or {@code null} if the stream is not - * available. - */ - public IntStream getInputStream() { - return input; - } - - - public Token getOffendingToken() { - return offendingToken; - } - - protected final void setOffendingToken(Token offendingToken) { - this.offendingToken = offendingToken; - } - - /** - * Gets the {@link Recognizer} where this exception occurred. - * - *If the recognizer is not available, this method returns {@code null}.
- * - * @return The recognizer where this exception occurred, or {@code null} if - * the recognizer is not available. - */ - public Recognizer getRecognizer() { - return recognizer; - } -} diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/Recognizer.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/Recognizer.java deleted file mode 100644 index 5944f4c2f..000000000 --- a/runtime/Java/src/main/java/org/antlr/v5/runtime/Recognizer.java +++ /dev/null @@ -1,276 +0,0 @@ -/* - * Copyright (c) 2012-present The ANTLR Project. All rights reserved. - * Use of this file is governed by the BSD 3-clause license that - * can be found in the LICENSE.txt file in the project root. - */ - -package org.antlr.v5.runtime; - -import org.antlr.v5.runtime.atn.ATN; -import org.antlr.v5.runtime.atn.ATNSimulator; -import org.antlr.v5.runtime.atn.ParseInfo; -import org.antlr.v5.runtime.misc.Utils; - -import java.util.Collections; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.WeakHashMap; -import java.util.concurrent.CopyOnWriteArrayList; - -public abstract class RecognizerUsed for XPath and tree pattern compilation.
- */ - public MapUsed for XPath and tree pattern compilation.
- */ - public MapFor interpreters, we don't know their serialized ATN despite having - * created the interpreter from it.
- */ - public String getSerializedATN() { - throw new UnsupportedOperationException("there is no serialized ATN"); - } - - /** For debugging and other purposes, might want the grammar name. - * Have ANTLR generate an implementation for this method. - */ - public abstract String getGrammarFileName(); - - /** - * Get the {@link ATN} used by the recognizer for prediction. - * - * @return The {@link ATN} used by the recognizer for prediction. - */ - public abstract ATN getATN(); - - /** - * Get the ATN interpreter used by the recognizer for prediction. - * - * @return The ATN interpreter used by the recognizer for prediction. - */ - public ATNInterpreter getInterpreter() { - return _interp; - } - - /** If profiling during the parse/lex, this will return DecisionInfo records - * for each decision in recognizer in a ParseInfo object. - * - * @since 4.3 - */ - public ParseInfo getParseInfo() { - return null; - } - - /** - * Set the ATN interpreter used by the recognizer for prediction. - * - * @param interpreter The ATN interpreter used by the recognizer for - * prediction. - */ - public void setInterpreter(ATNInterpreter interpreter) { - _interp = interpreter; - } - - /** What is the error header, normally line/character position information? */ - public String getErrorHeader(RecognitionException e) { - int line = e.getOffendingToken().getLine(); - int charPositionInLine = e.getOffendingToken().getCharPositionInLine(); - return "line "+line+":"+charPositionInLine; - } - - /** How should a token be displayed in an error message? The default - * is to display just the text, but during development you might - * want to have a lot of information spit out. Override in that case - * to use t.toString() (which, for CommonToken, dumps everything about - * the token). This is better than forcing you to override a method in - * your token objects because you don't have to go modify your lexer - * so that it creates a new Java type. - * - * @deprecated This method is not called by the ANTLR 4 Runtime. Specific - * implementations of {@link ANTLRErrorStrategy} may provide a similar - * feature when necessary. For example, see - * {@link DefaultErrorStrategy#getTokenErrorDisplay}. - */ - @Deprecated - public String getTokenErrorDisplay(Token t) { - if ( t==null ) return "
- * Since tokens on hidden channels (e.g. whitespace or comments) are not
- * added to the parse trees, they will not appear in the output of this
- * method.
- */
- @Override
- public String getText() {
- if (getChildCount() == 0) {
- return "";
- }
-
- StringBuilder builder = new StringBuilder();
- for (int i = 0; i < getChildCount(); i++) {
- builder.append(getChild(i).getText());
- }
-
- return builder.toString();
- }
-
- public int getRuleIndex() { return -1; }
-
- /** For rule associated with this parse tree internal node, return
- * the outer alternative number used to match the input. Default
- * implementation does not compute nor store this alt num. Create
- * a subclass of ParserRuleContext with backing field and set
- * option contextSuperClass.
- * to set it.
- *
- * @since 4.5.3
- */
- public int getAltNumber() { return ATN.INVALID_ALT_NUMBER; }
-
- /** Set the outer alternative number for this context node. Default
- * implementation does nothing to avoid backing field overhead for
- * trees that don't need it. Create
- * a subclass of ParserRuleContext with backing field and set
- * option contextSuperClass.
- *
- * @since 4.5.3
- */
- public void setAltNumber(int altNumber) { }
-
- /** @since 4.7. {@see ParseTree#setParent} comment */
- @Override
- public void setParent(RuleContext parent) {
- this.parent = parent;
- }
-
- @Override
- public ParseTree getChild(int i) {
- return null;
- }
-
- @Override
- public int getChildCount() {
- return 0;
- }
-
- @Override
- public
- * The runtime version information is provided by {@link #VERSION} and
- * {@link #getRuntimeVersion()}. Detailed information about these values is
- * provided in the documentation for each member.
- * The runtime version check is implemented by {@link #checkVersion}. Detailed
- * information about incorporating this call into user code, as well as its use
- * in generated code, is provided in the documentation for the method.
- * Version strings x.y and x.y.z are considered "compatible" and no error
- * would be generated. Likewise, version strings x.y-SNAPSHOT and x.y.z are
- * considered "compatible" because the major and minor components x.y
- * are the same in each.
- * To trap any error messages issued by this code, use System.setErr()
- * in your main() startup code.
- *
- * This compile-time constant value allows generated parsers and other
- * libraries to include a literal reference to the version of the ANTLR 4
- * runtime library the code was compiled against. At each release, we
- * change this value. Version numbers are assumed to have the form
- *
- * major.minor.patch.revision-suffix,
- *
- * with the individual components defined as follows.
- * This method provides runtime access to the {@link #VERSION} field, as
- * opposed to directly referencing the field as a compile-time constant.
- * The version check is designed to detect the following two specific
- * scenarios.
- * Starting with ANTLR 4.3, the code generator emits a call to this method
- * using two constants in each generated lexer and parser: a hard-coded
- * constant indicating the version of the tool used to generate the parser
- * and a reference to the compile-time constant {@link #VERSION}. At
- * runtime, this method is called during the initialization of the generated
- * parser to detect mismatched versions, and notify the registered listeners
- * prior to creating instances of the parser.
- * This method does not perform any detection or filtering of semantic
- * changes between tool and runtime versions. It simply checks for a
- * version match and emits an error to stderr if a difference
- * is detected.
- * Note that some breaking changes between releases could result in other
- * types of runtime exceptions, such as a {@link LinkageError}, prior to
- * calling this method. In these cases, the underlying version mismatch will
- * not be reported here. This method is primarily intended to
- * notify users of potential semantic changes between releases that do not
- * result in binary compatibility problems which would be detected by the
- * class loader. As with semantic changes, changes that break binary
- * compatibility between releases are mentioned in the release notes
- * accompanying the affected release.
- * Additional note for target developers: The version check
- * implemented by this class is designed to address specific compatibility
- * concerns that may arise during the execution of Java applications. Other
- * targets should consider the implementation of this method in the context
- * of that target's known execution environment, which may or may not
- * resemble the design provided for the Java target.
- * The non-negative numbers less than {@link #MIN_USER_CHANNEL_VALUE} are
- * assigned to the predefined channels {@link #DEFAULT_CHANNEL} and
- * {@link #HIDDEN_CHANNEL}. Errors from the lexer are never passed to the parser. Either you want to keep
- * going or you do not upon token recognition error. If you do not want to
- * continue lexing then you do not want to continue parsing. Just throw an
- * exception not under {@link RecognitionException} and Java will naturally toss
- * you all the way out of the recognizers. If you want to continue lexing then
- * you should not throw an exception to the parser--it has already requested a
- * token. Keep lexing until you get a valid one. Just report errors and keep
- * going, looking for a valid token. The preconditions for this method are the same as the preconditions of
- * {@link IntStream#seek}. If the behavior of {@code seek(index)} is
- * unspecified for the current state and given {@code index}, then the
- * behavior of this method is also unspecified. The symbol referred to by {@code index} differs from {@code seek()} only
- * in the case of filtering streams where {@code index} lies before the end
- * of the stream. Unlike {@code seek()}, this method does not adjust
- * {@code index} to point to a non-ignored symbol. If {@code ctx.getSourceInterval()} does not return a valid interval of
- * tokens provided by this stream, the behavior is unspecified. If the specified {@code start} or {@code stop} token was not provided by
- * this stream, or if the {@code stop} occurred before the {@code start}
- * token, the behavior is unspecified. For streams which ensure that the {@link Token#getTokenIndex} method is
- * accurate for all of its provided tokens, this method behaves like the
- * following code. Other streams may implement this method in other ways
- * provided the behavior is consistent with this at a high level. This is not the buffer capacity, that's {@code tokens.length}. The {@code LT(1)} token is {@code tokens[p]}. If {@code p == n}, we are
- * out of buffered tokens. This value is used to set the token indexes if the stream provides tokens
- * that implement {@link WritableToken}. The specific marker value used for this class allows for some level of
- * protection against misuse where {@code seek()} is called on a mark or
- * {@code release()} is called in the wrong order. The following table shows examples of lexer rules and the literal
- * names assigned to the corresponding token types. This method supports token types defined by any of the following
- * methods: The following table shows examples of lexer rules and the literal
- * names assigned to the corresponding token types. ANTLR provides a default implementation of this method, but
- * applications are free to override the behavior in any manner which makes
- * sense for the application. The default implementation returns the first
- * result from the following list which produces a non-{@code null}
- * result.
- * No literal or symbol names are assigned to token types, so
- * {@link #getDisplayName(int)} returns the numeric value for all tokens
- * except {@link Token#EOF}. The resulting vocabulary instance returns {@code null} for
- * {@link #getLiteralName(int)} and {@link #getSymbolicName(int)}, and the
- * value from {@code tokenNames} for the display names.
- *
- */
- public static final String VERSION = "0.0.1";
-
- /**
- * Gets the currently executing version of the ANTLR 4 runtime library.
- *
- *
- *
- *
- *
- * TokenStream stream = ...;
- * String text = "";
- * for (int i = interval.a; i <= interval.b; i++) {
- * text += stream.get(i).getText();
- * }
- *
- *
- * @param interval The interval of tokens within this stream to get text
- * for.
- * @return The text of all tokens within the specified interval in this
- * stream.
- *
- * @throws NullPointerException if {@code interval} is {@code null}
- */
- public String getText(Interval interval);
-
- /**
- * Return the text of all tokens in the stream. This method behaves like the
- * following code, including potential exceptions from the calls to
- * {@link IntStream#size} and {@link #getText(Interval)}, but may be
- * optimized by the specific implementation.
- *
- *
- * TokenStream stream = ...;
- * String text = stream.getText(new Interval(0, stream.size()));
- *
- *
- * @return The text of all tokens in the stream.
- */
- public String getText();
-
- /**
- * Return the text of all tokens in the source interval of the specified
- * context. This method behaves like the following code, including potential
- * exceptions from the call to {@link #getText(Interval)}, but may be
- * optimized by the specific implementation.
- *
- *
- * TokenStream stream = ...;
- * String text = stream.getText(ctx.getSourceInterval());
- *
- *
- * @param ctx The context providing the source interval of tokens to get
- * text for.
- * @return The text of all tokens within the source interval of {@code ctx}.
- */
- public String getText(RuleContext ctx);
-
- /**
- * Return the text of all tokens in this stream between {@code start} and
- * {@code stop} (inclusive).
- *
- *
- * TokenStream stream = ...;
- * String text = "";
- * for (int i = start.getTokenIndex(); i <= stop.getTokenIndex(); i++) {
- * text += stream.get(i).getText();
- * }
- *
- *
- * @param start The first token in the interval to get text for.
- * @param stop The last token in the interval to get text for (inclusive).
- * @return The text of all tokens lying between the specified {@code start}
- * and {@code stop} tokens.
- *
- * @throws UnsupportedOperationException if this stream does not support
- * this method for the specified tokens
- */
- public String getText(Token start, Token stop);
-}
diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/TokenStreamRewriter.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/TokenStreamRewriter.java
index 0bdff4b1f..c5c19d38e 100644
--- a/runtime/Java/src/main/java/org/antlr/v5/runtime/TokenStreamRewriter.java
+++ b/runtime/Java/src/main/java/org/antlr/v5/runtime/TokenStreamRewriter.java
@@ -5,7 +5,9 @@
*/
package org.antlr.v5.runtime;
-import org.antlr.v5.runtime.misc.Interval;
+import org.antlr.v5.runtime.core.Token;
+import org.antlr.v5.runtime.core.TokenStream;
+import org.antlr.v5.runtime.core.misc.Interval;
import java.util.ArrayList;
import java.util.HashMap;
@@ -357,14 +359,14 @@ private List
- *
- *
- * @param tokenType The token type.
- *
- * @return The string literal associated with the specified token type, or
- * {@code null} if no string literal is associated with the type.
- */
- String getLiteralName(int tokenType);
-
- /**
- * Gets the symbolic name associated with a token type. The string returned
- * by this method, when not {@code null}, can be used unaltered in a parser
- * grammar to represent this token type.
- *
- *
- *
- * Rule
- * Literal Name
- * Java String Literal
- *
- *
- * {@code THIS : 'this';}
- * {@code 'this'}
- * {@code "'this'"}
- *
- *
- * {@code SQUOTE : '\'';}
- * {@code '\''}
- * {@code "'\\''"}
- *
- *
- * {@code ID : [A-Z]+;}
- * n/a
- * {@code null}
- *
- *
- *
- * tokens{} block in a lexer or parser
- * grammar.
- *
- *
- * @param tokenType The token type.
- *
- * @return The symbolic name associated with the specified token type, or
- * {@code null} if no symbolic name is associated with the type.
- */
- String getSymbolicName(int tokenType);
-
- /**
- * Gets the display name of a token type.
- *
- *
- *
- * Rule
- * Symbolic Name
- *
- *
- * {@code THIS : 'this';}
- * {@code THIS}
- *
- *
- * {@code SQUOTE : '\'';}
- * {@code SQUOTE}
- *
- *
- * {@code ID : [A-Z]+;}
- * {@code ID}
- *
- *
- *
- * @param tokenType The token type.
- *
- * @return The display name of the token type, for use in error reporting or
- * other user-visible messages which reference specific token types.
- */
- String getDisplayName(int tokenType);
-}
diff --git a/runtime/Java/src/main/java/org/antlr/v5/runtime/VocabularyImpl.java b/runtime/Java/src/main/java/org/antlr/v5/runtime/VocabularyImpl.java
deleted file mode 100644
index cdf593db9..000000000
--- a/runtime/Java/src/main/java/org/antlr/v5/runtime/VocabularyImpl.java
+++ /dev/null
@@ -1,190 +0,0 @@
-/*
- * Copyright (c) 2012-present The ANTLR Project. All rights reserved.
- * Use of this file is governed by the BSD 3-clause license that
- * can be found in the LICENSE.txt file in the project root.
- */
-package org.antlr.v5.runtime;
-
-import java.util.Arrays;
-
-/**
- * This class provides a default implementation of the {@link Vocabulary}
- * interface.
- *
- * @author Sam Harwell
- */
-public class VocabularyImpl implements Vocabulary {
- private static final String[] EMPTY_NAMES = new String[0];
-
- /**
- * Gets an empty {@link Vocabulary} instance.
- *
- *
For efficiency, you can compile a tree pattern in string form to a
@@ -187,7 +192,7 @@ public ParseTreePattern compile(String pattern, int patternRuleIndex) {
ParserInterpreter parserInterp = new ParserInterpreter(parser.getGrammarFileName(),
parser.getVocabulary(),
Arrays.asList(parser.getRuleNames()),
- parser.getATNWithBypassAlts(),
+ parser.getAtnWithBypassAlts(),
tokens);
ParseTree tree = null;
@@ -236,7 +241,7 @@ public Parser getParser() {
/**
* Recursively walk {@code tree} against {@code patternTree}, filling
- * {@code match.}{@link ParseTreeMatch#labels labels}.
+ * {@code match.}{@link ParseTreeMatch#getLabels labels}.
*
* @return the first node encountered in {@code tree} which does not match
* a corresponding node in {@code patternTree}, or {@code null} if the match
@@ -340,7 +345,7 @@ else if ( t1.getText().equals(t2.getText()) ) {
/** Is {@code t} {@code (expr If {@code context} is {@code null}, it is treated as {@link ParserRuleContext#EMPTY}.
- * closure() tracks the depth of how far we dip into the outer context:
- * depth > 0. Note that it may not be totally accurate depth since I
- * don't ever decrement. TODO: make it a boolean then
- * For memory efficiency, the {@link #isPrecedenceFilterSuppressed} method
- * is also backed by this field. Since the field is publicly accessible, the
- * highest bit which would not cause the value to become negative is used to
- * store this field. This choice minimizes the risk that code which only
- * compares this value to 0 would be affected by the new purpose of the
- * flag. It also ensures the performance of the existing {@link ATNConfig}
- * constructors as well as certain operations like
- * {@link ATNConfigSet#add(ATNConfig, DoubleKeyMap)} method are
- * completely unaffected by the change. This method updates {@link #dipsIntoOuterContext} and
- * {@link #hasSemanticContext} when necessary.