Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

**WIP DO NOT MERGE IT** #54

Open
wants to merge 1 commit into
base: develop
Choose a base branch
from

Conversation

vinipsmaker
Copy link
Contributor

RATIONALE

A chunked parser that failed to consume the next token means either:

  1. Insufficient tokens. The user must buffer more and try again.
  2. Invalid token.

There's nothing polemic about the first case. As for the second one:

I. Either the stream is invalid and it'll abort parsing;
II. Or we're using a syntactic extension to the JSON stream and want to
trigger a fallback mini-parser to consume that part of the stream.

If a parser doesn't report invalid token the user will try to buffer the
stream forever. However if invalid token is a terminal state then the
user can't combine parsers to support an extended JSON syntax. A chunked
parser meant to be used combined with other parsers should report all 3
scenarios:

  • Valid token.
  • Insufficient tokens.
  • Invalid token (and invalid token cannot be a terminal state).

If we exclude the first event from the notification set the parser isn't
useful at all. If we exclude the second event then we don't have a
chunked parser. If we exclude the third event the user might buffer
forever on invalid tokens. Thus all three events must be part of the
notification set.

Updates #53

== RATIONALE

A chunked parser that failed to consume the next token means either:

1. Insufficient tokens. The user must buffer more and try again.
2. Invalid token.

There's nothing polemic about the first case. As for the second one:

I.  Either the stream is invalid and it'll abort parsing;
II. Or we're using a syntactic extension to the JSON stream and want to
    trigger a fallback mini-parser to consume that part of the stream.

If a parser doesn't report invalid token the user will try to buffer the
stream forever. However if invalid token is a terminal state then the
user can't combine parsers to support an extended JSON syntax. A chunked
parser meant to be used combined with other parsers should report all 3
scenarios:

* Valid token.
* Insufficient tokens.
* Invalid token (and invalid token cannot be a terminal state).

If we exclude the first event from the notification set the parser isn't
useful at all. If we exclude the second event then we don't have a
chunked parser. If we exclude the third event the user might buffer
forever on invalid tokens. Thus all three events must be part of the
notification set.
@vinipsmaker
Copy link
Contributor Author

Not sure how to handle the requirement to reach a non-terminal invalid-next-token state. I mean the interface would have to change here. Which small interface change should be implemented then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant