De-globalize all input/parsing state#261
Merged
jpco merged 13 commits intowryun:masterfrom Mar 10, 2026
Merged
Conversation
Mostly empty now, but as much input and parsing state as possible should
be moved into Parser soon, for two reasons:
1. Parser is properly allocated (on the stack, for now, but can easily
change), while global variables make it impossible to run multiple
parsers at once.
2. A Parser's lifetime is very well scoped: one call to $&parse.
Inputs are long-lived and induce spooky, invisible behavior in the
shell. The more that Input can be shrunk, the more of that spooky,
invisible behavior is removed, the more orthogonal and predictable
the shell becomes.
Situation is still a little suboptimal; there are too many allocations for memory that can be reused (pspace, tokenbuf), and the code is generally less tidy than it ought to be.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces a new struct
Parserdefined in input.h. This struct is created and destroyed once per call toparse(), and is meant to contain any state with a lifetime equal to (or less than) that call.Longer-term, I would like to simplify and shrink
Inputas much as possible, moving some of its state intoParserand some of it elsewhere.Inputis completely invisible, often long-lived state with dynamic scope, which make it really difficult to reason about, and which directly determines how$&parseworks. It is possible that with enough workInputcould turn into the internal representation of a "file handle" like in #252. That's all down the line, though.Shorter term, this PR makes it possible to have multiple parsers safely running concurrently, which is the final prerequisite for running es code while reading input. So while this PR doesn't itself change anything for a user, it should open up a few pathways for significant future changes to the shell's input behaviors.
Performance note: This PR slows down the shell's reading and parsing behavior a bit. I will (optimistically) assert that this is okay, at least temporarily. There are some potential optimizations we can make to fix it, but I would like to do some refactoring and code-moving-around before then.
Portability note: This PR makes use of
%code requires,%parse-paramand%lex-paramin parse.y. This is supported by both byacc and bison.