Provide infrastructure to migrate legacy analyzers to Spicy inside the Zeek tree. #1333
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rsmmr
force-pushed
the
topic/robin/zeek-spicy-infra
branch
3 times, most recently
from
ab26e3b
to
e18e3f2
Compare
bbannier
previously approved these changes
Dec 20, 2022
This uniques them and avoids duplicates when using multiple namespaces.
This can be used if triggering it through a static constructor isn't reliable.
rsmmr
force-pushed
the
topic/robin/zeek-spicy-infra
branch
from
6e83200
to
8279345
Compare
This adds support for extracting and retaining Zeek-style documentation strings (`##`, `##!`) in Spicy source code to all declarations: if they are prefixed with blocks of such comments, that information is retained along with the declaration for later access. This commit also prepares for supporting `##<` to document unit and struct fields, but that isn't implemented yet. We don't actually use the extracted information further yet, but may later leverage it for auto-generating Spicy-side documentation (which we currently do through a Python script ingesting source code directly.) Originally, this was implemented to support the Zeek plugin providing documentation to Zeekygen, but we ended up using a different approach there (prototyping events Zeek-side). Still, seems worth keeping this capability.
This prepares for having multiple linker units inside the same symbol namespace. Doing so however breaks the assumptions that there's only one, global linker scope, hence we remove the corresponding accessor. This will break the Spicy plugin, but we'll provide it with a new mechanism in a subsequent commit.
This provides access to the current module's linker scope. Behind the scenes, it takes care of binding to the appropriate C++-side global, even if multiple of them are visible inside the same symbol space.
`-x <prefix>` now writes out all generated C++ code as `<prefix>_*.cc`, also setting the internal C++ namespace prefix to `<prefix>`. Test will come in subsequent commit that's making use of this.
… code. This replicates a test we already have for HLTO modules, but now compiling multiple parser modules statically into a single C++ application. This ensures our linker scopes are working as expected in that case. It mimics what Zeek will be doing for internal Spicy analyzers.
We didn't have a mechanism yet for a host application to configure the runtime library's behavior. This follows what we do on the HILTI-side to provide that capability. Right now, there are no actual options available yet, but that will come.
This generalizes the confirmation/rejection from being a Zeek thing to being part of Spicy. Any Spicy grammar can call these functions as to tell its host application if it recognizes the input format. The host application can (but doesn't have to) install callbacks to receive that information and react appropriately. For example, the Zeek plugin will just forward it to Zeek's standard DPD hooks. For testing, `spicy-driver` gets a new option `--require-confirmation` that lets it exit with success only if `spicy::confirm_input()` has been called, and `spicy::reject_input()` hasn't.
With, e.g., `spicyz -L /x/y/z`, we used to append the given module path to the internal system paths, so that it was searched last. This change turns around the priority to search `/x/y/x` first, before the internal ones. That seems both more useful and more intuitive.
rsmmr
force-pushed
the
topic/robin/zeek-spicy-infra
branch
from
8279345
to
296c813
Compare
We were a bit inconsistent in the structure of exceptions that could
happen during runtime, making it hard for host applications for
consistently handle trouble. We tweak this, as follows:
- We now only have two general subcategories of exceptions under the
top-level `hilti::rt::Exception`:
- `RuntimeError`: non-fatal errors that mean a host
applications cannot complete the current line of processing,
but can otherwise continue. This is the main exception that
host applications should catch, and then proceed
appropriately.
- `UsageError`: fatal errors that indicate something about
using the runtime environment was fundamentally wrong, and a
host application should abort.
- Almost all other exceptions are now derived from `RuntimeError`,
including Spicy's `ParseError` and `RecoverableFailure`, so that
they can treated consistently.
|
I'm splitting this out into a series of individual PRs. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Goes with zeek/zeek#2651.