Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fleet-compatible generation for Grammar-Kit #359

Open
wants to merge 40 commits into
base: master
Choose a base branch
from

Conversation

KrylovBoris
Copy link

What is this PR for?

This PR aims to introduce additional functionality to the Grammar-Kit plugin that allows users to generate a parser and adjacent code that is compatible with Fleet's API.

Why were these changes made?

Currently, for plugin developers to port their IDEA plugins over to Fleet, they need to edit generated parser files or write them from scratch. To reduce the workload and allow for a more streamlined development process we introduce a way to generate Fleet-compatible code.

What is Fleet-compatible code?

For the parser to be compatible with Fleet, it needs to import packages from Fleet's own parsing core and not have PSI-related features. This could be achieved by tweaking imports and stopping Grammar-Kit from generating PSI.

What has been done?

For Fleet-specific generation logic, we created a separate FleetParserGenerator, a subclass of ParserGenerator. To avoid copying large chunks of code from the parent, we introduced a bunch of small methods that are responsible for adjusting original namespaces and generating small parts of the parser code. We also introduced FleetGenOptions, a subclass of GenOptions, to control file generation.

To interact with Fleet-specific parser, we added 2 actions to the plugin:

  1. Generate Parser Code For Fleet;
  2. Generate JFlex Lexer For Fleet;

To generate lexers we use a new template file, which includes Fleet-specific imports.

We also introduced 2 Fleet-only attributes to BNF files:

  1. adjustPackagesForFleet allows users to suppress moving generated files to "fleet/" directory. By default, when we generate files for Fleet, they are moved to "fleet/" directory, and "fleet." prefix is added to their respective package names. This is done to mimic Fleet's parser code layout and this option allows users to turn it off.
  2. generateFileTypeForFleet was added by request of the Fleet team. Since almost all IFileType implementations in Fleet are the same, this attribute was added to reduce the amount of boilerplate code, plugin developers have to write to make their parsers compilable. This attribute holds all the parameters to generate IFileType implementation for Fleet.

Finally, we added an optional parameter [-f] to the main function to switch from ParserGenerator to FleetParserGenerator

How have the other parts of the plugin been affected?

Since all Fleet-specific generation is stored in a separate class, it should not affect other parts of the program.

What's your plans for the future?

In the future we plan to add support of introduced functionality in grammar-kit-gradle-plugin.

@gregsh gregsh closed this May 7, 2024
@gregsh gregsh reopened this May 13, 2024
Boris Krylov added 28 commits May 13, 2024 19:41
@KrylovBoris KrylovBoris force-pushed the Boris.Krylov/grammar-kit-for-fleet branch from eb3c0eb to 129503f Compare May 13, 2024 17:14
gradle.properties Outdated Show resolved Hide resolved
gradle.properties Outdated Show resolved Hide resolved
@gregsh
Copy link
Collaborator

gregsh commented May 14, 2024

Please avoid changing any configuration files, including libraries xmls, idea/*.xml, and build scripts.

src/org/intellij/grammar/Main.java Outdated Show resolved Hide resolved
src/org/intellij/grammar/Main.java Outdated Show resolved Hide resolved
src/org/intellij/grammar/LightPsi.java Show resolved Hide resolved
resources/META-INF/plugin-java.xml Show resolved Hide resolved
resources/messages/GrammarKitBundle.properties Outdated Show resolved Hide resolved
@YannCebron
Copy link
Member

please add entry in CHANGELOG.md

@gregsh
Copy link
Collaborator

gregsh commented Jul 15, 2024

Overall

We don't want to have a separate FleetParserGenerator. We wish to parametrize ParsersGenerator and BNF PSI to simultaneously solve IntelliJ and Fleet tasks.

@@ -159,7 +159,7 @@ public ParserGenerator(@NotNull BnfFile psiFile,
myOutputPath = outputPath;
myPackagePrefix = packagePrefix;

G = new GenOptions(myFile);
G = createGenerator(myFile);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make genOptions a constructor param

super(psiFile, sourcePath, outputPath, packagePrefix);
myPossibleImports = new LinkedList<>();
var importList = psiFile.getAllPossibleAttributeValues(KnownAttribute.ELEMENT_TYPE_CLASS);
if (importList != null) myPossibleImports.addAll(psiFile.getAllPossibleAttributeValues(KnownAttribute.ELEMENT_TYPE_CLASS));
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Call getAllPossibleAttributeValues for the second time?


public class FleetParserGenerator extends ParserGenerator {

private final Collection<String> myPossibleImports;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HashSet maybe?

}
}

private void generateFileTypeClass() {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Already outdated. FleetPsiBuilder is already gone.

  2. Parser does not refer to this class, so it shall not be generated as part of parser generation. We can have a separate generator action for that if needed.

}

@NotNull
protected String adjustName(String importName) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adjustName must happen in BNF PSI, not in the generator.

As we need to support both "generate-time" and "edit-time" (in-IDE support)

ve.evaluate(context, out, "lexer.flex.template", new InputStreamReader(stream));
return StringUtil.convertLineSeparators(out.toString());
}

protected void putPackageName(@NotNull VelocityContext context, BnfFile bnfFile, @Nullable String packageName) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a method for each property is not good. It's a famous OO-spaghetti code.

One method for context creation would be better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants