Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Basic grammar support for data driven CSS property parsing
https://bugs.webkit.org/show_bug.cgi?id=248275 rdar://102630092 Reviewed by Darin Adler. Adds initial support for using a grammar to drive generation of CSS property parsing functions. This first cut only supports simple grammars with a single result term, but builds some foundations to build on. Not too certain with the syntax choices used for the grammar yet, but the current model is that every type of term is represented as a dictionary with a 'kind' property defining the kind and 'value' property for the value. The current set of kinds are: 'match-one' (disjunction e.g. [ foo | bar | baz ]), 'reference' (e.g. [ <foo> ]) and 'keyword' (e.g. [ foo ]). The value property of a 'match-one' term must be a list of terms, and the value type of a 'reference' or 'keyword' term must be a string. As an example: 'property-grammar': { 'kind': 'match-one', 'value': [ { "kind": "keyword", "value": "auto" }, { "kind": "reference", "value": "<length unitless-allowed>" } ] } would be equilent to the following grammar using CSS spec syntax: auto | <length unitless-allowed> Given how much wordier this base syntax is, there is also a shorthand syntax for these: - 'match-one' terms can just use a list directly, bypassing the dictionary. - 'reference' terms can use a string starting with '<' and ending with '>'. - 'keyword' terms can use a string (as long as it doesn't conflict with the reference syntax. This allows the example to be written as: 'property-grammar': [ "auto", "<length unitless-allowed>" ] While this works for now, once we introduce multiple consecutive terms, the ambiguity may become too burdensome and we may need to find a better syntax for 'match-one' than just a list. * Source/WebCore/css/CSSProperties.json: Replace 'custom-parser' and 'parser-function' values with the new 'parser-grammar' for all supported properties. Also adds new 'shared-grammar-rules' section with productions that are needed by multiple properities. * Source/WebCore/css/MediaQueryExpression.cpp: (WebCore::consumeFirstValue): * Source/WebCore/css/parser/CSSParserImpl.cpp: (WebCore::CSSParserImpl::consumeFontFeatureValuesRuleBlock): Update CSSPropertyParserHelpers::consumeIntegerZeroAndGreater() to use new name CSSPropertyParserHelpers::consumeNonNegativeInteger(), which is more consistent with our value range naming. * Source/WebCore/css/parser/CSSPropertyParser.cpp: (WebCore::CSSPropertyParser::canParseTypedCustomPropertyValue): (WebCore::CSSPropertyParser::collectParsedCustomPropertyValueDependencies): (WebCore::CSSPropertyParser::parseTypedCustomPropertyValue): Replace hand written consumeWidthOrHeight() with generated one. (WebCore::consumeCounterStylePad): (WebCore::consumeCounterStyleAdditiveSymbols): Update more uses of CSSPropertyParserHelpers::consumeIntegerZeroAndGreater(). (WebCore::CSSPropertyParser::parseFontFaceDescriptor): Replace hand written consumeFontVariantCaps() and consumeFontVariantPosition() with generated counterparts. (WebCore::consumeBasePaletteDescriptor): (WebCore::consumeOverrideColorsDescriptor): Update more uses of CSSPropertyParserHelpers::consumeIntegerZeroAndGreater(). (WebCore::CSSPropertyParser::consumeFont): Replace hand written consumeFontSize() and consumeLineHeight() with generated counterparts. (WebCore::CSSPropertyParser::consumeFontVariantShorthand): Replace more uses of consumeFontVariantCaps() and consumeFontVariantPosition(). (WebCore::CSSPropertyParser::consumeColumns): Replace hand written consumeColumnWidth() and consumeColumnCount() with generated counterparts. (WebCore::CSSPropertyParser::consumeBorder): Replace hand written consumeLineWidth() with generated counterpart. (WebCore::CSSPropertyParser::consumeOverscrollBehaviorShorthand): Replace hand written consumeOverscrollBehavior() with generated counterparts. (WebCore::CSSPropertyParser::parseShorthand): Replace hand written consumeGapLength() with generated counterparts. * Source/WebCore/css/parser/CSSPropertyParserHelpers.cpp: (WebCore::CSSPropertyParserHelpers::computeMinimumValue): (WebCore::CSSPropertyParserHelpers::consumeIntegerTypeRaw): (WebCore::CSSPropertyParserHelpers::consumeIntegerType): (WebCore::CSSPropertyParserHelpers::consumeIntegerRaw): (WebCore::CSSPropertyParserHelpers::consumeInteger): (WebCore::CSSPropertyParserHelpers::consumeNonNegativeIntegerRaw): (WebCore::CSSPropertyParserHelpers::consumeNonNegativeInteger): (WebCore::CSSPropertyParserHelpers::consumePositiveIntegerRaw): (WebCore::CSSPropertyParserHelpers::consumePositiveInteger): Replace file specific IntegerRange with new IntegerValueRange enum and update naming to be more inline with the ValueRange enum (e.g. non-negative rather than zero-or-greater, and positive rather than one-or-greater). Adds a new consumeInteger overload that takes the new enum as a parameter to allow similar overloading of <integer> and <number> in the grammars. (WebCore::CSSPropertyParserHelpers::consumeSingleAxisPosition): (WebCore::CSSPropertyParserHelpers::isPredefinedCounterStyle): (WebCore::CSSPropertyParserHelpers::consumeFontWeightRaw): (WebCore::CSSPropertyParserHelpers::consumeFontStyleRaw): (WebCore::CSSPropertyParserHelpers::consumeAutoOrLengthOrPercent): Mark a few functions that are only used in this file as static (and remove there declarations from the header. (WebCore::CSSPropertyParserHelpers::consumeIntegerZeroAndGreaterRaw): Deleted. (WebCore::CSSPropertyParserHelpers::consumeIntegerZeroAndGreater): Deleted. (WebCore::CSSPropertyParserHelpers::consumeAutoOrString): Deleted. (WebCore::CSSPropertyParserHelpers::consumeAutoOrColor): Deleted. (WebCore::CSSPropertyParserHelpers::consumeAutoOrPosition): Deleted. (WebCore::CSSPropertyParserHelpers::consumeNoneOrURL): Deleted. (WebCore::CSSPropertyParserHelpers::consumePage): Deleted. (WebCore::CSSPropertyParserHelpers::consumeFontVariantCaps): Deleted. (WebCore::CSSPropertyParserHelpers::consumeFontVariantPosition): Deleted. (WebCore::CSSPropertyParserHelpers::consumeFontPalette): Deleted. (WebCore::CSSPropertyParserHelpers::consumeLetterSpacing): Deleted. (WebCore::CSSPropertyParserHelpers::consumeWordSpacing): Deleted. (WebCore::CSSPropertyParserHelpers::consumeTabSize): Deleted. (WebCore::CSSPropertyParserHelpers::consumeTextSizeAdjust): Deleted. (WebCore::CSSPropertyParserHelpers::consumeFontSize): Deleted. (WebCore::CSSPropertyParserHelpers::consumeFontSizeAdjust): Deleted. (WebCore::CSSPropertyParserHelpers::consumeLineHeight): Deleted. (WebCore::CSSPropertyParserHelpers::consumeScrollPadding): Deleted. (WebCore::CSSPropertyParserHelpers::consumeMaxWidthOrHeight): Deleted. (WebCore::CSSPropertyParserHelpers::consumeWidthOrHeight): Deleted. (WebCore::CSSPropertyParserHelpers::consumeMarginOrOffset): Deleted. (WebCore::CSSPropertyParserHelpers::consumeWebkitLineClamp): Deleted. (WebCore::CSSPropertyParserHelpers::consumeHyphenateLimit): Deleted. (WebCore::CSSPropertyParserHelpers::consumeColumnWidth): Deleted. (WebCore::CSSPropertyParserHelpers::consumeColumnCount): Deleted. (WebCore::CSSPropertyParserHelpers::consumeGapLength): Deleted. (WebCore::CSSPropertyParserHelpers::consumeZoom): Deleted. (WebCore::CSSPropertyParserHelpers::consumeZIndex): Deleted. (WebCore::CSSPropertyParserHelpers::consumeOutlineColor): Deleted. (WebCore::CSSPropertyParserHelpers::consumeLineWidth): Deleted. (WebCore::CSSPropertyParserHelpers::consumeTextStrokeWidth): Deleted. (WebCore::CSSPropertyParserHelpers::consumeColumnRuleWidth): Deleted. (WebCore::CSSPropertyParserHelpers::consumeOpacity): Deleted. (WebCore::CSSPropertyParserHelpers::consumeGlyphOrientationHorizontal): Deleted. (WebCore::CSSPropertyParserHelpers::consumeGlyphOrientationVertical): Deleted. (WebCore::CSSPropertyParserHelpers::consumeFlexBasis): Deleted. (WebCore::CSSPropertyParserHelpers::consumeKerning): Deleted. (WebCore::CSSPropertyParserHelpers::consumeBaselineShift): Deleted. (WebCore::CSSPropertyParserHelpers::consumeRxOrRy): Deleted. (WebCore::CSSPropertyParserHelpers::consumeOverscrollBehavior): Deleted. (WebCore::CSSPropertyParserHelpers::consumeTextUnderlineOffset): Deleted. (WebCore::CSSPropertyParserHelpers::consumeTextDecorationThickness): Deleted. (WebCore::CSSPropertyParserHelpers::consumeVerticalAlign): Deleted. (WebCore::CSSPropertyParserHelpers::consumeWebkitLineGrid): Deleted. (WebCore::CSSPropertyParserHelpers::consumeWebkitMarqueeIncrement): Deleted. (WebCore::CSSPropertyParserHelpers::consumeWebkitMarqueeRepetition): Deleted. (WebCore::CSSPropertyParserHelpers::consumeWebkitMarqueeSpeed): Deleted. (WebCore::CSSPropertyParserHelpers::consumeListStyleType): Deleted. Remove a nice large set of now generated consumer functions! * Source/WebCore/css/parser/CSSPropertyParserHelpers.h: (WebCore::CSSPropertyParserHelpers::identMatches): (WebCore::CSSPropertyParserHelpers::consumeIdentRaw): (WebCore::CSSPropertyParserHelpers::consumeIdent): (WebCore::CSSPropertyParserHelpers::consumeIdentWorkerSafe): (WebCore::CSSPropertyParserHelpers::isFontStyleAngleInRange): (WebCore::CSSPropertyParserHelpers::isSystemFontShorthand): (WebCore::CSSPropertyParserHelpers::lowerFontShorthand): (WebCore::CSSPropertyParserHelpers::createPrimitiveValuePair): Remove the declarations for no longer needed consumers and resort remaining declarations so the inline and template functions are again at the bottom as the comment in the header reads. * Source/WebCore/css/parser/CSSPropertyParserWorkerSafe.cpp: (WebCore::CSSPropertyParserHelpersWorkerSafe::consumeFontFeatureTag): Update more uses of CSSPropertyParserHelpers::consumeIntegerZeroAndGreater(). * Source/WebCore/css/process-css-properties.py: (compact): (compact_map): (flatten): Add a few more helper functions. (Schema.Entry.__init__): (Schema.__add__): (Schema.validate_keys): (Schema.apply_conversions): Add support to Schema for merging two schemas together (via + operator) and for automatically applying a type conversion via a new "convert_to" property on the entries. (Name): (Name.convert_name_to_id): Support a special case list for conversions to allow "url" to be canonicalized as URL instead of Url (needed to keep consumeURL() working without a stub). (ValueKeywordName.from_json): (Status.from_json): (Specification.from_json): Update for new validate_dictionary signature. (Value._build_keyword_term): As a syntatic convenience, we allow properties with only keywords to avoid defining a 'parser-grammar' and use the 'values' array instead. But for processing, it is easier if everthing is a grammar, so we automatically create KeywordTerms for every value. (LogicalPropertyGroup.from_json): (Longhand.from_json): Update for new validate_dictionary signature. (CodeGenProperties): (CodeGenProperties.from_json): Add support for the 'parser-exported', 'parser-grammar' and 'parser-grammar-comment' properties, and remove support for 'partial-keyword-property'. When processing the 'parser-grammar' property we 'perform fixups' on it, which replaces any references to shared grammar rules with the contents of the rule and flattens the final grammar out. (Property): Replace _values_sorted_by_name with _fast_path_keyword_terms_sorted_by_name to better support generation and add a few helper predicates to support selecting the correct generation needed. Additionally, support another convenience, the "<<values>>" (note double arrow-brackets) reference that can be used in a grammar to indicate that the keywords from the properities values array should replace it, via perform_fixups_for_values_references. This allows for existing properties like 'list-style-type' to have a grammar like: ["<<values>>", "<string>"] rather than repeating the entire list of values in the grammar. Eventually, we should likely consolidate the values array and the parser-grammar into a single property, but for now, keeping the values array around reduces changes and allows the other users of CSSProperties.json (like webkit.org) to continue working without changes. (Properties): (Properties.__init__): (Properties._perform_fixups): Move top level parsing out of the Properties class and into a new ParsingContext.TopLevelObject class. But, keep the encapsulated fixup concept in Properties by hoisting it into the init function. (Properties.all_with_fast_path_keyword_terms): Replace another 'values' centric property with a new one that better expresses the real intent, fast-path keyword only property parsing. (Term): Base type for grammar terms. Used to aid parsing of the recursive grammar data structure. (BuiltinSchema): (BuiltinSchema.OptionalParameter): (BuiltinSchema.RequiredParameter): (BuiltinSchema.Entry): (BuiltinSchema.validate_and_construct_if_builtin): The BuiltinSchema class provides support for defining the leaf parser terms; those that currently require calling into WebCore like consumeLength() and consumeColor(). The builtin terms are allowed to define optional and required parameters to augment how the 'consume' function is called. For instance, some have an optional range parameter that looks like '[0,inf]' (to mimic the spec syntax) which corresponds with passing 'ValueRange::NonNegative' to the consume function. Any reference term (that is, any term of the form "<...>") that does not have a shared grammar rule or a builtin defined for it is assumed to have the calling convention of being passed the token range and the parser context. (ReferenceTerm): (ReferenceTerm.__init__): (ReferenceTerm.__str__): (ReferenceTerm.__repr__): (ReferenceTerm.is_reference_string): (ReferenceTerm.from_json): (ReferenceTerm.perform_fixups): (ReferenceTerm.perform_fixups_for_values_references): (ReferenceTerm.is_builtin): Reference terms are strings that start with '<' and end with '>'. They come in a few forms: - Reference to a shared grammar rule defined in the "shared-grammar-rule" section. - Reference to a builtin rule defined as BuiltinSchema above. - Reference to a builtin rule without a BuiltinSchema - Reference to a special. Currently the only special is '<<values>>' which is a constucted reference to the properties 'values' array. At fixup time, any reference that can be resolved is resolved and simplified. (KeywordTerm): (KeywordTerm.__init__): (KeywordTerm.__str__): (KeywordTerm.__repr__): (KeywordTerm.from_json): (KeywordTerm.perform_fixups): (KeywordTerm.perform_fixups_for_values_references): (KeywordTerm.requires_context): (KeywordTerm.is_eligible_for_fast_path): (KeywordTerm.name): Keyword terms match CSSValueID identifiers. In most cases, the match is as simple as that, but if an 'aliased-to' property is defined, the keyword must match but return the other value. This makes the keyword in-eligible for the fast path. (MatchOneTerm): (MatchOneTerm.__init__): (MatchOneTerm.__str__): (MatchOneTerm.__repr__): (MatchOneTerm.from_json): (MatchOneTerm.perform_fixups): (MatchOneTerm.perform_fixups_for_values_references): (MatchOneTerm.is_values_reference): (MatchOneTerm.has_keyword_term): (MatchOneTerm.has_only_keyword_terms): (MatchOneTerm.keyword_terms): (MatchOneTerm.fast_path_keyword_terms): (MatchOneTerm.has_fast_path_keyword_terms): (MatchOneTerm.has_only_fast_path_keyword_terms): Base disjuction type, workhorse of the simple grammar. Contains a list of terms, exactly one of which, must match.0 (Grammar): (Grammar.__init__): (Grammar.__str__): (Grammar.__repr__): (Grammar.from_json): (Grammar.perform_fixups): (Grammar.perform_fixups_for_values_references): (Grammar.has_fast_path_keyword_terms): (Grammar.has_only_keyword_terms): (Grammar.has_only_fast_path_keyword_terms): (Grammar.fast_path_keyword_terms): The grammar is where matching starts. It owns the root term. It is in turn owned by either a Property (via codegen-properties.property-grammar') or by a SharedGrammarRule. (SharedGrammarRule): (SharedGrammarRule.__init__): (SharedGrammarRule.__str__): (SharedGrammarRule.__repr__): (SharedGrammarRule.from_json): (SharedGrammarRule.perform_fixups): (SharedGrammarRules): (SharedGrammarRules.__init__): (SharedGrammarRules.__str__): (SharedGrammarRules.__repr__): (SharedGrammarRules._perform_fixups): (SharedGrammarRules.all): Set of grammar rules that can use as the target of a ReferenceTerm. Used to define a sub-grammar that can be use by multiple properties or by other shared grammar rules. (ParsingContext): (ParsingContext.TopLevelObject): (ParsingContext.__init__): (ParsingContext.parse_shared_grammar_rules): (ParsingContext.parse_properties): (ParsingContext.is_enabled): (ParsingContext.select_enabled_variant): Moved and expanded. Now parses the 'shared-grammar-rules' section and stores them so they can be available when parsing the properties. (GenerationContext): (GenerationContext.__init__): (GenerationContext.generate_heading): (GenerationContext.generate_open_namespace): (GenerationContext.generate_close_namespace): (GenerationContext.generate_using_namespace_declarations): (GenerationContext.generate_property_id_switch_function): (GenerationContext.generate_property_id_switch_function_bool): GenerationContext no longer contains all the generation functions, but rather serves more a similar purpose as ParsingContext, holding onto shared state useful during generation and providing a few helper functions for common generation tasks. The generation functions now live in a set of "Generate*" classes, one for each logical set of files to generate: - GenerateCSSPropertyNames - GenerateCSSPropertyParsing - GenerateCSSStyleDeclarationPropertyNames - GenerateStyleBuilderGenerated - GenerateStylePropertyShorthandFunctions (TermGenerator): To complement the grammar terms, we have TermGenerators which encapsulate code generation for a specific term. (TermGeneratorMatchOneTerm): Owns an builds sub term generators for each sub term of. If the caller passes a KeywordFastPathGenerator that means this a top level generator and any eligible keywords can use a fast path parsing path specialization. (TermGeneratorReferenceTerm): Generator for non-simplied references. These end up being the builtins, so this is where the logic for converting the builtin schema entries into "consume*" functions lives. NOTE: the Builtin*Consumer classes are dynamically created and injected into the global object in BuiltinSchema's Entry class. (TermGeneratorNonFastPathKeywordTerm): Generator for a set of keyword terms that can't use the fast path. At the moment, this is only keywords that have an 'aliased-to' property. (TermGeneratorFastPathKeywordTerms): Generator for a set of keyword terms that can use the fast path. Uses the KeywordFastPathGenerator that it is passed to identify the name of the generated predicate, and uses the predicate taking 'consumeIdent'. (KeywordFastPathGenerator): Extracted to build the 'isKeywordValidFor*' functions used by the keyword-only fast path parser. (SharedGrammarRuleConsumer): The shared grammar rule consumer class cluster is responsible for generating functions for any shared grammar rules that are marked 'exported'. It utilizes the TermGenerators to actually generate the parser text. There are currently only two kinds: - SkipSharedGrammarRuleConsumer (does nothing as you might expect) - GeneratedSharedGrammarRuleConsumer (used for expored rules) (PropertyConsumer): The property consumer class cluster is, like for SharedGrammarRuleConsumer, responsible for generating consume functions, but for CSS properties. It comes in a few more flavors than SharedGrammarRuleConsumer: - SkipPropertyConsumer - CustomPropertyConsumer (used for 'custom-parser' or 'parser-fuction') - FastPathKeywordOnlyPropertyConsumer (used when the main parse function can use the predicate taking consumeIdent function with a fast path predicate directly). - DirectPropertyConsumer (used when a grammar contains only a single non-simplifiable reference term, in which case the main parse function can call it directly). - GeneratedPropertyConsumer (default generator, used for any other variaty of property / grammar). * Tools/Scripts/webkitpy/style/checkers/jsonchecker.py: (JSONCSSPropertiesChecker.check): (JSONCSSPropertiesChecker.check_category): (JSONCSSPropertiesChecker.check_categories): (JSONCSSPropertiesChecker.check_shared_grammar_rule): (JSONCSSPropertiesChecker): (JSONCSSPropertiesChecker.check_shared_grammar_rules): (JSONCSSPropertiesChecker.check_properties): (JSONCSSPropertiesChecker.validate_status_type): (JSONCSSPropertiesChecker.validate_grammar): (JSONCSSPropertiesChecker.validate_grammar_term): (JSONCSSPropertiesChecker.validate_status): (JSONCSSPropertiesChecker.validate_specification): (JSONCSSPropertiesChecker.check_property): (JSONCSSPropertiesChecker.check_codegen_properties): (JSONCSSPropertiesChecker.validate_property_specification): Deleted. Canonical link: https://commits.webkit.org/257026@main
- Loading branch information