Repository navigation
·
17 commits
to master
since this release
Release v0.17.0
High-Level Summary
- Core Evaluator & Value Architecture Overhaul: Replaced virtual step dispatch in the core interpreter loop (
evaluator_core.cc) with direct instruction dispatch for constants, identifiers, boolean logic, jumps, comprehensions, list construction, lazy slots, and fast built-ins (==,!=,@in). Madecel::Value,cel::StringValue,cel::BytesValue, andcel::ErrorValuetrivial, removed legacy reference-counting/memory-management primitives (cel::MemoryManager,cel::Data,cel::Owner/cel::Borrowed), and removed deprecated recursive program planning and evaluation. - Optimized Runtime Builder & Planner Optimizations:
- Added
cel::CreateOptimizedRuntimeBuilderandcel::OptimizedRuntimeOptions()(runtime/optimized_runtime_builder_factory.h) to preconfigure the standard runtime with extended planner and evaluation optimizations. - Added set membership (
@in) optimizations (extensions::EnableSetMembershipOptimization) using precomputed bitmasks for small integer/enum sets and hash sets for constant string lists. - Added trivial lazy subexpression inlining for
cel.bind/cel.@block(enable_subexpression_inlining), root identifier slot memoization, and constant folding across inlined constants. - Enabled typed field access (
enable_typed_field_access) and eagerly resolved field descriptor lookups in select optimization by default, addedmin_field_depth_for_optimizationtoSelectOptimizationOptions, and added generated-message fast paths for JSON well-known types.
- Added
- Unknowns & Attribute Tracking Redesign: Switched to a new unknown, missing, and known attribute tracking implementation (
AttributeMatcher,AttributeQualifierView, copy-on-writecel::AttributeSet, and an arena-backed red-black tree), supporting in-placecel::UnknownValueupdates and skipping unused attribute pattern lookups during execution frame setup. - Policy Compiler & Proto Predicate Enhancements: Added support for
aggregaterules in the CEL Policy AST, YAML parser, and policy compiler (cel::CompilePolicy), and pruned stale metadata / re-typechecked ASTs after policy unnesting. Extendedproto_to_predicateto supportmatch_pathannotations on non-repeated primitive fields, maps, and messages. - Pratt Parser Maturity & ANTLR-Free Target: Added a
pratt_parserbuild target (EXCLUDE_CEL_ANTLR_PARSER) that excludes the ANTLR4 implementation, along with enriched source info support, configurable repeated unary operator folding, iterative parsing of nested parentheses, recursion depth limits for ternary expressions, and AST node ID/precedence parity fixes with the ANTLR parser. - API Modernization & Deprecations: Changed the type checker default to omit function names from the AST reference map, added
cel::Source::GetPositionFromByteOffset,cel::Env::GetCompilerOptions, andoverload_idonFunctionDescriptor, and updatedFieldMaskJSON conversion. Deprecated CEL status macros/StatusBuilderin favor of upstream Abseil, decoupledLegacyTypeProviderfromcel::TypeProvider, removedLegacyMutationApisand deprecated casting utilities, and dropped the Bazel dependency onflatbuffers.
Detailed Changes
Features
Runtime & Planner Optimizations
- Add
CreateOptimizedRuntimeBuilderandOptimizedRuntimeOptionsfactory for CEL runtime (#2406) - Add set membership (
@in) optimization using precomputed bitmasks for small integer/enum sets (#2393) - Add string set membership (
@in) optimization for constant string lists (#2399) - Inline trivial lazy subexpressions (
cel.bind/cel.@block) when planning CEL expressions (#2400) - Allow constant folding to recognize identifiers replaced with inlined constants (#2401)
- Optimize select paths for
ParsedMessageValueusing eagerly resolved field descriptors and enable by default (#2387, #2388) - Add
min_field_depth_for_optimizationoption toSelectOptimizationOptions(#2380) - Allow planner to accept 3rd/4th arguments on select-optimized ASTs from Java (#2379)
- Expose resolved types through
PlannerContextinFlatExprBuilder(#2370)
Policy & Proto Predicate Compilers
- Add support for
aggregaterules in CEL Policy AST, YAML parser, and policy compiler (cel::CompilePolicy) (#2418, #2422, #2425) - Support
match_pathannotations for non-repeated primitive fields, maps, and messages inproto_to_predicate(#2230)
Handcrafted Pratt Parser
- [Pratt Parser] Add
pratt_parserBUILD target that excludes the ANTLR4 implementation (#2211) - [Pratt Parser] Add support for enriched source info (#2207)
- Add option to enable/disable repeated unary operator folding in parser (#2226)
Language, Compiler & Value APIs
- Change default type checker behavior to drop function names from the AST reference map (#1878)
- Add
GetPositionFromByteOffsettocel::Source(#2407) - Add
GetCompilerOptionsaccessor tocel::Envand makecel::Env/cel::EnvRuntimebuilder methodsconst(#2215, #2229) - Add
cel::StringValue::Literalto create values from compile-time constants (#2279) - Add
overload_idtoFunctionDescriptor(#2378) - Support
iter_var2inComprehensionExprequality andClear(#2413) - Update
MessageToJsonconversion forFieldMaskin CEL C++ (#2185) - Add utility to clone
CelValueonto the same or differentgoogle::protobuf::Arena(#2284) - Add helper for parsing a single test case from YAML (#2268)
Performance and Maintenance
Core Interpreter & Step Dispatch
- Refactor expression step representation and move core step evaluation into
evaluator_core.cc(#2267, #2409, #2410, #2411, #2414, #2417, #2420, #2421) - Add dedicated interpreter instructions for constants, identifiers, lazy slots, boolean logic (
&&,||,!,@not_strictly_false), short-circuit jumps, comprehensions, list steps, and fast built-ins (==,!=,@in,#list_append,#mutable_list) (#2286, #2287, #2288, #2289, #2294, #2295, #2296, #2361) - Rework function step implementation with dedicated instructions for lazy and eager resolved functions, avoiding unnecessary
StatusOrwrapping (#2301, #2325, #2397) - Map root identifiers to slots and memoize (#2367)
- Tune inlining and report terminal errors via
Frame::Abortin core interpreter loop (#2302, #2303) - Untie
CelEvaluationStatefrom the arena and track dirty state inFlatExpressionEvaluatorStateto avoid unnecessary resets (#2311, #2392) - Cache evaluation option flags in
ExecutionFrameBaseand tighten bound on value stack estimate (#2293, #2349) - Track iterator stack limit separately from comprehension slots (#2366, #2368, #2369)
- Deprecate and remove recursive program planning and evaluation (#2328, #2341)
Value Representation, Protobuf & Memory Optimizations
- Make
cel::Value,cel::BytesValue,cel::StringValue, andcel::ErrorValuetrivial and inline common accessors (#2270, #2280, #2307, #2337, #2339, #2340, #2357) - Default opt-in to updated internal reflection and field selection implementations, and re-enable typed field accesses by default (#2217, #2224, #2227, #2235, #2240, #2255, #2374)
- Add generated-message fast paths for JSON well-known types (#2358)
- Eliminate temporary
std::stringcopies in CEL protobuf map andStructlookups (#2327) - Unwrap compat and parsed legacy list/map wrappers in
FromLegacyListandFromLegacyMap, and avoid per-call scratch arenas in legacy mapHas(#2363) - Update string and bytes literal formatting to support appending without flattening and optimize
BytesValueInputStream/BytesValueOutputStream(#2312, #2313)
Unknowns & Attribute Tracking
- Switch to new unknown, missing, and known attribute tracking implementation with
AttributeMatcher,AttributeQualifierView, copy-on-writecel::AttributeSet, and arena-tailored RBTree (#2275, #2285, #2291, #2298, #2316, #2346, #2347) - Allow updating
cel::UnknownValuein-place, skip unused attribute pattern lookups when constructing anExecutionFrame, and updateFunctionResultSetto only hold function names (#2281, #2300, #2371) - Implement three-way comparison for
cel::Attributeand changeActivationattribute pattern mutators to returnabsl::Status(#2335, #2372)
Pratt Parser Optimizations
- [Pratt Parser] Optimize call stack depth via in-place AST mutation, lean dispatch, and iterative parsing of nested parentheses (#2208, #2209, #2278)
- [Pratt Parser] Cache source content size in Lexer to avoid repeated variant dispatch in character loops (#2274)
API Cleanup & Modernization
- Remove
cel::Owner,cel::Borrower,cel::Unique,cel::Owned,cel::Borrowed,cel::Data,cel::ReferenceCount, andcel::MemoryManagerinternals (#2319, #2321, #2322, #2376) - Deprecate CEL status macros and
StatusBuilderin favor of upstream Abseil versions (#1995) - Decouple
LegacyTypeProviderandcel::TypeProvider, removeLegacyMutationApis, and remove deprecated CEL casting utilities (#2192, #2195, #2197, #2198, #2199, #2204, #2323) - Remove
std::shared_ptrimplementation inFunctionDescriptor(#2377) - Drop Bazel dependency on
flatbuffers(#2222) - Enable conformance tests for namespace shadowing and
lists_ext, and add end-to-endsetsextension tests and benchmarks forcel::Runtime(#2201, #2210, #2394, #2398)
Bug Fixes
- Pass
message_factorytoreflection->GetMessageincel::Value::WrapField(#2186, #2191) - Fix CEL policy YAML parser to use codepoint positions instead of byte positions (#2179)
- Prune stale metadata and re-typecheck ASTs after policy unnesting (#2408)
- Handle
TypeKind::kEnumin CEL type inference context (#2424) - Fix type unification for type parameters and type checking around comprehensions with empty iteration range (#2260, #2263)
- Ignore unspecified expression kinds in CEL AST traversal (#2426)
- Tighten validation and reject unexpected receiver targets in comprehension list append and map insert optimizations (#2350, #2362)
- Fix truncated list indices in select-optimized field qualification (#2382)
- Fix bug in optimized regex matching step (#2373)
- Fix bug in
cel::common_internal::ByteString::TryFlat(#2336) - Fix bugs in repeated field
::Cloneand map field to JSON conversion (#2220, #2221) - Update field access behavior to tolerate out-of-range timestamps and propagate unsafe field access for arena-owned messages (#2218, #2237, #2306)
- Update custom map
FindandHaserror handling and consistency fixes for legacy adapter types (#2219, #2236) - Limit
Tracecall toExprIDs in range[0, std::numeric_limits<int32_t>::max()](#2257) - Fix parsing of unescaped newlines in string literals (#2251)
- [Pratt Parser] Enforce recursion depth limits for ternary expressions, disallow trailing commas in function call argument lists, fix operation precedence following negative numbers (
-42.foo), and align AST node IDs and compatibility with the ANTLR parser (#2234, #2238, #2239, #2241, #2272, #2276) - Address portability issues and Clang-CL compiler bug workarounds (#2256, #2324)
Cleanups
- Various automated cleanup changes (internal Piper syncs, clang-tidy readability updates, unused-variable warning fixes, and internal refactors).
Full Changelog: v0.16.0...v0.17.0