Skip to content

v0.17.0

Latest

Choose a tag to compare

@dmitriplotnikov dmitriplotnikov released this 03 Nov 23:29
· 17 commits to master since this release

Release v0.17.0

High-Level Summary

  • Core Evaluator & Value Architecture Overhaul: Replaced virtual step dispatch in the core interpreter loop (evaluator_core.cc) with direct instruction dispatch for constants, identifiers, boolean logic, jumps, comprehensions, list construction, lazy slots, and fast built-ins (==, !=, @in). Made cel::Value, cel::StringValue, cel::BytesValue, and cel::ErrorValue trivial, removed legacy reference-counting/memory-management primitives (cel::MemoryManager, cel::Data, cel::Owner/cel::Borrowed), and removed deprecated recursive program planning and evaluation.
  • Optimized Runtime Builder & Planner Optimizations:
    • Added cel::CreateOptimizedRuntimeBuilder and cel::OptimizedRuntimeOptions() (runtime/optimized_runtime_builder_factory.h) to preconfigure the standard runtime with extended planner and evaluation optimizations.
    • Added set membership (@in) optimizations (extensions::EnableSetMembershipOptimization) using precomputed bitmasks for small integer/enum sets and hash sets for constant string lists.
    • Added trivial lazy subexpression inlining for cel.bind / cel.@block (enable_subexpression_inlining), root identifier slot memoization, and constant folding across inlined constants.
    • Enabled typed field access (enable_typed_field_access) and eagerly resolved field descriptor lookups in select optimization by default, added min_field_depth_for_optimization to SelectOptimizationOptions, and added generated-message fast paths for JSON well-known types.
  • Unknowns & Attribute Tracking Redesign: Switched to a new unknown, missing, and known attribute tracking implementation (AttributeMatcher, AttributeQualifierView, copy-on-write cel::AttributeSet, and an arena-backed red-black tree), supporting in-place cel::UnknownValue updates and skipping unused attribute pattern lookups during execution frame setup.
  • Policy Compiler & Proto Predicate Enhancements: Added support for aggregate rules in the CEL Policy AST, YAML parser, and policy compiler (cel::CompilePolicy), and pruned stale metadata / re-typechecked ASTs after policy unnesting. Extended proto_to_predicate to support match_path annotations on non-repeated primitive fields, maps, and messages.
  • Pratt Parser Maturity & ANTLR-Free Target: Added a pratt_parser build target (EXCLUDE_CEL_ANTLR_PARSER) that excludes the ANTLR4 implementation, along with enriched source info support, configurable repeated unary operator folding, iterative parsing of nested parentheses, recursion depth limits for ternary expressions, and AST node ID/precedence parity fixes with the ANTLR parser.
  • API Modernization & Deprecations: Changed the type checker default to omit function names from the AST reference map, added cel::Source::GetPositionFromByteOffset, cel::Env::GetCompilerOptions, and overload_id on FunctionDescriptor, and updated FieldMask JSON conversion. Deprecated CEL status macros/StatusBuilder in favor of upstream Abseil, decoupled LegacyTypeProvider from cel::TypeProvider, removed LegacyMutationApis and deprecated casting utilities, and dropped the Bazel dependency on flatbuffers.

Detailed Changes

Features

Runtime & Planner Optimizations

  • Add CreateOptimizedRuntimeBuilder and OptimizedRuntimeOptions factory for CEL runtime (#2406)
  • Add set membership (@in) optimization using precomputed bitmasks for small integer/enum sets (#2393)
  • Add string set membership (@in) optimization for constant string lists (#2399)
  • Inline trivial lazy subexpressions (cel.bind / cel.@block) when planning CEL expressions (#2400)
  • Allow constant folding to recognize identifiers replaced with inlined constants (#2401)
  • Optimize select paths for ParsedMessageValue using eagerly resolved field descriptors and enable by default (#2387, #2388)
  • Add min_field_depth_for_optimization option to SelectOptimizationOptions (#2380)
  • Allow planner to accept 3rd/4th arguments on select-optimized ASTs from Java (#2379)
  • Expose resolved types through PlannerContext in FlatExprBuilder (#2370)

Policy & Proto Predicate Compilers

  • Add support for aggregate rules in CEL Policy AST, YAML parser, and policy compiler (cel::CompilePolicy) (#2418, #2422, #2425)
  • Support match_path annotations for non-repeated primitive fields, maps, and messages in proto_to_predicate (#2230)

Handcrafted Pratt Parser

  • [Pratt Parser] Add pratt_parser BUILD target that excludes the ANTLR4 implementation (#2211)
  • [Pratt Parser] Add support for enriched source info (#2207)
  • Add option to enable/disable repeated unary operator folding in parser (#2226)

Language, Compiler & Value APIs

  • Change default type checker behavior to drop function names from the AST reference map (#1878)
  • Add GetPositionFromByteOffset to cel::Source (#2407)
  • Add GetCompilerOptions accessor to cel::Env and make cel::Env / cel::EnvRuntime builder methods const (#2215, #2229)
  • Add cel::StringValue::Literal to create values from compile-time constants (#2279)
  • Add overload_id to FunctionDescriptor (#2378)
  • Support iter_var2 in ComprehensionExpr equality and Clear (#2413)
  • Update MessageToJson conversion for FieldMask in CEL C++ (#2185)
  • Add utility to clone CelValue onto the same or different google::protobuf::Arena (#2284)
  • Add helper for parsing a single test case from YAML (#2268)

Performance and Maintenance

Core Interpreter & Step Dispatch

  • Refactor expression step representation and move core step evaluation into evaluator_core.cc (#2267, #2409, #2410, #2411, #2414, #2417, #2420, #2421)
  • Add dedicated interpreter instructions for constants, identifiers, lazy slots, boolean logic (&&, ||, !, @not_strictly_false), short-circuit jumps, comprehensions, list steps, and fast built-ins (==, !=, @in, #list_append, #mutable_list) (#2286, #2287, #2288, #2289, #2294, #2295, #2296, #2361)
  • Rework function step implementation with dedicated instructions for lazy and eager resolved functions, avoiding unnecessary StatusOr wrapping (#2301, #2325, #2397)
  • Map root identifiers to slots and memoize (#2367)
  • Tune inlining and report terminal errors via Frame::Abort in core interpreter loop (#2302, #2303)
  • Untie CelEvaluationState from the arena and track dirty state in FlatExpressionEvaluatorState to avoid unnecessary resets (#2311, #2392)
  • Cache evaluation option flags in ExecutionFrameBase and tighten bound on value stack estimate (#2293, #2349)
  • Track iterator stack limit separately from comprehension slots (#2366, #2368, #2369)
  • Deprecate and remove recursive program planning and evaluation (#2328, #2341)

Value Representation, Protobuf & Memory Optimizations

  • Make cel::Value, cel::BytesValue, cel::StringValue, and cel::ErrorValue trivial and inline common accessors (#2270, #2280, #2307, #2337, #2339, #2340, #2357)
  • Default opt-in to updated internal reflection and field selection implementations, and re-enable typed field accesses by default (#2217, #2224, #2227, #2235, #2240, #2255, #2374)
  • Add generated-message fast paths for JSON well-known types (#2358)
  • Eliminate temporary std::string copies in CEL protobuf map and Struct lookups (#2327)
  • Unwrap compat and parsed legacy list/map wrappers in FromLegacyList and FromLegacyMap, and avoid per-call scratch arenas in legacy map Has (#2363)
  • Update string and bytes literal formatting to support appending without flattening and optimize BytesValueInputStream / BytesValueOutputStream (#2312, #2313)

Unknowns & Attribute Tracking

  • Switch to new unknown, missing, and known attribute tracking implementation with AttributeMatcher, AttributeQualifierView, copy-on-write cel::AttributeSet, and arena-tailored RBTree (#2275, #2285, #2291, #2298, #2316, #2346, #2347)
  • Allow updating cel::UnknownValue in-place, skip unused attribute pattern lookups when constructing an ExecutionFrame, and update FunctionResultSet to only hold function names (#2281, #2300, #2371)
  • Implement three-way comparison for cel::Attribute and change Activation attribute pattern mutators to return absl::Status (#2335, #2372)

Pratt Parser Optimizations

  • [Pratt Parser] Optimize call stack depth via in-place AST mutation, lean dispatch, and iterative parsing of nested parentheses (#2208, #2209, #2278)
  • [Pratt Parser] Cache source content size in Lexer to avoid repeated variant dispatch in character loops (#2274)

API Cleanup & Modernization

  • Remove cel::Owner, cel::Borrower, cel::Unique, cel::Owned, cel::Borrowed, cel::Data, cel::ReferenceCount, and cel::MemoryManager internals (#2319, #2321, #2322, #2376)
  • Deprecate CEL status macros and StatusBuilder in favor of upstream Abseil versions (#1995)
  • Decouple LegacyTypeProvider and cel::TypeProvider, remove LegacyMutationApis, and remove deprecated CEL casting utilities (#2192, #2195, #2197, #2198, #2199, #2204, #2323)
  • Remove std::shared_ptr implementation in FunctionDescriptor (#2377)
  • Drop Bazel dependency on flatbuffers (#2222)
  • Enable conformance tests for namespace shadowing and lists_ext, and add end-to-end sets extension tests and benchmarks for cel::Runtime (#2201, #2210, #2394, #2398)

Bug Fixes

  • Pass message_factory to reflection->GetMessage in cel::Value::WrapField (#2186, #2191)
  • Fix CEL policy YAML parser to use codepoint positions instead of byte positions (#2179)
  • Prune stale metadata and re-typecheck ASTs after policy unnesting (#2408)
  • Handle TypeKind::kEnum in CEL type inference context (#2424)
  • Fix type unification for type parameters and type checking around comprehensions with empty iteration range (#2260, #2263)
  • Ignore unspecified expression kinds in CEL AST traversal (#2426)
  • Tighten validation and reject unexpected receiver targets in comprehension list append and map insert optimizations (#2350, #2362)
  • Fix truncated list indices in select-optimized field qualification (#2382)
  • Fix bug in optimized regex matching step (#2373)
  • Fix bug in cel::common_internal::ByteString::TryFlat (#2336)
  • Fix bugs in repeated field ::Clone and map field to JSON conversion (#2220, #2221)
  • Update field access behavior to tolerate out-of-range timestamps and propagate unsafe field access for arena-owned messages (#2218, #2237, #2306)
  • Update custom map Find and Has error handling and consistency fixes for legacy adapter types (#2219, #2236)
  • Limit Trace call to Expr IDs in range [0, std::numeric_limits<int32_t>::max()] (#2257)
  • Fix parsing of unescaped newlines in string literals (#2251)
  • [Pratt Parser] Enforce recursion depth limits for ternary expressions, disallow trailing commas in function call argument lists, fix operation precedence following negative numbers (-42.foo), and align AST node IDs and compatibility with the ANTLR parser (#2234, #2238, #2239, #2241, #2272, #2276)
  • Address portability issues and Clang-CL compiler bug workarounds (#2256, #2324)

Cleanups

  • Various automated cleanup changes (internal Piper syncs, clang-tidy readability updates, unused-variable warning fixes, and internal refactors).

Full Changelog: v0.16.0...v0.17.0