feat(trino): parse routine characteristics for inline UDFs [CLAUDE]

[rusackas]

I'm back! :)

PR 2 of 7-ish, continuing to build on #7934.

Adds parsing/generation for the routine characteristics that can follow a Trino inline WITH FUNCTION UDF's RETURNS clause: LANGUAGE, DETERMINISTIC/NOT DETERMINISTIC, CALLED ON NULL INPUT/RETURNS NULL ON NULL INPUT, SECURITY DEFINER/INVOKER, and COMMENT (https://trino.io/docs/current/udf/sql.html):

WITH FUNCTION custom_sqrt(a integer)
  RETURNS double
  COMMENT 'Custom sqrt function'
  RETURNS NULL ON NULL INPUT
  NOT DETERMINISTIC
  LANGUAGE SQL
  SECURITY DEFINER
  RETURN a
SELECT custom_sqrt(4)

Most of this already worked for free: _parse_function_specification (from #7934) calls the base _parse_properties() between RETURNS and RETURN, so LANGUAGE, CALLED/RETURNS NULL ON NULL INPUT, SECURITY DEFINER/INVOKER, and COMMENT all parsed and generated correctly with zero new code, via existing exp.LanguageProperty/exp.CalledOnNullInputProperty/exp.SqlSecurityProperty/exp.SchemaCommentProperty + their existing generic generators.

Two things actually needed fixing:

1. DETERMINISTIC/NOT DETERMINISTIC — reused exp.StabilityProperty (base already parses bare DETERMINISTIC into it), and mirrored BigQuery's existing StabilityProperty handling exactly: a NOT DETERMINISTIC entry in TrinoParser.PROPERTY_PARSERS, the same TrinoGenerator.TRANSFORMS lambda BigQuery uses, and the same tokenizer-level "NOT DETERMINISTIC" keyword-merge BigQuery has (_match_texts only inspects one token, so the two words need to merge into one before the parser can dispatch on them). Before this, bare DETERMINISTIC round-tripped as the invalid-for-Trino IMMUTABLE, and NOT DETERMINISTIC didn't parse at all.

2. LANGUAGE SQL immediately followed by a SECURITY clause — the base tokenizer already merges SQL SECURITY into one TokenType.SQL_SECURITY token (for MySQL/StarRocks' ... SQL SECURITY DEFINER VIEW), which greedily ate the SQL that was supposed to be LANGUAGE's value, e.g. LANGUAGE SQL SECURITY DEFINER tokenized as LANGUAGE / SQL SECURITY / DEFINER instead of LANGUAGE / SQL / SECURITY / DEFINER. Trino has no SQL SECURITY phrase in its own grammar (only bare SECURITY DEFINER/INVOKER), and the base parser's PROPERTY_PARSERS already maps plain "SECURITY" to the identical _parse_sql_security the merged token maps to, so Trino.Tokenizer just pops the merged keyword (same idiom as its existing KEYWORDS.pop("/*+") on the Presto side, or BigQuery's KEYWORDS.pop("DIV")). Confirmed this doesn't affect MySQL/StarRocks, whose tokenizers are untouched.

Next steps (remaining follow-up PRs, each gated on the previous)

1. Core WITH FUNCTION ... RETURNS ... RETURN ... (feat(trino): parse WITH FUNCTION ... RETURNS ... RETURN inline UDFs [CLAUDE] #7934)
2. This PR — routine characteristics
3. BEGIN...END block bodies + DECLARE/SET, including the semicolon/chunk-continuation handling so routine bodies don't get split as separate statements
4. IF/ELSEIF/ELSE — reusing/extending exp.IfBlock
5. CASE...WHEN...END CASE
6. WHILE...DO...END WHILE — reusing/extending exp.WhileBlock with a label arg
7. LOOP/REPEAT/ITERATE/LEAVE

Related: apache/superset#26162 (Superset issue asking for Trino inline SQL UDF support end-to-end).

Disclosure: implemented with Claude Code, reviewed, tested (make unit, make style) and understood by me.

[treysp]

Thanks for the PR, it's looking good!

I noticed that the Trino UDF docs show an optional WITH () clause for specifying properties:

FUNCTION name ( [ parameter_name data_type [, ...] ] )
  RETURNS type
  [ LANGUAGE language]
  [ NOT? DETERMINISTIC ]
  [ RETURNS NULL ON NULL INPUT ]
  [ CALLED ON NULL INPUT ]
  [ SECURITY { DEFINER | INVOKER } ]
  [ COMMENT description]
  [ WITH ( property_name = expression [, ...] ) ]   <-------------- This guy
  { statements | AS definition }

Let's include that in this PR so we have the full spec covered.

Annoyingly, base sqlglot property handling expects either all or none of the properties to be wrapped in WITH (). Since these UDFs can have both, we'll need parse WITH separately from the other characteristics in _parse_function_specification.

So the parsing would happen in three steps: this, characteristics/properties, expression. The characteristics/properties parsing would look something like this:

       characteristics = []                                                                                             
       function_properties = []                                                                                         
                                                                                                                        
       while True:                                                                                                      
           if self._match(TokenType.WITH):                                                                              
               function_properties.extend(                                                                              
                   self._parse_wrapped_csv(self._parse_key_value_property)                                              
               )                                                                                                        
               continue                                                                                                 
                                                                                                                        
           characteristic = self._parse_property()                                                                      
           if not characteristic:                                                                                       
               break

           characteristics.extend(ensure_list(characteristic))                                                                                                    

We'll also need to add an arg to the FunctionSpecification expression so we can store the two sets separately. How about adding a new arg characteristics to store the non-WITH and existing arg properties to store the WITH?

Then, in the generator, generate the two sets separately. We won't always be able to retain the original order - it will always be WITH at the end. But that doesn't affect the semantics so should be fine.

[rusackas]

Addressed in 87bd0a1. Split the routine characteristics from the trailing WITH (...) clause as suggested: _parse_function_specification now loops _parse_property() for the fixed-keyword characteristics and _parse_key_value_property() (via _parse_wrapped_csv) for WITH's arbitrary key/value pairs, stopping only when neither branch matches. FunctionSpecification picked up a new characteristics arg for the former; the existing properties arg now holds the WITH set. Generator renders WITH (...) last via the existing with_properties() helper, so it always lands right before the body regardless of where it appeared in the source. Added test coverage for WITH (...) alone, combined with other characteristics, and combined with the full characteristics set from the earlier example.

[rusackas]

Thanks for the review @treysp - will open the next item in the lineup once this merges.

Or, if y'all want to try the new Github PR Stack feature, this might be a fun excuse to do so.

[georgesittas]

There are several invalid tests, because we use catalog-stored UDF syntax in inline UDF declarations, which don't seem to support the full grammar spec.

If the machinery is reused for the former category of statements (catalog-stored UDFs), then it may be helpful adding tests for them and/or cleaning up existing invalid tests to ensure they represent runnable workloads.

Other than that the approach looks good.

[rusackas]

Thanks for the thorough review, I think everything is addressed. Thanks also for actually running these against Trino, @georgesittas, it caught some legit issues:

• SELECT NOT deterministic FROM t was misparsing into a single bare column named "NOT DETERMINISTIC" instead of NOT applied to a column, because of the tokenizer-level keyword merge. Fixed by moving that check into TrinoParser._parse_property() (a two-token _match_text_seq) instead, so NOT is never swallowed outside of a FunctionSpecification. Scoped to Trino; flagged that BigQuery has the same tokenizer-merge pattern but a cross-dialect cleanup felt out of scope here.
• Confirmed Athena inherits both the SQL SECURITY fix and this new one automatically, since its tokenizer builds off Trino.Tokenizer.KEYWORDS by reference.
• Added a comment clarifying the SECURITY / WITH (...) test cases assert grammar round-tripping against Trino's documented function-specification grammar, not that this exact combination runs on an inline LANGUAGE SQL UDF (confirmed by your Trino output). Left the underlying parsing/generation in place since it's generic reuse that should carry over if catalog-stored CREATE FUNCTION support gets added later.
• Trimmed/reworded the two comments flagged as unclear or inaccurate.

Full unit suite (1257 tests) plus ruff/ruff-format green.

[georgesittas]

Thank you!