## Summary
This PR adds an optional `WITH TIMEZONE` clause to Spark 3.5 retention
SQL. Statements without the clause continue to use UTC with unchanged
behavior. The backend policy and retention behavior are provided by [PR
#732](https://github.com/linkedin/openhouse/pull/732).
https://github.com/linkedin/openhouse/pull/735 was mistakenly merged
into a branch rather than main.
## Selecting a timezone
This policy keeps the previous 30 whole Pacific calendar days and the
current day so far:
```sql
ALTER TABLE db.events SET POLICY (
RETENTION=30d WITH TIMEZONE 'America/Los_Angeles'
ON COLUMN datepartition WHERE PATTERN='yyyy-MM-dd'
);
```
For a run at `2024-02-01 01:00 UTC`, Pacific time is still January 31.
The policy keeps dates `2024-01-01` through `2024-01-31`; December 31
and earlier are eligible for deletion.
## Using UTC
This statement uses UTC and removes any previously selected timezone:
```sql
ALTER TABLE db.events SET POLICY (
RETENTION=30d
ON COLUMN datepartition WHERE PATTERN='yyyy-MM-dd'
);
```
At the same run time, the UTC policy keeps January 2 through February 1.
January 1 is eligible for deletion because UTC has already reached
February 1.
## Validation
`WITH TIMEZONE` requires `ON COLUMN` naming a string column. Existing
default date patterns apply when `WHERE PATTERN` is omitted: a daily
policy uses `yyyy-MM-dd`.
Invalid timezones, non-string columns, and patterns that already contain
a timezone are rejected before the policy changes. Native timestamp
policies without the clause remain unchanged.
## Testing Done
- [x] The 11 timezone statement-test invocations pass on current `main`.
- [x] The 4 existing retention statement tests pass unchanged.
- [x] Spotless checks pass for the Spark 3.5 runtime and
integration-test modules.
Co-authored-by: mkuchenbecker <mkuchenbecker@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>