[SPARK-54123][PYTHON] Add timezone to make the timestamp an absolute time

[ueshin]

What changes were proposed in this pull request?

Adds timezone to make the timestamp in the log record JSON string the absolute time.

Why are the changes needed?

Without the timezone, the timestamp of each log record doesn't reflect the session timezone, which makes it confusing.

example

>>> from pyspark.sql.functions import *
>>> import logging
>>>
>>> @udf
... def logging_test_udf(x):
...     logger = logging.getLogger("test")
...     logger.warning(f"message")
...     return str(x)
...
>>>
>>> spark.conf.set("spark.sql.pyspark.worker.logging.enabled", True)
>>>
>>> spark.range(1).select(logging_test_udf("id")).show()
...

• Before

>>> spark.conf.get('spark.sql.session.timeZone')
'America/Los_Angeles'
>>> spark.sql("select ts from system.session.python_worker_logs").show(truncate=False)
+--------------------------+
|ts                        |
+--------------------------+
|2025-10-31 17:17:59.495541|
+--------------------------+

>>> spark.conf.set('spark.sql.session.timeZone', 'UTC')
>>> spark.sql("select ts from system.session.python_worker_logs").show(truncate=False)
+--------------------------+
|ts                        |
+--------------------------+
|2025-10-31 17:17:59.495541|
+--------------------------+

• After

>>> spark.conf.get('spark.sql.session.timeZone')
'America/Los_Angeles'
>>> spark.sql("select ts from system.session.python_worker_logs").show(truncate=False)
+--------------------------+
|ts                        |
+--------------------------+
|2025-10-31 17:19:52.152868|
+--------------------------+

>>> spark.conf.set('spark.sql.session.timeZone', 'UTC')
>>> spark.sql("select ts from system.session.python_worker_logs").show(truncate=False)
+--------------------------+
|ts                        |
+--------------------------+
|2025-11-01 00:19:52.152868|
+--------------------------+

Does this PR introduce any user-facing change?

Yes, the timestamp of log record is now absolute time.

How was this patch tested?

Manually.

Was this patch authored or co-authored using generative AI tooling?

No.

[HyukjinKwon]

Merged to master and branch-4.1.

[dongjoon-hyun]

+1, LGTM.