[SPARK-53933] Add Apache Iceberg example

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to add Apache Iceberg example.

Why are the changes needed?

To provide an working example of Apache Spark 4.0.1 and Apache Iceberg 1.10.0.

1. Prepare the storage

$ kubectl apply -f examples/localstack.ym

2. Launch Spark Connect Server with Apache Iceberg setting.

$ kubectl apply -f examples/spark-connect-server-iceberg.yaml

3. Setup port-forwarding to test

$ kubectl port-forward spark-connect-server-iceberg-0-driver 15002

4. Test with Apache Iceberg Spark Quickstart guideline.

• https://iceberg.apache.org/spark-quickstart/

$ bin/spark-connect-shell --remote sc://localhost:15002

scala> sql("""CREATE TABLE taxis(vendor_id bigint, trip_id bigint, trip_distance float, fare_amount double, store_and_fwd_flag string) PARTITIONED BY (vendor_id);""").show()

scala> sql("""INSERT INTO taxis VALUES (1, 1000371, 1.8, 15.32, 'N'), (2, 1000372, 2.5, 22.15, 'N'), (2, 1000373, 0.9, 9.01, 'N'), (1, 1000374, 8.4, 42.13, 'Y');""").show()

scala> sql("SELECT * FROM taxis").show(false)
+---------+-------+-------------+-----------+------------------+
|vendor_id|trip_id|trip_distance|fare_amount|store_and_fwd_flag|
+---------+-------+-------------+-----------+------------------+
|1        |1000374|8.4          |42.13      |Y                 |
|1        |1000371|1.8          |15.32      |N                 |
|2        |1000372|2.5          |22.15      |N                 |
|2        |1000373|0.9          |9.01       |N                 |
+---------+-------+-------------+-----------+------------------+

scala> sql("SELECT * FROM taxis.history").show(false)
+-----------------------+-------------------+---------+-------------------+
|made_current_at        |snapshot_id        |parent_id|is_current_ancestor|
+-----------------------+-------------------+---------+-------------------+
|2025-10-16 03:53:04.063|6463217948421571140|NULL     |true               |
+-----------------------+-------------------+---------+-------------------+

scala> sql("SELECT * FROM taxis VERSION AS OF 6463217948421571140").show(false)
+---------+-------+-------------+-----------+------------------+
|vendor_id|trip_id|trip_distance|fare_amount|store_and_fwd_flag|
+---------+-------+-------------+-----------+------------------+
|1        |1000374|8.4          |42.13      |Y                 |
|1        |1000371|1.8          |15.32      |N                 |
|2        |1000372|2.5          |22.15      |N                 |
|2        |1000373|0.9          |9.01       |N                 |
+---------+-------+-------------+-----------+------------------+

5. Check the data in the storage.

root@localstack:/opt/code/localstack# awslocal s3 ls s3://warehouse/ --recursive
2025-10-16 03:53:03       1545 taxis/data/vendor_id=1/00000-3-749fe2e5-bbe3-4f0e-b976-a21749550705-0-00002.parquet
2025-10-16 03:53:03       1590 taxis/data/vendor_id=2/00000-3-749fe2e5-bbe3-4f0e-b976-a21749550705-0-00001.parquet
2025-10-16 03:53:04       7559 taxis/metadata/9f629d40-ce91-4822-aeee-283d53ec5ef6-m0.avro
2025-10-16 03:53:04       4446 taxis/metadata/snap-6463217948421571140-1-9f629d40-ce91-4822-aeee-283d53ec5ef6.avro
2025-10-16 03:52:47       1006 taxis/metadata/v1.metadata.json
2025-10-16 03:53:04       1970 taxis/metadata/v2.metadata.json
2025-10-16 03:53:04          1 taxis/metadata/version-hint.text

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manually tested.

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

cc @viirya , @peter-toth , @jiangzho

[dongjoon-hyun]

Thank you, @viirya . Merged to main.

[peter-toth]

Late LGTM.

[dongjoon-hyun]

Thank you, @peter-toth .