Steps to run the docker file

docker compose up --build

The URLs

For spark logging

http://localhost:8080/

For MiniO

http://127.0.0.1:9001

The most used command inside the docker containers.

docker exec -it spark-iceberg bash

ls /home/spark/scripts

cd /home/spark/scripts

spark-submit iceberg_orc.py

python3 iceberg_orc.py

apt-get update && apt-get install -y iputils-ping

curl http://minio:9000/minio/health/live

ping minio