GitHub - RTIInternational/teehr-spark-iceberg: A repository to hold code related to testing Spark and Iceberg for use in the TEEHR system.

RTIInternational / teehr-spark-iceberg Public

Notifications You must be signed in to change notification settings
Fork 0
Star 0

A repository to hold code related to testing Spark and Iceberg for use in the TEEHR system.

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 4 Commits
autoscaler		autoscaler
cert-manager		cert-manager
eksctl		eksctl
iceberg-rest		iceberg-rest
ingress		ingress
kind		kind
minio		minio
notebooks		notebooks
scripts		scripts
secrets		secrets
spark-iceberg		spark-iceberg
x-spark-iceberg-old		x-spark-iceberg-old
x-spark-operator		x-spark-operator
x-spark		x-spark
x-workflows		x-workflows
.gitignore		.gitignore
README		README
docker-compose.yml		docker-compose.yml
notes.md		notes.md
project.garden.yml		project.garden.yml

Repository files navigation

This repo contains files related to our testing for spark and iceberg for the TEEHR evaluation system.

To work with Spark we created a Kubernetes cluster to run the Spark Executors and Drivers as well as host the Iceberg rest server and Minio bucket for these tests.  The steps taken to create the cluster are shown in `eksctl/README.md`.  The notebooks for the test are in `notebooks`.