0.69

Spark data source for reading Esri File Geodatabases.

Artifacts

Jar	Spark	Scala	Java
filegdb-0.69-3.5-2.12.jar	3.5.9	2.12.21	8+
filegdb-0.69-4.3-2.13.jar	4.3.0	2.13.18	17+

These artifacts are not on Maven Central.

Changes since 0.68

• New sortByOffset option reads each partition in file order instead of OBJECTID order. It defaults to off. On edited tables, local-disk scans are about 5× faster, and S3 scans on Spark 3.5 go from more than 5 minutes to about 8 seconds. See the README sections "Reading in file order" and "Benchmark".
• Spark 4.3 replaces Spark 4.0 as the Spark 4 build line (-Pspark-4.3). The Spark 3.3 and 3.4 profiles are gone. This affects the build only.
• smoke_test.py uses the JVM of the pyspark it imports, loads the jar for that Spark line, reads s3a:// paths, and binds the driver to loopback. Verified on Miami, World and NorthSea with pyspark 3.5.9 and 4.3.0.

Install into your local Maven repository

mvn install:install-file -Dfile=filegdb-0.69-4.3-2.13.jar \
  -DgroupId=com.esri -DartifactId=filegdb -Dversion=0.69-4.3-2.13 -Dpackaging=jar

Usage

Registers a gdb Spark data source — options path, name, numPartitions, wkid, sortByOffset:

df = spark.read.format("gdb").option("path", "X.gdb").option("name", "LayerName").load()

Points come back as Shape: struct<x, y>; lines and polygons as struct<xmin, ymin, xmax, ymax, parts, coords>.