Spark 3.2 Create Branch Error

[AlexMercedCoder]

To replicate my environment run the following using docker-compose (This is a simple setup I use for quick demonstration, but running into an issue, this setup used to work fine on older versions or spark/iceberg/nessie)

docker-compose.yml

#### Nessie + Iceberg Playground Environment
services:
 spark-iceberg:
   image: alexmerced/nessie-sandbox-072722
   ports:
     - "8080:8080"
     - "7077:7077"
     - "8081:8081"
 nessie:
   image: projectnessie/nessie
   ports:
     - "19120:19120"

• run docker-compose up
• run docker-compose run spark-iceberg /bin/bash
• then open up SparkSQL with the command below

spark-sql --packages org.apache.iceberg:iceberg-spark-runtime-3.2_2.12:0.13.0,org.projectnessie:nessie-spark-3.2-extensions:0.40.1 --conf spark.sql.extensions="org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,org.projectnessie.spark.extensions.NessieSpark32SessionExtensions" --conf spark.sql.catalog.nessie.uri="http://nessie:19120/api/v1" -- conf spark.sql.catalog.nessie.ref=main  -- conf spark.sql.catalog.nessie.authentication.type=NONE --conf spark.sql.catalog.nessie.catalog-impl=org.apache.iceberg.nessie.NessieCatalog --conf spark.sql.catalog.nessie=org.apache.iceberg.spark.SparkCatalog --conf spark.sql.catalog.nessie.warehouse=$PWD/warehouse

• I am able to create, insert and query a table just fine
• LIST REFERENCES works just fine
• but when I start using CREATE BRANCH I get this:

spark-sql> CREATE BRANCH IF NOT EXISTS hello IN nessie;
22/07/27 22:55:50 ERROR SparkSQLDriver: Failed in [CREATE BRANCH IF NOT EXISTS hello IN nessie]
java.util.NoSuchElementException: spark.sql.catalog.nessie.ref
        at org.apache.spark.SparkConf.$anonfun$get$1(SparkConf.scala:245)
        at scala.Option.getOrElse(Option.scala:189)
        at org.apache.spark.SparkConf.get(SparkConf.scala:245)
        at org.apache.spark.sql.execution.datasources.v2.NessieUtils$.getCurrentRefName(NessieUtils.scala:266)
        at org.apache.spark.sql.execution.datasources.v2.NessieUtils$.getCurrentRef(NessieUtils.scala:247)
        at org.apache.spark.sql.execution.datasources.v2.BaseCreateReferenceExec.runInternal(BaseCreateReferenceExec.scala:46)
        at org.apache.spark.sql.execution.datasources.v2.NessieExec.run(NessieExec.scala:34)

[dimas-b]

@AlexMercedCoder : The SQL extension involved in this case is part of the Nessie repo. It might be preferable to move this issue to https://github.com/projectnessie/nessie/issues :)

[AlexMercedCoder]

oops, that's where I meant to post, lol, will, repost there

projectnessie/nessie#4801