From 9dde724eb5fcbe14574a0130436de4c086031ef0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dominik=20D=C4=99bowczyk?= Date: Mon, 8 Jul 2024 16:25:17 +0200 Subject: [PATCH] Update Spark development setup docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Dominik Dębowczyk --- docs/development/developing/spark/setup.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/docs/development/developing/spark/setup.md b/docs/development/developing/spark/setup.md index d2c1f7a04f..51b7863292 100644 --- a/docs/development/developing/spark/setup.md +++ b/docs/development/developing/spark/setup.md @@ -13,7 +13,7 @@ Testing requires a Java 8 JVM to test the Scala Spark components. ## Preparation -The integration depends on three libraries that are build locally `openlineage-java`, `spark-interfaces-scala` and `openlineage-sql-java`, +The integration depends on four libraries that are build locally `openlineage-java`, `spark-extension-interfaces`, `spark-extension-entrypoint` and `openlineage-sql-java`, so before any testing or building of a package you need to publish the appropriate artifacts to local maven repository. To build the packages you need to execute. @@ -22,9 +22,14 @@ To install `openlineage-java` in local maven repo run: cd ../../client/java/ && ./gradlew publishToMavenLocal ``` -For `spark-interfaces-scala` run: +For `spark-extension-interfaces` run: ```sh -cd ../../integration/spark-interfaces-scala && ./gradlew publishToMavenLocal -Pscala.binary.version=2.12 +cd ../../integration/spark-extension-interfaces && ./gradlew publishToMavenLocal +``` + +For `spark-extension-entrypoint` run: +```sh +cd ../../integration/spark-extension-entrypoint && ./gradlew publishToMavenLocal ``` For `openlineage-sql-java` run: