Skip to content

Commit

Permalink
Merge pull request #46 from rovio/release_1_0_6
Browse files Browse the repository at this point in the history
Set version release 1.0.6 & the following SNAPSHOT release
  • Loading branch information
vivek-balakrishnan-rovio committed Oct 2, 2023
2 parents 532fd22 + c843705 commit 0cd0725
Show file tree
Hide file tree
Showing 5 changed files with 19 additions and 19 deletions.
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ Set the following spark conf:

```python
.conf("spark.jars.packages",
"com.rovio.ingest:rovio-ingest:1.0.5_spark_3.0.1") \
"com.rovio.ingest:rovio-ingest:1.0.6_spark_3.0.1") \
```

#### PySpark job example
Expand Down Expand Up @@ -206,14 +206,14 @@ A `Dataset[Row]` extension is provided to repartition the dataset for the `Druid
For an interactive spark session you can set the following spark conf:

```scala
("spark.jars.packages", "com.rovio.ingest:rovio-ingest:1.0.5_spark_3.0.1")
("spark.jars.packages", "com.rovio.ingest:rovio-ingest:1.0.6_spark_3.0.1")
```

To use a snapshot version:

```scala
("spark.jars.repositories", "https://s01.oss.sonatype.org/content/repositories/snapshots"),
("spark.jars.packages", "com.rovio.ingest:rovio-ingest:1.0.6_spark_3.0.1-SNAPSHOT")
("spark.jars.packages", "com.rovio.ingest:rovio-ingest:1.0.7_spark_3.0.1-SNAPSHOT")
```

```scala
Expand Down Expand Up @@ -249,7 +249,7 @@ Maven (for a full example, see [examples/rovio-ingest-maven-example](examples/ro
<dependency>
<groupId>com.rovio.ingest</groupId>
<artifactId>rovio-ingest</artifactId>
<version>1.0.5_spark_3.0.1</version>
<version>1.0.6_spark_3.0.1</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
Expand Down
4 changes: 2 additions & 2 deletions examples/rovio-ingest-maven-example/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,9 @@
<dependency>
<groupId>com.rovio.ingest</groupId>
<artifactId>rovio-ingest</artifactId>
<version>1.0.5_spark_3.0.1</version>
<version>1.0.6_spark_3.0.1</version>
<!-- NOTE: This requires the sonatype snapshot repository. See <repositories>.
<version>1.0.6_spark_3.0.1-SNAPSHOT</version>
<version>1.0.7_spark_3.0.1-SNAPSHOT</version>
-->
</dependency>
<dependency>
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

<groupId>com.rovio.ingest</groupId>
<artifactId>rovio-ingest</artifactId>
<version>1.0.6_spark_3.0.1-SNAPSHOT</version>
<version>1.0.7_spark_3.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>rovio-ingest</name>
<description>An implementation of the DatasourceV2 interface of Apache Spark™ for writing Spark Datasets to Apache Druid™</description>
Expand Down
14 changes: 7 additions & 7 deletions python/notebooks/druid_ingestion_test.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -66,14 +66,14 @@
"2. A) Copy the shaded jar to s3:\n",
"\n",
" aws s3 --profile $AWS_PROFILE cp \\\n",
" target/rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/juho/druid/jars/rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar\n",
" target/rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/juho/druid/jars/rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar\n",
"\n",
"2. B) Copy the plain jar to s3: \n",
"\n",
" aws s3 --profile $AWS_PROFILE cp \\\n",
" target/original-rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/juho/druid/jars/original-rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar\n",
" target/original-rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/juho/druid/jars/original-rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar\n",
"\n",
"Then invert the boolean in the cell below to use it in spark_conf."
]
Expand Down Expand Up @@ -200,8 +200,8 @@
" \"spark.sql.session.timeZone\": \"UTC\",\n",
" # alternative if using a snapshot version\n",
"# \"spark.jars.repositories\": \"https://s01.oss.sonatype.org/content/repositories/snapshots\",\n",
"# \"spark.jars.packages\": \"com.rovio.ingest:rovio-ingest:1.0.6_spark_3.0.1-SNAPSHOT\"\n",
" \"spark.jars.packages\": \"com.rovio.ingest:rovio-ingest:1.0.5_spark_3.0.1\"\n",
"# \"spark.jars.packages\": \"com.rovio.ingest:rovio-ingest:1.0.7_spark_3.0.1-SNAPSHOT\"\n",
" \"spark.jars.packages\": \"com.rovio.ingest:rovio-ingest:1.0.6_spark_3.0.1\"\n",
" }\n",
"}\n",
"\n",
Expand All @@ -220,7 +220,7 @@
"# Enable this to test with a manually built & copied jar instead of published package from maven\n",
"if False:\n",
" spark_conf[\"conf\"][\"spark.jars\"] = \\\n",
" f\"s3://{packages_bucket}/{PREFIX}druid/jars/rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar\"\n",
" f\"s3://{packages_bucket}/{PREFIX}druid/jars/rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar\"\n",
" del spark_conf[\"conf\"][\"spark.jars.packages\"]\n",
"\n",
"set_spark_config(spark_conf)\n",
Expand Down
10 changes: 5 additions & 5 deletions python/notebooks/druid_sketch_ingestion_test.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -66,14 +66,14 @@
"2. A) Copy the shaded jar to s3:\n",
"\n",
" aws s3 --profile $AWS_PROFILE cp \\\n",
" target/rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/vivek/druid/jars/rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar\n",
" target/rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/vivek/druid/jars/rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar\n",
"\n",
"2. B) Copy the plain jar to s3: \n",
"\n",
" aws s3 --profile $AWS_PROFILE cp \\\n",
" target/original-rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/vivek/druid/jars/original-rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar\n",
" target/original-rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar \\\n",
" s3://$JAR_BUCKET/tmp/vivek/druid/jars/original-rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar\n",
"\n",
"Then invert the boolean in the cell below to use it in spark_conf."
]
Expand Down Expand Up @@ -166,7 +166,7 @@
"if True:\n",
" jars_base_path = \"s3://{packages_bucket}/{PREFIX}druid/jars\"\n",
" jars = (\n",
" f\"{jars_base_path}/rovio-ingest-1.0.6_spark_3.0.1-SNAPSHOT.jar,\"\n",
" f\"{jars_base_path}/rovio-ingest-1.0.7_spark_3.0.1-SNAPSHOT.jar,\"\n",
" f\"{jars_base_path}/datasketches-hive-1.2.0.jar,\"\n",
" f\"{jars_base_path}/datasketches-java-4.1.0.jar,\"\n",
" f\"{jars_base_path}/datasketches-memory-2.0.0.jar\"\n",
Expand Down

0 comments on commit 0cd0725

Please sign in to comment.