Skip to content

Commit

Permalink
Revert artifact IDs for Hadoop and Spark jars (#98)
Browse files Browse the repository at this point in the history
Revert the artifact IDs to tensorflow-hadoop and tensorflow-spark-connector
  • Loading branch information
skavulya authored and jhseu committed Aug 22, 2018
1 parent a310dda commit 76a7cd0
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 19 deletions.
7 changes: 4 additions & 3 deletions hadoop/README.md
Expand Up @@ -11,8 +11,9 @@ This can also be used with [Apache Spark](http://spark.apache.org/).
2. Tested with Hadoop 2.6.0. Patches are welcome if there are incompatibilities
with your Hadoop version.

## Changelog
## Breaking changes

* 08/20/2018 - Reverted artifactId back to `org.tensorflow.tensorflow-hadoop`
* 05/29/2018 - Changed the artifactId from `org.tensorflow.tensorflow-hadoop` to `org.tensorflow.hadoop`

## Build and install
Expand Down Expand Up @@ -41,8 +42,8 @@ This can also be used with [Apache Spark](http://spark.apache.org/).
```xml
<dependency>
<groupId>org.tensorflow</groupId>
<artifactId>hadoop</artifactId>
<version>1.8.0</version>
<artifactId>tensorflow-hadoop</artifactId>
<version>1.10.0</version>
</dependency>
```

Expand Down
6 changes: 3 additions & 3 deletions hadoop/pom.xml
Expand Up @@ -3,9 +3,9 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.tensorflow</groupId>
<artifactId>hadoop</artifactId>
<artifactId>tensorflow-hadoop</artifactId>
<packaging>jar</packaging>
<version>1.9.0-rc2</version>
<version>1.10.0</version>
<name>tensorflow-hadoop</name>
<url>https://www.tensorflow.org</url>
<description>TensorFlow TFRecord InputFormat/OutputFormat for Apache Hadoop</description>
Expand All @@ -15,7 +15,7 @@
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
<hadoop.version>2.6.0</hadoop.version>
<protobuf.version>3.3.1</protobuf.version>
<protobuf.version>3.5.1</protobuf.version>
<junit.version>4.11</junit.version>
</properties>

Expand Down
18 changes: 9 additions & 9 deletions spark/spark-tensorflow-connector/README.md
Expand Up @@ -6,11 +6,11 @@ into Spark SQL DataFrames, and data export from DataFrames to TensorFlow records

The library supports both the Scala and PySpark APIs. See [Usage examples](#usage-examples) for sample PySpark and Scala code.

## Changelog
## Breaking changes

* 08/20/2018 - Reverted artifactId back to `org.tensorflow.spark-tensorflow-connector`
* 05/29/2018 - Changed the artifactId from `org.tensorflow.spark-tensorflow-connector` to `org.tensorflow.spark-connector`


## Prerequisites

1. [Apache Spark 2.0 (or later)](http://spark.apache.org/)
Expand Down Expand Up @@ -54,19 +54,19 @@ mvn clean install -Dspark.version=2.2.1

After installation (or deployment), the package can be used with the following dependency:

```
```xml
<dependency>
<groupId>org.tensorflow</groupId>
<artifactId>spark-connector_2.11</artifactId>
<version>1.8.0</version>
<artifactId>spark-tensorflow-connector_2.11</artifactId>
<version>1.10.0</version>
</dependency>
```

## Using Spark Shell
Run this library in Spark using the `--jars` command line option in `spark-shell`, `pyspark` or `spark-submit`. For example:

```sh
$SPARK_HOME/bin/spark-shell --jars target/spark-connector_2.11-1.8.0.jar
$SPARK_HOME/bin/spark-shell --jars target/spark-tensorflow-connector_2.11-1.10.0.jar
```

## Features
Expand Down Expand Up @@ -138,7 +138,7 @@ The supported Spark data types are listed in the table below:
### Python API
Run PySpark with the spark_connector in the jars argument as shown below:

`$SPARK_HOME/bin/pyspark --jars target/spark-connector_2.11-1.8.0.jar`
`$SPARK_HOME/bin/pyspark --jars target/spark-tensorflow-connector_2.11-1.10.0.jar`

The following Python code snippet demonstrates usage on test data.

Expand All @@ -164,7 +164,7 @@ df.show()
### Scala API
Run Spark shell with the spark_connector in the jars argument as shown below:
```sh
$SPARK_HOME/bin/spark-shell --jars target/spark-connector_2.11-1.8.0.jar
$SPARK_HOME/bin/spark-shell --jars target/spark-tensorflow-connector_2.11-1.10.0.jar
```

The following Scala code snippet demonstrates usage on test data.
Expand Down Expand Up @@ -216,7 +216,7 @@ cd /tmp/youtube-8m-videos
curl data.yt8m.org/download.py | shard=1,3844 partition=2/video/train mirror=us python
popd

$SPARK_HOME/bin/spark-shell --jars target/spark-connector_2.11-1.8.0.jar
$SPARK_HOME/bin/spark-shell --jars target/spark-tensorflow-connector_2.11-1.10.0.jar
```

```scala
Expand Down
8 changes: 4 additions & 4 deletions spark/spark-tensorflow-connector/pom.xml
Expand Up @@ -4,9 +4,9 @@
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.tensorflow</groupId>
<artifactId>spark-connector_2.11</artifactId>
<artifactId>spark-tensorflow-connector_2.11</artifactId>
<packaging>jar</packaging>
<version>1.9.0-rc2</version>
<version>1.10.0</version>
<name>spark-tensorflow-connector</name>
<url>https://www.tensorflow.org</url>
<description>TensorFlow TFRecord connector for Apache Spark DataFrames</description>
Expand Down Expand Up @@ -120,7 +120,7 @@
<artifactSet>
<includes>
<include>com.google.protobuf:protobuf-java</include>
<include>org.tensorflow:hadoop</include>
<include>org.tensorflow:tensorflow-hadoop</include>
<include>org.tensorflow:proto</include>
</includes>
</artifactSet>
Expand Down Expand Up @@ -305,7 +305,7 @@
<dependencies>
<dependency>
<groupId>org.tensorflow</groupId>
<artifactId>hadoop</artifactId>
<artifactId>tensorflow-hadoop</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
Expand Down

0 comments on commit 76a7cd0

Please sign in to comment.