Skip to content

Commit

Permalink
prepare to release v1.6.0
Browse files Browse the repository at this point in the history
  • Loading branch information
@uncleGen
uncleGen committed Feb 19, 2019
1 parent a60dda2 commit 9cc0d47
Show file tree
Hide file tree
Showing 22 changed files with 38 additions and 38 deletions.
12 changes: 6 additions & 6 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -34,37 +34,37 @@
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-maxcompute_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-logservice_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-tablestore</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-ons_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-mns_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-core</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
```
Expand Down
4 changes: 2 additions & 2 deletions docs/HadoopMR-on-TableStore.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ OK, let the program run.

```
$ rm -rf out
$ HADOOP_CLASSPATH=emr-tablestore-1.6.0-SNAPSHOT.jar:tablestore-4.1.0-jar-with-dependencies.jar::joda-time-2.9.4.jar:YourRowCounter.jar bin/hadoop YourRowCounterClass
$ HADOOP_CLASSPATH=emr-tablestore-1.6.0.jar:tablestore-4.1.0-jar-with-dependencies.jar::joda-time-2.9.4.jar:YourRowCounter.jar bin/hadoop YourRowCounterClass
...
$ find out -type f
out/_SUCCESS
Expand All @@ -122,7 +122,7 @@ $ cat out/part-r-00000
TOTAL 9
```

FYI, in emr-examples_2.11-1.6.0-SNAPSHOT.jar, we provides an executable row-counting program, `com.aliyun.openservices.tablestore.hadoop.RowCounter`.
FYI, in emr-examples_2.11-1.6.0.jar, we provides an executable row-counting program, `com.aliyun.openservices.tablestore.hadoop.RowCounter`.

## TableStore as Data Sink

Expand Down
4 changes: 2 additions & 2 deletions docs/Hive-SparkSQL-on-TableStore.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@
### Hive CLI

```
$ HADOOP_HOME=YourHadoopDir HADOOP_CLASSPATH=emr-tablestore-1.6.0-SNAPSHOT.jar:tablestore-4.1.0-jar-with-dependencies.jar:joda-time-2.9.4.jar bin/hive
$ HADOOP_HOME=YourHadoopDir HADOOP_CLASSPATH=emr-tablestore-1.6.0.jar:tablestore-4.1.0-jar-with-dependencies.jar:joda-time-2.9.4.jar bin/hive
```

### SparkSQL CLI

```
$ bin/spark-sql --master local --jars emr-tablestore-1.6.0-SNAPSHOT.jar,tablestore-4.1.0-jar-with-dependencies.jar
$ bin/spark-sql --master local --jars emr-tablestore-1.6.0.jar,tablestore-4.1.0-jar-with-dependencies.jar
```


Expand Down
4 changes: 2 additions & 2 deletions docs/Spark-on-TableStore.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,10 @@ If you prefer to scala, please replace `JavaSparkContext` to `SparkContext` and
Let it run.

```
$ bin/spark-submit --master local --jars emr-tablestore-1.6.0-SNAPSHOT.jar,tablestore-4.1.0-jar-with-dependencies.jar YourRowCounter.jar
$ bin/spark-submit --master local --jars emr-tablestore-1.6.0.jar,tablestore-4.1.0-jar-with-dependencies.jar YourRowCounter.jar
TOTAL: 9
```

FYI,
* for more details about `TableStoreInputFormat`, please refer to (HadoopMR-on-TableStore.md).
* in emr-examples_2.11-1.6.0-SNAPSHOT.jar, we provides an executable row-counting program, `com.aliyun.openservices.tablestore.spark.RowCounter`.
* in emr-examples_2.11-1.6.0.jar, we provides an executable row-counting program, `com.aliyun.openservices.tablestore.spark.RowCounter`.
8 changes: 4 additions & 4 deletions docs/how_to_run_spark_with_python_sdk.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,26 +26,26 @@ Bellow document will show you how to use Loghub (ODPS, ONS, MNS and so on) in py
## LogService

1. download the [**loghub.py**](https://github.com/aliyun/aliyun-emapreduce-sdk/blob/master-2.x/external/emr-logservice/src/main/python/pyspark/streaming/loghub.py);
2. download the pre-build [**emr-logservice_2.11-1.6.0-SNAPSHOT.jar**](http://central.maven.org/maven2/com/aliyun/emr/emr-logservice_2.11/1.6.0-SNAPSHOT/emr-logservice_2.11-1.6.0-SNAPSHOT.jar);
2. download the pre-build [**emr-logservice_2.11-1.6.0.jar**](http://central.maven.org/maven2/com/aliyun/emr/emr-logservice_2.11/1.6.0/emr-logservice_2.11-1.6.0.jar);
3. download loghub's dependencies, i.e. [**fastjson-1.2.45.jar**](http://central.maven.org/maven2/com/alibaba/fastjson/1.2.45/fastjson-1.2.45.jar), [**commons-validator-1.4.0.jar**](http://central.maven.org/maven2/commons-validator/commons-validator/1.4.0/commons-validator-1.4.0.jar), [**loghub-client-lib-0.6.13.jar**](http://central.maven.org/maven2/com/aliyun/openservices/loghub-client-lib/0.6.13/loghub-client-lib-0.6.13.jar), [**aliyun-log-0.6.11.jar**](http://central.maven.org/maven2/com/aliyun/openservices/aliyun-log/0.6.11/aliyun-log-0.6.11.jar), [**json-lib-2.4-jdk15.jar**](http://central.maven.org/maven2/net/sf/json-lib/json-lib/2.4/json-lib-2.4-jdk15.jar). Above-mentioned library version pass test;
4. download [**loghub-wordcount.py**](https://github.com/aliyun/aliyun-emapreduce-sdk/blob/master-2.x/examples/src/main/python/streaming/loghub-wordcount.py);
5. run the example, use
```
spark-submit --master local[4] --jars fastjson-1.2.45.jar,commons-validator-1.4.0.jar,
emr-logservice_2.11-1.6.0-SNAPSHOT.jar,loghub-client-lib-0.6.13.jar,aliyun-log-0.6.11.jar,json-lib-2.4-jdk15.jar
emr-logservice_2.11-1.6.0.jar,loghub-client-lib-0.6.13.jar,aliyun-log-0.6.11.jar,json-lib-2.4-jdk15.jar
--py-files loghub.py loghub-wordcount.py <logServiceProject> <logsStoreName> <logHubConsumerGroupName>
<loghubEndpoint> <numReceiver> <accessKeyId> <accessKeySecret>
```

## ODPS

1. download the [**odps.py**](https://github.com/aliyun/aliyun-emapreduce-sdk/blob/master-2.x/external/emr-maxcompute/src/main/python/pyspark/odps.py);
2. download the pre-build [**emr-maxcompute_2.11-1.6.0-SNAPSHOT.jar**](http://central.maven.org/maven2/com/aliyun/emr/emr-maxcompute_2.11/1.6.0-SNAPSHOT/emr-maxcompute_2.10-1.6.0-SNAPSHOT.jar);
2. download the pre-build [**emr-maxcompute_2.11-1.6.0.jar**](http://central.maven.org/maven2/com/aliyun/emr/emr-maxcompute_2.11/1.6.0/emr-maxcompute_2.10-1.6.0.jar);
3. download odps's dependencies, i.e. [**fastjson-1.2.23.jar**](http://mvnrepository.com/artifact/com.alibaba/fastjson/1.2.23), [**aspectjrt-1.8.2.jar**](http://mvnrepository.com/artifact/org.aspectj/aspectjrt/1.8.2), [**odps-sdk-commons-0.27.2-public.jar**](http://mvnrepository.com/artifact/com.aliyun.odps/odps-sdk-commons/0.27.2-public), [**odps-sdk-core-0.27.2-public.jar**](http://mvnrepository.com/artifact/com.aliyun.odps/odps-sdk-core/0.27.2-public). Above-mentioned library version pass test;
4. download [**odps-sample.py**](https://github.com/aliyun/aliyun-emapreduce-sdk/blob/master/examples/src/main/python/odps-sample.py);
5. run the example, use
```
spark-submit --master local[4] --jars fastjson-1.2.23.jar,aspectjrt-1.8.2.jar,emr-maxcompute_2.11-1.6.0-SNAPSHOT.jar,
spark-submit --master local[4] --jars fastjson-1.2.23.jar,aspectjrt-1.8.2.jar,emr-maxcompute_2.11-1.6.0.jar,
odps-sdk-commons-0.27.2-public.jar,odps-sdk-core-0.27.2-public.jar --py-files odps.py
odps-sample.py <accessKeyId> <accessKeySecret> <project> <table> <partition> <numPartitions>
```
12 changes: 6 additions & 6 deletions docs/manual.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -31,37 +31,37 @@
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-maxcompute_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-logservice_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-tablestore</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-ons_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-mns_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr-core</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</dependency>
```
Expand Down
2 changes: 1 addition & 1 deletion emr-baseline-testing/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>

Expand Down
2 changes: 1 addition & 1 deletion emr-datahub/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-dts/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-kafka/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-logservice/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-logservice/src/main/python/pyspark/streaming/loghub.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -193,4 +193,4 @@ def _printErrorMsg():
________________________________________________________________________________________________
""" % ('1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT'))
""" % ('1.6.0', '1.6.0', '1.6.0'))
2 changes: 1 addition & 1 deletion emr-maxcompute/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-maxcompute/src/main/python/pyspark/odps.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -141,4 +141,4 @@ def _printErrorMsg():
________________________________________________________________________________________________
""" % ('1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT'))
""" % ('1.6.0', '1.6.0', '1.6.0'))
2 changes: 1 addition & 1 deletion emr-mns/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-mns/src/main/python/pyspark/streaming/mns.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -96,4 +96,4 @@ def _printErrorMsg():
________________________________________________________________________________________________
""" % ('1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT'))
""" % ('1.6.0', '1.6.0', '1.6.0'))
2 changes: 1 addition & 1 deletion emr-ons/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-ons/src/main/python/pyspark/streaming/ons.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,4 +73,4 @@ def _printErrorMsg():
________________________________________________________________________________________________
""" % ('1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT', '1.6.0-SNAPSHOT'))
""" % ('1.6.0', '1.6.0', '1.6.0'))
2 changes: 1 addition & 1 deletion emr-oss/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion emr-tablestore/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<artifactId>emr_2.11</artifactId>
<groupId>com.aliyun.emr</groupId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
Expand Down
2 changes: 1 addition & 1 deletion examples/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
<parent>
<groupId>com.aliyun.emr</groupId>
<artifactId>emr_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>
<relativePath>../pom.xml</relativePath>
</parent>

Expand Down
2 changes: 1 addition & 1 deletion pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

<groupId>com.aliyun.emr</groupId>
<artifactId>emr_2.11</artifactId>
<version>1.6.0-SNAPSHOT</version>
<version>1.6.0</version>

<packaging>pom</packaging>
<name>Aliyun E-MapReduce Root</name>
Expand Down

0 comments on commit 9cc0d47

Please sign in to comment.