Skip to content

Commit

Permalink
Docs (#106)
Browse files Browse the repository at this point in the history
* bump version

* bump version

* Update KustoSource.md

* docs

Co-authored-by: ohbitton <42732233+ohbitton@users.noreply.github.com>
  • Loading branch information
ohadbitt and ohbitton committed Feb 23, 2020
1 parent d32812e commit 1f3a054
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 11 deletions.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,14 +32,14 @@ link your application with the artifact below in order to use the Azure Data Exp
```
groupId = com.microsoft.azure.kusto
artifactId = spark-kusto-connector
version = 1.0.2
version = 1.1.0
```

**In Maven**:

Look for the following coordinates:
```
com.microsoft.azure.kusto:spark-kusto-connector:1.0.2
com.microsoft.azure.kusto:spark-kusto-connector:1.1.0
```

Or clone this repository and build it locally to add it to your local maven repository,
Expand All @@ -49,7 +49,7 @@ the jar can also be found under the [released package](https://github.com/Azure/
<dependency>
<groupId>com.microsoft.azure.kusto</groupId>
<artifactId>spark-kusto-connector</artifactId>
<version>1.0.2</version>
<version>1.1.0</version>
</dependency>
```

Expand All @@ -58,7 +58,7 @@ the jar can also be found under the [released package](https://github.com/Azure/
Create Library -> Maven with the following coordinates:

```
com.microsoft.azure.kusto:spark-kusto-connector:1.0.2
com.microsoft.azure.kusto:spark-kusto-connector:1.1.0
```

#### Building Samples Module
Expand Down Expand Up @@ -112,8 +112,8 @@ on [Microsoft Azure SDK For Key Vault](https://mvnrepository.com/artifact/com.mi

> **Note:** When working with Databricks, Azure Data Explorer connector requires Azure Data Explorer java client libraries (and azure key-vault library if used) to be installed.
This can be done by accessing Databricks Create Library -> Maven and specifying the following coordinates:
- com.microsoft.azure.kusto:spark-kusto-connector:1.0.2
- com.microsoft.azure.kusto:spark-kusto-connector:1.0.2
- com.microsoft.azure.kusto:spark-kusto-connector:1.1.0
- com.microsoft.azure.kusto:spark-kusto-connector:1.1.0

## Documentation

Expand Down
7 changes: 3 additions & 4 deletions docs/KustoSource.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ that is using it. Please verify the following before using Kusto connector:

Kusto connector implements Spark 'Datasource V1' API.
Kusto data source identifier is "com.microsoft.kusto.spark.datasource".
Kusto table schema is translated into a spark as explained in [DataTypes](Spark-Kusto DataTypes mapping.md).
Kusto table schema is translated into a spark schema as explained in [Spark-Kusto DataTypes](mapping.md).
The reading is done in one of two modes: 'Single' and 'Distributed'. By default if the user did not override the read mode - an estimated
count for the query is made; for small row count - a 'Single' mode is chosen with a fallback to the 'Distributed' mode (as the first might
fail due to [kusto limit policy](https://docs.microsoft.com/en-us/azure/kusto/concepts/querylimits)). In 'Distributed' mode, a distributed
Expand All @@ -35,15 +35,14 @@ that is using it. Please verify the following before using Kusto connector:
**Simplified Command Syntax**:
```
<dataframe-name> =
spark.read.kusto(<cluster-name.region-name>, <database-name>, <kusto-query>, <parameters map>)
spark.read.kusto(<cluster-name.region-name>, <database-name>, <kusto-query>, <parameters map>, <ClientRequestProperties>)
```
where:
* **Kusto-query** is any valid Kusto query. For details, please refer to [Query statements documentation](https://docs.microsoft.com/en-us/azure/kusto/query/statements). To read the whole table, just provide the table name as query.
>Note:
Kusto commands cannot be executed via the connector.
* **Parameters map** is a set of key-value pairs, where the key is the parameter name. See [Supported Options](#supported-options)
section for details


**Elaborate Command Syntax**:
```
Expand Down Expand Up @@ -89,7 +88,7 @@ All the options that can be use in the Kusto source are under the object KustoSo

* **KUSTO_CLIENT_REQUEST_PROPERTIES_JSON**:
A json representation for [ClientRequestProperties](https://github.com/Azure/azure-kusto-java/blob/master/data/src/main/java/com/microsoft/azure/kusto/data/ClientRequestProperties.java)
used in the call for reading from kusto (used in the single query for 'single' mode or for the export command for 'distributed' mode). Use toString to create the json.
used in the call for reading from Kusto (used in the single query for 'single' mode or for the export command for 'distributed' mode). Use toString to create the json.


#### Transient Storage Parameters
Expand Down
2 changes: 1 addition & 1 deletion pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
<version>${revision}</version>

<properties>
<revision>1.0.2</revision>
<revision>1.1.0</revision>
<scala.version>2.11</scala.version>
<specs2.version>3.6.5</specs2.version>
<spark.version>2.4.0</spark.version>
Expand Down

0 comments on commit 1f3a054

Please sign in to comment.