Skip to content

Commit

Permalink
chore: bump to verion 0.9.5 (#1243)
Browse files Browse the repository at this point in the history
Co-authored-by: Serena <serena.rxy@gmail.com>
  • Loading branch information
serena-ruan and serena-ruan committed Jan 12, 2022
1 parent a1ddb56 commit 79d92d3
Show file tree
Hide file tree
Showing 112 changed files with 18,821 additions and 1,234 deletions.
26 changes: 13 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@

[![Build Status](https://msdata.visualstudio.com/A365/_apis/build/status/microsoft.SynapseML?branchName=master)](https://msdata.visualstudio.com/A365/_build/latest?definitionId=17563&branchName=master) [![codecov](https://codecov.io/gh/Microsoft/SynapseML/branch/master/graph/badge.svg)](https://codecov.io/gh/Microsoft/SynapseML) [![Gitter](https://badges.gitter.im/Microsoft/MMLSpark.svg)](https://gitter.im/Microsoft/MMLSpark?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)

[![Release Notes](https://img.shields.io/badge/release-notes-blue)](https://github.com/Microsoft/SynapseML/releases) [![Scala Docs](https://img.shields.io/static/v1?label=api%20docs&message=scala&color=blue&logo=scala)](https://mmlspark.blob.core.windows.net/docs/0.9.4/scala/index.html#package) [![PySpark Docs](https://img.shields.io/static/v1?label=api%20docs&message=python&color=blue&logo=python)](https://mmlspark.blob.core.windows.net/docs/0.9.4/pyspark/index.html) [![Academic Paper](https://img.shields.io/badge/academic-paper-7fdcf7)](https://arxiv.org/abs/1810.08744)
[![Release Notes](https://img.shields.io/badge/release-notes-blue)](https://github.com/Microsoft/SynapseML/releases) [![Scala Docs](https://img.shields.io/static/v1?label=api%20docs&message=scala&color=blue&logo=scala)](https://mmlspark.blob.core.windows.net/docs/0.9.5/scala/index.html#package) [![PySpark Docs](https://img.shields.io/static/v1?label=api%20docs&message=python&color=blue&logo=python)](https://mmlspark.blob.core.windows.net/docs/0.9.5/pyspark/index.html) [![Academic Paper](https://img.shields.io/badge/academic-paper-7fdcf7)](https://arxiv.org/abs/1810.08744)

[![Version](https://img.shields.io/badge/version-0.9.4-blue)](https://github.com/Microsoft/SynapseML/releases) [![Snapshot Version](https://mmlspark.blob.core.windows.net/icons/badges/master_version3.svg)](#sbt)
[![Version](https://img.shields.io/badge/version-0.9.5-blue)](https://github.com/Microsoft/SynapseML/releases) [![Snapshot Version](https://mmlspark.blob.core.windows.net/icons/badges/master_version3.svg)](#sbt)


SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines.
Expand All @@ -28,8 +28,8 @@ sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.
See the API documentation [for
Scala](https://mmlspark.blob.core.windows.net/docs/0.9.4/scala/index.html#package) and [for
PySpark](https://mmlspark.blob.core.windows.net/docs/0.9.4/pyspark/index.html).
Scala](https://mmlspark.blob.core.windows.net/docs/0.9.5/scala/index.html#package) and [for
PySpark](https://mmlspark.blob.core.windows.net/docs/0.9.5/pyspark/index.html).

<details>
<summary><strong><em>Table of Contents</em></strong></summary>
Expand Down Expand Up @@ -82,7 +82,7 @@ the above example, or from python:
```python
import pyspark
spark = pyspark.sql.SparkSession.builder.appName("MyApp") \
.config("spark.jars.packages", "com.microsoft.azure:synapseml_2.12:0.9.4") \
.config("spark.jars.packages", "com.microsoft.azure:synapseml_2.12:0.9.5") \
.config("spark.jars.repositories", "https://mmlspark.azureedge.net/maven") \
.getOrCreate()
import synapse.ml
Expand All @@ -95,7 +95,7 @@ your `build.sbt`:

```scala
resolvers += "SynapseML" at "https://mmlspark.azureedge.net/maven"
libraryDependencies += "com.microsoft.azure" % "synapseml_2.12" % "0.9.4"
libraryDependencies += "com.microsoft.azure" % "synapseml_2.12" % "0.9.5"

```

Expand All @@ -105,9 +105,9 @@ SynapseML can be conveniently installed on existing Spark clusters via the
`--packages` option, examples:

```bash
spark-shell --packages com.microsoft.azure:synapseml_2.12:0.9.4 --conf spark.jars.repositories=https://mmlspark.azureedge.net/maven
pyspark --packages com.microsoft.azure:synapseml_2.12:0.9.4 --conf spark.jars.repositories=https://mmlspark.azureedge.net/maven
spark-submit --packages com.microsoft.azure:synapseml_2.12:0.9.4 MyApp.jar --conf spark.jars.repositories=https://mmlspark.azureedge.net/maven
spark-shell --packages com.microsoft.azure:synapseml_2.12:0.9.5 --conf spark.jars.repositories=https://mmlspark.azureedge.net/maven
pyspark --packages com.microsoft.azure:synapseml_2.12:0.9.5 --conf spark.jars.repositories=https://mmlspark.azureedge.net/maven
spark-submit --packages com.microsoft.azure:synapseml_2.12:0.9.5 MyApp.jar --conf spark.jars.repositories=https://mmlspark.azureedge.net/maven
```

This can be used in other Spark contexts too. For example, you can use SynapseML
Expand All @@ -122,15 +122,15 @@ cloud](http://community.cloud.databricks.com), create a new [library from Maven
coordinates](https://docs.databricks.com/user-guide/libraries.html#libraries-from-maven-pypi-or-spark-packages)
in your workspace.

For the coordinates use: `com.microsoft.azure:synapseml_2.12:0.9.4`
For the coordinates use: `com.microsoft.azure:synapseml_2.12:0.9.5`
with the resolver: `https://mmlspark.azureedge.net/maven`. Ensure this library is
attached to your target cluster(s).

Finally, ensure that your Spark cluster has at least Spark 3.12 and Scala 2.12.

You can use SynapseML in both your Scala and PySpark notebooks. To get started with our example notebooks import the following databricks archive:

`https://mmlspark.blob.core.windows.net/dbcs/SynapseMLExamplesv0.9.4.dbc`
`https://mmlspark.blob.core.windows.net/dbcs/SynapseMLExamplesv0.9.5.dbc`

### Apache Livy and HDInsight

Expand All @@ -143,7 +143,7 @@ Excluding certain packages from the library may be necessary due to current issu
{
"name": "synapseml",
"conf": {
"spark.jars.packages": "com.microsoft.azure:synapseml_2.12:0.9.4",
"spark.jars.packages": "com.microsoft.azure:synapseml_2.12:0.9.5",
"spark.jars.repositories": "https://mmlspark.azureedge.net/maven",
"spark.jars.excludes": "org.scala-lang:scala-reflect,org.apache.spark:spark-tags_2.12,org.scalactic:scalactic_2.12,org.scalatest:scalatest_2.12"
}
Expand All @@ -157,7 +157,7 @@ In Azure Synapse, "spark.yarn.user.classpath.first" should be set to "true" to o
{
"name": "synapseml",
"conf": {
"spark.jars.packages": "com.microsoft.azure:synapseml_2.12:0.9.4",
"spark.jars.packages": "com.microsoft.azure:synapseml_2.12:0.9.5",
"spark.jars.repositories": "https://mmlspark.azureedge.net/maven",
"spark.jars.excludes": "org.scala-lang:scala-reflect,org.apache.spark:spark-tags_2.12,org.scalactic:scalactic_2.12,org.scalatest:scalatest_2.12",
"spark.yarn.user.classpath.first": "true"
Expand Down
19 changes: 1 addition & 18 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,6 @@ def pomPostFunc(node: XmlNode): scala.xml.Node = {

pomPostProcess := pomPostFunc

val speechResolver = "Speech" at "https://mmlspark.blob.core.windows.net/maven/"

val getDatasetsTask = TaskKey[Unit]("getDatasets", "download datasets used for testing")
val datasetName = "datasets-2021-12-10.tgz"
val datasetUrl = new URL(s"https://mmlspark.blob.core.windows.net/installers/$datasetName")
Expand Down Expand Up @@ -213,20 +211,6 @@ publishDocs := {
uploadToBlob(unifiedDocDir.toString, version.value, "docs")
}

val release = TaskKey[Unit]("release", "publish the library to synapseml blob")
release := Def.taskDyn {
val v = isSnapshot.value
if (!v) {
Def.task {
sonatypeBundleRelease.value
}
} else {
Def.task {
"Not a release"
}
}
}

val publishBadges = TaskKey[Unit]("publishBadges", "publish badges to synapseml blob")
publishBadges := {
def enc(s: String): String = {
Expand Down Expand Up @@ -310,11 +294,10 @@ lazy val cognitive = (project in file("cognitive"))
.dependsOn(core % "test->test;compile->compile")
.settings(settings ++ Seq(
libraryDependencies ++= Seq(
"com.microsoft.cognitiveservices.speech" % "client-sdk" % "1.14.0",
"com.microsoft.cognitiveservices.speech" % "client-jar-sdk" % "1.14.0",
"com.azure" % "azure-storage-blob" % "12.14.2",
"com.azure" % "azure-ai-textanalytics" % "5.1.4",
),
resolvers += speechResolver,
name := "synapseml-cognitive"
): _*)

Expand Down
Loading

0 comments on commit 79d92d3

Please sign in to comment.