Skip to content

Commit

Permalink
sparksql source doc
Browse files Browse the repository at this point in the history
  • Loading branch information
Yuqing-cat committed May 23, 2023
1 parent 845eff4 commit 800479d
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions docs/how-to-guides/sparksql-source-notes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
layout: default
title: Storing secrets in Azure Key Vault
title: Using `SparkSQLSource` as Data Source
parent: How-to Guides
---

Expand All @@ -19,7 +19,7 @@ sql_source = SparkSqlSource(name="sparkSqlQuerySource", sql="SELECT * FROM green

```

1. SparkSQL table
2. SparkSQL table
If your source is already defined as a table in Databricks, you can directly use its name as data source in Feathr job.

```python
Expand All @@ -36,4 +36,6 @@ agg_anchor = FeatureAnchor(name="aggregationFeatures",
features=agg_features)
```

When using SparkSQL table as data source, you need to make sure the table can be accessed by Spark session as the Feathr job.
When using SparkSQL table as data source, you need to make sure the table can be accessed by Spark session as the Feathr job.

Similarly, tables in Blob storages can also be used as this `SparkSQLSrouce` when using synapse as spark provider.

0 comments on commit 800479d

Please sign in to comment.