Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Databricks- Support of Delta live tables #1306

Open
zacayd opened this issue Mar 5, 2024 · 0 comments
Open

Databricks- Support of Delta live tables #1306

zacayd opened this issue Mar 5, 2024 · 0 comments

Comments

@zacayd
Copy link

zacayd commented Mar 5, 2024

Background [Optional]

Hi
I have a case of## Background [Optional]

Hi
I have a case of using delta live tables that runs SQL notebooks
I cannot add cell of python
sc._jvm.za.co.absa.spline.harvester.SparkLineageInitializer.enableLineageTracking(spark._jsparkSession)
since DLT doesnt support magic like %python
So I created a seperate notebook of python - but still fails- although the spline Jar is installed on the cluster
I think that becasue DLT manages dynamiclly the clusters - i think that all the installation of jars on the cluster take no effect

Question

Does someone has experiense using spline in databricks in DLT ?
using delta live tables that runs SQL notebooks
I cannot add cell of python
sc._jvm.za.co.absa.spline.harvester.SparkLineageInitializer.enableLineageTracking(spark._jsparkSession)
since DLT doesnt support magic like %python
So I created a seperate notebook of python - but still fails- although the spline Jar is installed on the cluster
I think that becasue DLT manages dynamiclly the clusters - i think that all the installation of jars on the cluster take no effect

Question

Does someone has experiense using spline in databricks in DLT ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: New
Development

No branches or pull requests

1 participant