Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove Support for Databricks 10.4 [databricks] #10474

Merged
merged 8 commits into from
Feb 26, 2024

Conversation

razajafri
Copy link
Collaborator

This PR drops support for Databricsk 10.4

Changes:

Tests:

  • No tests were added
  • Built for all the supported major versions of Spark. No IT were run.

fixes #10429

razajafri and others added 4 commits February 22, 2024 10:01
Copy link
Member

@jlowe jlowe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are other references to 321db that have been missed. git grep 321db will show where they are, e.g.: profiles in aggregator/pom.xml, delta-spark321db project should be removed.

@@ -1,6 +1,6 @@
#!/usr/local/env groovy
/*
* Copyright (c) 2023, NVIDIA CORPORATION.
* Copyright (c) 2023-2024, NVIDIA CORPORATION.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check copyrights in every file. Since you haven't used the pre-commit, the easiest is probably to run

./scripts/auto-copyrighter.sh $(git diff --name-only origin/branch-24.04..HEAD)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! My pre-commit is broken somehow and I assumed it ran.

@razajafri
Copy link
Collaborator Author

This should've been a draft PR
@jlowe @gerashegalov thanks for reviewing, I apologize for the extra work you guys had to do.

PTAL again

@sameerz sameerz added the feature request New feature or request label Feb 23, 2024
@jlowe
Copy link
Member

jlowe commented Feb 23, 2024

build

Copy link
Collaborator

@gerashegalov gerashegalov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one optional comment

@@ -24,7 +24,7 @@ import org.apache.spark.sql.rapids.shims.ArrowUtilsShim
import org.apache.spark.sql.types._
import org.apache.spark.sql.vectorized.ColumnarBatch

//TODO is this needed? we already have a similar version in spark321db
//TODO is this needed? we already have a similar version in spark330db
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you'll have to run CI another time, please drop this obsolete comment beforehand.

Copy link
Collaborator

@NvTimLiu NvTimLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, +1

@razajafri razajafri merged commit 0b003dd into NVIDIA:branch-24.04 Feb 26, 2024
39 of 40 checks passed
@razajafri razajafri deleted the SP-10429-drop-10.4 branch February 26, 2024 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[FEA] Drop support for Databricks 10.4 ML LTS
5 participants