Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1321] Make update command return the number of updated rows. #1331

Closed
wants to merge 2 commits into from
Closed

[1321] Make update command return the number of updated rows. #1331

wants to merge 2 commits into from

Conversation

Silverlight42
Copy link
Contributor

Description

Resolves #1321.

How was this patch tested?

Modified existing tests.

Does this PR introduce any user-facing changes?

Yes. The update command returns the following output:

image

Signed-off-by: Carlos Peña <cdpm42@gmail.com>
Copy link
Collaborator

@scottsand-db scottsand-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requested 1 minor change. Otherwise LGTM. Thanks!

@@ -22,18 +22,18 @@ import org.apache.spark.sql.delta.actions.{AddCDCFile, AddFile, FileAction}
import org.apache.spark.sql.delta.commands.cdc.CDCReader.{CDC_TYPE_COLUMN_NAME, CDC_TYPE_NOT_CDC, CDC_TYPE_UPDATE_POSTIMAGE, CDC_TYPE_UPDATE_PREIMAGE}
import org.apache.spark.sql.delta.files.{TahoeBatchFileIndex, TahoeFileIndex}
import org.apache.hadoop.fs.Path

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: we have blank lines separating our import groups, so the above group is other and the below group is org.apache.spark. So add back the blank line please.

@@ -51,6 +51,10 @@ case class UpdateCommand(
condition: Option[Expression])
extends LeafRunnableCommand with DeltaCommand {

override val output: Seq[Attribute] = {
Seq(AttributeReference("num_updated_rows", LongType)())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should've clarified this in the issue, my bad.

Can you make this num_affected_rows please? To be consistent with the DELETE command output.

https://github.com/delta-io/delta/blob/master/core/src/main/scala/org/apache/spark/sql/delta/commands/DeleteCommand.scala#L86

Signed-off-by: Carlos Peña <cdpm42@gmail.com>
@Silverlight42
Copy link
Contributor Author

Thanks @scottsand-db!

allisonport-db pushed a commit to allisonport-db/delta that referenced this pull request Aug 25, 2022
 ## Description

This PR makes the `UPDATE` command return the number of updated rows.

The update command returns the following output:

![image](https://user-images.githubusercontent.com/6467558/184264601-89506bf7-d816-4992-996a-2ade4c9e38a8.png)

Modified existing tests.

Closes delta-io#1331

Signed-off-by: Scott Sandre <scott.sandre@databricks.com>
GitOrigin-RevId: a00cdd1a8a4e10dae5afece4ec5414d9bc9fde89
GitOrigin-RevId: 160dce9de1a1e34a29d47c6e21c9117f151b9a30
GitOrigin-RevId: 71f954cfcaf4d3a14e0a0693525a577bd0898d3f
(cherry picked from commit b71ad65)
@allisonport-db allisonport-db added this to the 2.1.0 milestone Aug 28, 2022
@Silverlight42 Silverlight42 deleted the fix-1321 branch September 4, 2022 00:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature Request] Make UPDATE return the number of updated rows
3 participants