[SPARK-39722] [SQL] getString API for Dataset

[VindhyaG]

What changes were proposed in this pull request?

Add getString API for Dataset for usecases where string representation needs to be used other than stdout(println)
More details in the bug

Why are the changes needed?

But there are a lot of cases where we might need to get a String representation of the show output. For example

logging framework to which we need to push the representation of a df
send the string over a REST call from the driver
send the string to stderr instead of stdout

Does this PR introduce any user-facing change?

Yes

How was this patch tested?

Tested locally using spark-shell

[VindhyaG]

@rxin @ScrapCodes @maropu can you please review . thanks!

[jaceklaskowski]

Just one tiny nit

[jaceklaskowski]

nit: Why is this 3.4.0 while the latter changes are marked as 3.5.0?

[VindhyaG]

I missed changing version number . will change once i can figure out what is the current release cycle is on? Can you please point me where i can get that info .thanks!

[VindhyaG]

@jaceklaskowski updated to 3.5.0 based on recently(hours ago) closed bug fix version of SQL and Java API component

[jaceklaskowski]

nit: Why is this 3.5.0?

[VindhyaG]

Thanks for reviewing @jaceklaskowski . I took the version from one of the recent bugs but i realised i took the affected version instead of fix version which is wrong. I am trying to get details on how to get current release version to add but not able to find any links i can be confident on .

[ScrapCodes]

Hi @VindhyaG, this might be useful - may be we can benefit from the usecase you have for this. Is it just for logging?
Not sure what others think, it might be good to limit the API surface.

[VindhyaG]

Hi @VindhyaG, this might be useful - may be we can benefit from the usecase you have for this. Is it just for logging? Not sure what others think, it might be good to limit the API surface.

@ScrapCodes thanks for reviewing. Bug lists three ways API could be used which i have added in PR description above. Do you have any suggestions to limit API surface?

[ScrapCodes]

I see this as developer facing API, So just having

def getString(numRows: Int, truncate: Int): String =
     getString(numRows, truncate, vertical = false)

would suffice.

[ScrapCodes]

Do you think, a more interesting way can be returning a JSON representation?

[VindhyaG]

I see this as developer facing API, So just having

def getString(numRows: Int, truncate: Int): String =
     getString(numRows, truncate, vertical = false)

would suffice.

my intention was to keep it consistent with show() where if numrows and truncate have default values

[VindhyaG]

Do you think, a more interesting way can be returning a JSON representation?
@ScrapCodes If use case is to send the it via rest api(as listed in use case above) JSON would make more sense but for logging i suppose string in tabular form is more useful? may be have a separate API for that ?

[VindhyaG]

@ScrapCodes can you pls suggest how can we go ahead on this?
@jaceklaskowski have updated the versions. can you pls review again.

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!