[SPARK-56429][DOCS] Clarify differences between nullValue and emptyValue CSV options

[yadavay-amzn]

What changes were proposed in this pull request?

Update the CSV data source documentation to clarify how nullValue, emptyValue, and nanValue differ from each other:

• nullValue: when this exact string is encountered in the CSV input, Spark treats the field as SQL NULL.
• emptyValue: when a quoted empty string ("") is encountered, Spark substitutes this value instead. Only applies to string type columns.
• nanValue: when this string is encountered, Spark treats it as NaN for float/double columns.

Why are the changes needed?

The previous descriptions used the same pattern ("Sets the string representation of ...") for all three options, which is misleading because nullValue matches input to produce null, while emptyValue specifies the output value to substitute. See SPARK-56429.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Documentation-only change.

Was this patch authored or co-authored using generative AI tooling?

Yes.

[waterlx]

@yadavay-amzn Thanks for the patch! May you review my comment to see if it makes any sense to you?

[yadavay-amzn]

@yadavay-amzn Thanks for the patch! May you review my comment to see if it makes any sense to you?

@waterlx replied to JIRA comment asking for feedback from the previous developers who worked on this.

Looked to me like PR #22234 is unrelated to SPARK-56429, so we still need this PR to fix the issue. May be I'm missing something?

[waterlx]

@yadavay-amzn Thanks for the patch! May you review my comment to see if it makes any sense to you?

@waterlx replied to JIRA comment asking for feedback from the previous developers who worked on this.

Looked to me like PR #22234 is unrelated to SPARK-56429, so we still need this PR to fix the issue. May be I'm missing something?

@yadavay-amzn Sorry if I made you confused and I did not quite get you.
Yes, PR #22234 does not relate to this PR you are working on, and I do think your PR is great to remove the vagueness of current doc, especially for new comers (like me).

I am not sure if my second comment made you confused, sorry for that if it did. Please allow me to explain:
There are 2 cases if specifing.options("nullValue", "NA")
(1) NA in CSV will be read as null.
(2) NA quoted by the "quote" specification, for example "NA", in CSV, will be also read as null. "quote" is also an option which be specified when reading CSV. See https://spark.apache.org/docs/latest/sql-data-sources-csv.html

My point is may you consider adding (2) into your PR?. Making any sense to you?

[waterlx]

@mmolimar It is regarding #22234 about nullValue when reading CSV. I happened to find that the nullValue specification quoted by quote specification, will be also read as null. Say .options("nullValue", "NA") makes "NA" (quote defaults to a double quote) being read as null, in additional to NA. May you comment if it works as your original design?

@MaxGekk @HyukjinKwon May you review this PR by @yadavay-amzn, since you also worked on or reviewed #22234. Thank you!