Skip to content

[SPARK-56429][DOCS] Clarify differences between nullValue and emptyValue CSV options #55405

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
yadavay-amzn wants to merge 1 commit into
base: master
Choose a base branch
from
+3 −3
Open

[SPARK-56429][DOCS] Clarify differences between nullValue and emptyValue CSV options #55405

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/sql-data-sources-csv.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,13 +141,13 @@ Data source options of CSV can be set via:
<tr>
<td><code>nullValue</code></td>
<td></td>
<td>Sets the string representation of a null value. Since 2.0.1, this <code>nullValue</code> param applies to all supported types including the string type.</td>
<td>Sets the string that, when encountered in the CSV input, is treated as a SQL NULL. For example, if set to <code>"NA"</code>, any field containing <code>NA</code> will be read as null. Since 2.0.1, this applies to all supported types including the string type.</td>
<td>read/write</td>
</tr>
<tr>
<td><code>nanValue</code></td>
<td>NaN</td>
<td>Sets the string representation of a non-number value.</td>
<td>Sets the string that, when encountered in the CSV input, is treated as NaN (Not a Number) for float and double type columns.</td>
<td>read</td>
</tr>
<tr>
Expand Down Expand Up @@ -237,7 +237,7 @@ Data source options of CSV can be set via:
<tr>
<td><code>emptyValue</code></td>
<td><code></code> (for reading), <code>""</code> (for writing)</td>
<td>Sets the string representation of an empty value.</td>
<td>Sets the value to substitute when a quoted empty string (<code>""</code>) is encountered in the CSV input. Only applies to string type columns. Unlike <code>nullValue</code> (which matches input to produce null), this specifies the value to produce in the DataFrame.</td>
<td>read/write</td>
</tr>
<tr>
Expand Down