Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Option for handling user defined null value strings #34

Merged
merged 1 commit into from
Jul 9, 2023

Conversation

dazfuller
Copy link
Contributor

This PR resolves #31 by adding an option for a user to specify a null value string. The usage of this option is as follows

val df = spark.read
    .format("excel")
    .options("nullValue", "NA")
    .load("path")

It performs a case-insensitive comparison so that "na" matches with "NA"

@dazfuller dazfuller added this to the 0.1.12 Release milestone Jul 8, 2023
@dazfuller dazfuller self-assigned this Jul 8, 2023
@dazfuller dazfuller linked an issue Jul 8, 2023 that may be closed by this pull request
@codecov-commenter
Copy link

Codecov Report

❗ No coverage uploaded for pull request base (0_1_12-release@86b21c3). Click here to learn what that means.
Patch has no changes to coverable lines.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files
@@                Coverage Diff                @@
##             0_1_12-release      #34   +/-   ##
=================================================
  Coverage                  ?   92.72%           
=================================================
  Files                     ?       21           
  Lines                     ?      316           
  Branches                  ?        0           
=================================================
  Hits                      ?      293           
  Misses                    ?       23           
  Partials                  ?        0           

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@dazfuller dazfuller added the enhancement New feature or request label Jul 8, 2023
@dazfuller dazfuller merged commit 2380ea5 into 0_1_12-release Jul 9, 2023
11 checks passed
@dazfuller dazfuller deleted the dazfuller/nullvalues branch July 9, 2023 09:54
dazfuller added a commit that referenced this pull request Oct 25, 2023
* Update build.sbt

Set new snapshot version

* add ability to handle user define null value strings (#34)

* Update poi version to 5.2.3 (#32)

* Update poi version to 5.2.3

* Update build to account for log4j conflicts across spark versions

* Spark 3.4.1 Support (#36)

* add spark 3.4.1 to build process

* add missing copyright header

* add 3.4.1 to powershell build script

* update version for merge

* update for spark 3.5 support (#39)

---------

Co-authored-by: Jose Soto <josecsmorales@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Question config parameters supported
2 participants