[BUG] Fix CSV Parsing

[revans2]

Describe the bug
I am not totally sure if this is a bug or an epic, but I have marked it as both. CSV parsing has been on by default, but it has a lot of inconsistencies with what spark does and this can cause problems. The current plan is to mitigate this by disabling CSV by default and then working through issues until we can enable it again everywhere.

Important Issues:

• [BUG] a simple query returns wrong results #6435
• [BUG] CSV escaped " in quoted strings produce incorrect value #129 escape quote parsing
• [BUG] rereading a written csv file drops rows  #6917
• [FEA] Fuzz testing of CSV #6926
• [BUG] CUDF does not strip trailing white space after a quoted string value #2069 A quoted value with training white space, has the white space ignored by spark, but not cudf when interpreted as a string.
• [BUG] fall back to CPU if columnNameofCorruptRecord is in the CSV schema #2065 Fall back to CPU if schema includes columnNameofCorruptRecord
• [FEA] port CSV tests from Spark and univocity #6927
• [BUG] CSV parsing of malformed lines is empty string not null #2068 cudf interprets missing values at the end of the line as empty strings. Spark interprets them a null values. This is fine when the null value is the default (an empty string), but is a problem if you set it to anything else.
• [BUG] CSV Parseing Inf, -Inf, and NaN not supported #125 float parsing for infinity, negative infinity, and nan
• [BUG] CSV comment after partial line does not throw out the comment #2066 bug when parsing comments.
• [FEA] Implement more efficient parsing of dates and timestamp from JSON and CSV #4969 (Hopefully we can use this to fix some issues with [BUG] Incorrect values when parsing dates from timestamps stored in CSV files #1091)
• [BUG]Rapids Accelerator (0.2) failing to read csv file on Databricks 7.0 ML GPU Runtime #1322
• [BUG] Incorrect values when parsing dates from timestamps stored in CSV files #1091 Dates don't parse extra timestamps correctly
• [FEA] Support reading JSON numeric values as timestamps and dates #4940
• [FEA] Clarify whether we support LEGACY timeParserPolicy when parsing CSV files #2931
• [BUG] spark.rapids.sql.exec.CollectLimitExec=true can mess up the CSV header row #6814 CollectLimitExec when using header row
• [FEA] Audit_3.0.1: Test for not ignoring rows in CSV that start with null char. Test in cudf #958
• [BUG] CSV parser does not support special case date/timestamp strings #136 special case date/timestamp strings.
• [FEA] Create Cloudera shim layer #951 have null character disable comment parsing.
• [BUG] CSV parsing large number values overflow #126 parsing int/float values overflows
• [BUG] CSV/JSON Parsing some float values results in overflow #124 float values on the edge parse incorrectly
• [BUG] parsing invalid boolean CSV values return true instead of null #2071 boolean parsing of invalid values is wrong
• [FEA] support spark.sql.legacy.timeParserPolicy when parsing CSV files #1111 Dates and timestamps need to deal with spark.sql.legacy.timeParserPolicy

Investigate:

• [FEA] Verify unescapedQuoteHandling with CSV reader #1524 More control in newer versions of spark to handle how badly formed CSV is parsed. In this case around unescaped quotes.

Low Priority Issues:

• Header with lots of comments at the beginning. We currently play games with turning off header checking in all but the first partition for a file. This is fine, so long as the first partition is more than just comments. This is minor, and possibly a bug in Spark as well.
• [BUG] Plugin does not support CSV Header Checking #2862
• Support alternate character sets ("encoding" option).
• Support FAILFAST mode. By default when Spark sees malformed data it converts it into a null PERMISSIVE mode. In FAILFAST mode it throws an exception. There is also DROPMALFORMED mode that is supposed to drop bad data, but it looks like CSV does not support this.
• [FEA] Support parsing dates and timestamps with timezone IDs in the string #6846
• [BUG] spark.rapids.sql.exec.CollectLimitExec=true can mess up the CSV header row #6814
• [FEA] Support ANSI interval type(year-month) from CSV source #5115
• CSV and JSON parsing should check for nulls. #4654
• [FEA] Add support for CSV reader options positiveInf, negativeInf, and nanValue #4644
• [Audit][FEA][SPARK-36831] Support ANSI interval types from CSV source #4146
• [FEA] Explore ways to not use HadoopFileLinesReader for CSV parsing #6
• [FEA] look into CSV support for multi-line and ignore leading and trailing space #130 multiline support

Tests:

• Add tests for enforceSchema set to false. TODO file issues Oddly for Spark when you set this to false it does more schema enforcement. The enforce here really means force the set schema on the files. It looks like it works for our plugin, but we want to have tests to verify this
• [FEA] Improve CSV integration tests for parsing dates and timestamps #9990