remove embedded nul bytes if present in raw data to be read

[jozefhajnala]

When trying to collect data that contain embedded nul bytes into R the process fails with an error (reproducible example added as a unit test case). This proposes to work around this issue by omitting those bytes.

[javierluraschi]

Looks like arrow does not support this test case:

spark_read_csv() can read if embedded nuls present: error: embedded nul in string: 'test\0string' 

We can skip the test, added a suggestion, thank you for this fix @jozefhajnala!

[jozefhajnala]

Added the skip as suggested.

[falaki]

Databricks Connect tests failed. View logs here.

[falaki]

Databricks Connect tests succeeded. View logs here.

[yitao-li]

Looks good to me!
Let's try rebaseing past #2426 and see if arrow tests are skipped as expected.

[falaki]

Databricks Connect tests succeeded. View logs here.

[yitao-li]

Great success!

[rexdouglass]

I'm encountering an error on collect
Error in RecordBatch__to_dataframe(x, use_threads = option_use_threads()) :
embedded nul in string: '�\0'

I have the most updated 1.3.1 whose patch notes include "Embedded nul bytes are removed from strings when reading strings from Spark to R (#2250)"
https://spark.rstudio.com/news/

I do not have a reproducible example (the null appears somewhere in a 13 billion row table, and creating one artificially is difficult). So i'm just asking if this hotfix is supposed to cover the use case of running a collect() on a spark table and returning it to R, as is the case where I'm getting this error.

[yitao-li]

@rexdouglass Hey I don't think this PR covers the use case involving arrow. You can try disabling arrow as a workaround but then there will be some performance penalty as deserialization will be slower without arrow.

I'll try to address the use case involving arrow in #2633 -- I'm hoping it's non-complicated and can be shipped as part of Sparklyr 1.4.