You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have read through the quick start and installation sections of the README.
Info
Info
Value
Operating System
MacOS
Geni Version
0.3.8
JDK
1.8
Spark Version
3.0.2
Problem / Steps to reproduce
It seems like it is impossible to create a boolean column from all false values using records->dataset because they get recognized as null columns. Here is a failing tests.
FAIL On records->dataset - should work for bool columns at (dataset_creation_test.clj:143)
Expected:
#<org.apache.spark.sql.types.StructType@2e83f3f5 StructType(StructField(i,LongType,true), StructField(s,StringType,true), StructField(b,BooleanType,true))>
Actual:
#<org.apache.spark.sql.types.StructType@67b8b180 StructType(StructField(i,LongType,true), StructField(s,StringType,true), StructField(b,NullType,true))>
FAIL On records->dataset - should work for bool columns at (dataset_creation_test.clj:146)
Expected:
[[0 "A" false] [1 "B" false] [2 "C" false]]
Actual:
([0 "A" nil] [1 "B" nil] [2 "C" nil])
Diffs: in [0 2] expected false, was nil
in [1 2] expected false, was nil
in [2 2] expected false, was nil
The same behavior applies to map->dataset and table->dataset. If any of the booleans are true, then the schema is understood correctly.
The text was updated successfully, but these errors were encountered:
Info
Problem / Steps to reproduce
It seems like it is impossible to create a boolean column from all
false
values usingrecords->dataset
because they get recognized as null columns. Here is a failing tests.and here is the output.
The same behavior applies to
map->dataset
andtable->dataset
. If any of the booleans aretrue
, then the schema is understood correctly.The text was updated successfully, but these errors were encountered: