Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fixSchemaReferenceForEmptyArray #35746

Merged
merged 5 commits into from
Jul 10, 2023
Merged

fixSchemaReferenceForEmptyArray #35746

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
6deb871
fix schema reference for empty array
Jul 5, 2023
2f4bb62
revert changes
Jul 5, 2023
026dab1
Update CosmosTableSchemaInferrer.scala
FabianMeiswinkel Jul 10, 2023
307e238
Update CosmosTableSchemaInferrer.scala
FabianMeiswinkel Jul 10, 2023
6d679bd
updated changelog
Jul 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/cosmos/azure-cosmos-spark_3-1_2-12/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
#### Breaking Changes

#### Bugs Fixed
* Fixed schema reference issue for empty array - See [PR 35746](https://github.com/Azure/azure-sdk-for-java/pull/35746)

#### Other Changes

Expand Down
1 change: 1 addition & 0 deletions sdk/cosmos/azure-cosmos-spark_3-2_2-12/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
#### Breaking Changes

#### Bugs Fixed
* Fixed schema reference issue for empty array - See [PR 35746](https://github.com/Azure/azure-sdk-for-java/pull/35746)

#### Other Changes

Expand Down
1 change: 1 addition & 0 deletions sdk/cosmos/azure-cosmos-spark_3-3_2-12/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
#### Breaking Changes

#### Bugs Fixed
* Fixed schema reference issue for empty array - See [PR 35746](https://github.com/Azure/azure-sdk-for-java/pull/35746)

#### Other Changes

Expand Down
2 changes: 1 addition & 1 deletion ...cosmos-spark_3_2-12/src/main/scala/com/azure/cosmos/spark/CosmosTableSchemaInferrer.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ private object CosmosTableSchemaInferrer
case decimalNode: DecimalNode if decimalNode.isInt => IntegerType
case arrayNode: ArrayNode => inferDataTypeFromArrayNode(arrayNode, allowNullForInferredProperties) match {
case Some(valueType) => ArrayType(valueType)
case None => NullType
case None => ArrayType(NullType)
}
case objectNode: ObjectNode =>
inferDataTypeFromObjectNode(
Expand Down
4 changes: 2 additions & 2 deletions ...os-spark_3_2-12/src/test/scala/com/azure/cosmos/spark/CosmosTableSchemaInferrerSpec.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,8 @@ class CosmosTableSchemaInferrerSpec extends UnitSpec {
val schema = CosmosTableSchemaInferrer.inferSchema(
docs, includeSystemProperties = true, includeTimestamp = true, allowNullForInferredProperties = false)
schema.fields should have size 1
schema.fields(0).dataType shouldBe NullType
schema.fields(0).nullable shouldBe true
schema.fields(0).dataType shouldBe ArrayType(NullType)
schema.fields(0).nullable shouldBe false
}

"nested array properties" should "handle different schema elements" in {
Expand Down