fix: Hibernate would insert JSON as STRING #944

olavloite · 2024-03-04T18:03:35Z

Hibernate tried by default to insert columns that had been annotated with @JdbcTypeCode(SqlTypes.JSON) as a STRING instead of JSON. This change adds a default type registration for JSON that does include the correct type code with the value.

Hibernate tried by default to insert columns that had been annotated with `@JdbcTypeCode(SqlTypes.JSON)` as a STRING instead of JSON. This change adds a default type registration for JSON that does include the correct type code with the value.

anton-rynkovyi-exa · 2024-03-07T11:18:53Z

@olavloite how can I store null value to the json column?
I get INVALID_ARGUMENT: Value has type STRING which cannot be inserted into column, which has type JSON

olavloite · 2024-03-08T13:59:50Z

@olavloite how can I store null value to the json column? I get INVALID_ARGUMENT: Value has type STRING which cannot be inserted into column, which has type JSON

@anton-rynkovyi-exa Sorry, apparently we also need to override and explicitly type NULL values as JSON. See #952

tanushree-bs-exa · 2024-04-15T04:03:21Z

@olavloite I am currently using Hibernate 6.4.4 version, along with spanner-hibernate-dialect 3.3.0 version and google-cloud-spanner-jdbc 2.16.1 version. But still facing this error org.springframework.dao.InvalidDataAccessApiUsageException: Could not deserialize string to java type: JsonJavaType
Any suggestions? I want to store the java object in json format.
More details on Stack Overflow here

olavloite · 2024-04-15T06:39:55Z

@tanushree-bs-exa The StackOverflow question includes the error message, but not the entire stacktrace. Would you mind sharing that here?

olavloite · 2024-04-15T06:52:03Z

@tanushree-bs-exa The error message is most probably an indication that there is a mismatch between the actual data (the JSON string) and the Java class(es) that you are trying to deserialize it to. Would you mind sharing the relevant classes here as well? Meaning at least the following classes (but potentially more, depending on the further nesting of structured classes in these classes):

ConnectorTemplate
Descriptor
ConfigTemplate

tanushree-bs-exa · 2024-04-15T07:00:12Z

@olavloite I added annotation @JdbcTypeCode(SqlTypes.JSON) to my column

@Column(name="TemplateObj")
@JdbcTypeCode( SqlTypes.JSON )
private Template template;

Below is the model classes of Template, Descriptor and DetailTemplate and its nested structured classes

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Template {
    private String id;
    private String vendor;
    private String acc;
    private String schemaVersion;
    private String icon;
    private String type;
    private Descriptor descriptor;
    private DetailTemplate detailTemplate;
    private String accountId;
    private Boolean hasAccount;
    private Boolean canTestConnection;
    private String dataSourceType;
    private Boolean runNowSupported;
}

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Descriptor {
    private String displayName;
    private String description;
    private String docRef;
}

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class DetailTemplate {
    private String type;
    private Descriptor descriptor;
    @NotEmpty(message = "{detailTemplate.properties.required}")
    private List<Property> properties;
}

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Property {
    private String id;
    private boolean required;
    private boolean disabled;
    private String validation;
    private String default_value;
    private String type;
    private Descriptor descriptor;
    private List<Option> options;
    private DependsOn depends_on;
    private boolean secret;
    private boolean disableOnEdit;
    private Integer minDays;
    private String helperText;
}

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class Option {
    private String id;
    private String displayName;
}

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class DependsOn {
    private String field;
    private String value;
}

Also, this is the sample data which i have to store in the DB column - TemplateObj

{
  "canTestConnection": true,
  "detailTemplate": {
    "properties": [
      {
        "descriptor": {
          "description": "Select multiple regions",
          "displayName": "Region",
          "docRef": "https://guide.com"
        },
        "id": "region",
        "options": [
          {
            "displayName": "us-east",
            "id": "us-east"
          },
          {
            "displayName": "canada",
            "id": "canada"
          }
        ],
        "required": true,
        "type": "multi-select"
      }
    ]
  },
  "descriptor": {
    "description": "Sample",
    "displayName": "Test Name",
    "docRef": "https://guide.com"
  },
  "hasAccount": true,
  "icon": "http://svg_link",
  "id": "abc",
  "schemaVersion": "1.0",
  "type": "xyz",
  "vendor": "Test"
}

olavloite · 2024-04-15T07:14:28Z

I noticed that in the original example that you posted above, you had this definition for ConfigTemplate:

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
public class ConfigTemplate {
    private String type;
    @JdbcTypeCode( SqlTypes.JSON )
    private Descriptor descriptor;
    @NotEmpty(message = "{configTemplate.properties.required}")
    private List<Property> properties;
}

The @JdbcTypeCode annotation should only be on the field that is defined as a property of the Hibernate entity. Would you mind removing that from the code and retry?

tanushree-bs-exa · 2024-04-15T07:17:41Z

@olavloite Yes, have removed the @JdbcTypeCode annotation in ConfigTemplate class, but still doesnt work and get the same error.
Apologies, i had edited the example again.

olavloite · 2024-04-15T07:56:46Z

@tanushree-bs-exa

I just copy-pasted the example code into a test project and tried the following:

package com.google.cloud.spanner.sample;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.google.cloud.spanner.sample.entities.ConnectorTemplate;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.junit.runners.JUnit4;

@RunWith(JUnit4.class)
public class JsonDeserializerTest {

  @Test
  public void testDeserialize() throws JsonProcessingException {
    String json = "{\n"
        + "  \"canTestConnection\": true,\n"
        + "  \"configTemplate\": {\n"
        + "    \"properties\": [\n"
        + "      {\n"
        + "        \"descriptor\": {\n"
        + "          \"description\": \"Select multiple regions\",\n"
        + "          \"displayName\": \"Region\",\n"
        + "          \"docRef\": \"https://guide.com\"\n"
        + "        },\n"
        + "        \"id\": \"region\",\n"
        + "        \"options\": [\n"
        + "          {\n"
        + "            \"displayName\": \"ap-south-1\",\n"
        + "            \"id\": \"ap-south-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"eu-south-1\",\n"
        + "            \"id\": \"eu-south-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-gov-east-1\",\n"
        + "            \"id\": \"us-gov-east-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"ca-central-1\",\n"
        + "            \"id\": \"ca-central-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"eu-central-1\",\n"
        + "            \"id\": \"eu-central-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-iso-west-1\",\n"
        + "            \"id\": \"us-iso-west-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-west-1\",\n"
        + "            \"id\": \"us-west-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-west-2\",\n"
        + "            \"id\": \"us-west-2\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"af-south-1\",\n"
        + "            \"id\": \"af-south-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"eu-west-3\",\n"
        + "            \"id\": \"eu-west-3\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"eu-north-1\",\n"
        + "            \"id\": \"eu-north-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"eu-west-2\",\n"
        + "            \"id\": \"eu-west-2\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"eu-west-1\",\n"
        + "            \"id\": \"eu-west-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"ap-northeast-3\",\n"
        + "            \"id\": \"ap-northeast-3\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"ap-northeast-2\",\n"
        + "            \"id\": \"ap-northeast-2\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"ap-northeast-1\",\n"
        + "            \"id\": \"ap-northeast-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"me-south-1\",\n"
        + "            \"id\": \"me-south-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"sa-east-1\",\n"
        + "            \"id\": \"sa-east-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"ap-east-1\",\n"
        + "            \"id\": \"ap-east-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"cn-north-1\",\n"
        + "            \"id\": \"cn-north-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-gov-west-1\",\n"
        + "            \"id\": \"us-gov-west-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"ap-southeast-1\",\n"
        + "            \"id\": \"ap-southeast-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"ap-southeast-2\",\n"
        + "            \"id\": \"ap-southeast-2\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-iso-east-1\",\n"
        + "            \"id\": \"us-iso-east-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-east-1\",\n"
        + "            \"id\": \"us-east-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-east-2\",\n"
        + "            \"id\": \"us-east-2\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"cn-northwest-1\",\n"
        + "            \"id\": \"cn-northwest-1\"\n"
        + "          },\n"
        + "          {\n"
        + "            \"displayName\": \"us-isob-east-1\",\n"
        + "            \"id\": \"us-isob-east-1\"\n"
        + "          }\n"
        + "        ],\n"
        + "        \"required\": true,\n"
        + "        \"type\": \"multi-select\"\n"
        + "      }\n"
        + "    ]\n"
        + "  },\n"
        + "  \"descriptor\": {\n"
        + "    \"description\": \"Collector for CloudTrail (via API)\",\n"
        + "    \"displayName\": \"AWS CloudTrail (via API)\",\n"
        + "    \"docRef\": \"https://guide.com\"\n"
        + "  },\n"
        + "  \"hasAccount\": true,\n"
        + "  \"icon\": \"http://svg_link\",\n"
        + "  \"id\": \"aws-cloudtrail-events\",\n"
        + "  \"schemaVersion\": \"1.0\",\n"
        + "  \"type\": \"api\",\n"
        + "  \"vendor\": \"Amazon\"\n"
        + "}";

    ObjectMapper mapper = new ObjectMapper();
    ConnectorTemplate connectorTemplate = mapper.readValue(json, ConnectorTemplate.class);
  }

}

That failed at first, due to lack of public getters and setters in the JSON classes.

Do you have public getters and setters for all the fields in your application?
If yes; Do you get an error if you try to run the above test with your code? If so, what error?

tanushree-bs-exa · 2024-04-15T08:08:59Z

@olavloite

Yes, as all above classes are annotated with @Data annotation from Lombok, which automatically generates getters and setters for all fields present in class.
I tried to run the above test with my code, it dint give any errors and ran successfully.

tanushree-bs-exa · 2024-04-16T03:32:20Z

@olavloite any suggestions/workaround how this can be fixed?

olavloite · 2024-04-16T05:29:47Z

@olavloite any suggestions/workaround how this can be fixed?

I will try to take another look at this later today.

olavloite · 2024-04-16T08:31:08Z

@tanushree-bs-exa I've incorporated your schema into one of the sample applications, but there is works as expected. See #1000

Would you mind taking a look at that sample and see if that is consistent with how your application is set up? One thing to consider is that Hibernate requires you to have a JSON library on the classpath for serializing and de-serializing. This sample uses Jackson for that:

google-cloud-spanner-hibernate/google-cloud-spanner-hibernate-samples/spring-data-jpa-full-sample/pom.xml

Line 51 in fdfed5c

    
           <!-- JSON library that is needed to be able to use JSON data type in Hibernate -->

.

If you would not have that, then I would have expected a different error message.

If that does not reveal any obvious differences, could you then please share a small runnable project that reproduces the problem, preferably using the Spanner emulator?

olavloite · 2024-04-18T08:54:10Z

@tanushree-bs-exa Friendly ping. Were you able to solve the problem based on the sample above? Or are you still facing this problem? And if so, would you be able to share a full reproduction case?

fix: Hibernate would insert JSON as STRING

01682c2

Hibernate tried by default to insert columns that had been annotated with `@JdbcTypeCode(SqlTypes.JSON)` as a STRING instead of JSON. This change adds a default type registration for JSON that does include the correct type code with the value.

olavloite requested a review from ankiaga March 5, 2024 11:20

ankiaga approved these changes Mar 6, 2024

View reviewed changes

olavloite merged commit 1d8b855 into master Mar 6, 2024
6 checks passed

olavloite deleted the json-jdbc-type branch March 6, 2024 10:30

release-please bot mentioned this pull request Mar 5, 2024

chore(master): release 3.1.1 #938

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: Hibernate would insert JSON as STRING #944

fix: Hibernate would insert JSON as STRING #944

olavloite commented Mar 4, 2024

anton-rynkovyi-exa commented Mar 7, 2024

olavloite commented Mar 8, 2024

tanushree-bs-exa commented Apr 15, 2024 •

edited

olavloite commented Apr 15, 2024

olavloite commented Apr 15, 2024

tanushree-bs-exa commented Apr 15, 2024 •

edited

olavloite commented Apr 15, 2024

tanushree-bs-exa commented Apr 15, 2024

olavloite commented Apr 15, 2024

tanushree-bs-exa commented Apr 15, 2024 •

edited

tanushree-bs-exa commented Apr 16, 2024

olavloite commented Apr 16, 2024

olavloite commented Apr 16, 2024

olavloite commented Apr 18, 2024

fix: Hibernate would insert JSON as STRING #944

fix: Hibernate would insert JSON as STRING #944

Conversation

olavloite commented Mar 4, 2024

anton-rynkovyi-exa commented Mar 7, 2024

olavloite commented Mar 8, 2024

tanushree-bs-exa commented Apr 15, 2024 • edited

olavloite commented Apr 15, 2024

olavloite commented Apr 15, 2024

tanushree-bs-exa commented Apr 15, 2024 • edited

olavloite commented Apr 15, 2024

tanushree-bs-exa commented Apr 15, 2024

olavloite commented Apr 15, 2024

tanushree-bs-exa commented Apr 15, 2024 • edited

tanushree-bs-exa commented Apr 16, 2024

olavloite commented Apr 16, 2024

olavloite commented Apr 16, 2024

olavloite commented Apr 18, 2024

tanushree-bs-exa commented Apr 15, 2024 •

edited

tanushree-bs-exa commented Apr 15, 2024 •

edited

tanushree-bs-exa commented Apr 15, 2024 •

edited