Spanner @Embedded annotation doesn't work with findById #774

gaeshi · 2021-12-08T01:50:39Z

I'm trying to follow the latest official reference on how to use @Embedded annotation to reuse set of columns between multiple entities:
https://googlecloudplatform.github.io/spring-cloud-gcp/2.0.6/reference/html/index.html#embedded-objects

When I'm using SpannerRepository.save(Entity), everything works as expected, but when I'm doing SpannerRepository.findById(Entity.id), I'm getting SpannerDataException: Column not found. The logs from GapicSpannerRpc look good, it seems that the problem is somewhere during mapping.

I made a new sample project with Spring Initializr (Spring Boot 2.5.7, Gradle, Kotlin) and version 2.0.6 of spring-cloud-gcp-starter-data-spanner.

Here is the DDL I used to create a new table in Spanner:

CREATE TABLE Requests (
  requestId STRING(MAX),
  payload STRING(MAX),
  createdAt TIMESTAMP,
  createdBy STRING(MAX),
  updatedAt TIMESTAMP,
  updatedBy STRING(MAX)
) PRIMARY KEY (requestId)

Minimal example code:

@SpringBootApplication
class EmbeddedObjectsApplication

fun main(args: Array<String>) {
    runApplication<EmbeddedObjectsApplication>(*args)
}

data class EntityHistory(val createdAt: Date, val createdBy: String, val updatedAt: Date, val updatedBy: String) {
    constructor(userId: String) : this(
        createdAt = Date(System.currentTimeMillis()),
        createdBy = userId,
        updatedAt = Date(System.currentTimeMillis()),
        updatedBy = userId
    )
}

@Table(name = "Requests")
data class Request(@PrimaryKey val requestId: String, val payload: String, @Embedded val entityHistory: EntityHistory)

interface RequestRepository : SpannerRepository<Request, String>

@RestController
class ApiController(val requestRepository: RequestRepository) {

    @GetMapping("/requests/{requestId}")
    fun getRequest(@PathVariable requestId: String) = requestRepository.findById(requestId) // fails with "SpannerDataException: Column not found: entityHistory"

    @PostMapping("/requests")
    fun saveRequest(@RequestBody payload: String): String {
        val requestId = UUID.randomUUID().toString()
        requestRepository.save(Request(requestId, payload, EntityHistory("some-user-id"))) // works fine
        return requestId
    }
}

Spanner trace:

2021-12-08 10:46:59.718 DEBUG 12360 --- [ctor-http-nio-5] c.g.c.spanner.spi.v1.GapicSpannerRpc     : google.spanner.v1.Spanner/StreamingRead[6affe3f2]: Start
2021-12-08 10:46:59.718 DEBUG 12360 --- [ctor-http-nio-5] c.g.c.spanner.spi.v1.GapicSpannerRpc     : google.spanner.v1.Spanner/StreamingRead[6affe3f2]: Send:
session: "projects/***redacted***/instances/***redacted***/databases/***redacted***/sessions/AN4G3x_K2djlf6PfBqNlzQkITtL0dnjMqdJuEtKx-JAFqHipfTcyQ0kusDj1uw"
table: "Requests"
columns: "createdAt"
columns: "updatedBy"
columns: "payload"
columns: "createdBy"
columns: "requestId"
columns: "updatedAt"
key_set {
  keys {
    values {
      string_value: "bc2a9941-db7e-4b8e-b4ce-1149b8f20753"
    }
  }
}
request_options {
}

2021-12-08 10:46:59.772 DEBUG 12360 --- [sportChannel-13] c.g.c.spanner.spi.v1.GapicSpannerRpc     : google.spanner.v1.Spanner/StreamingRead[6affe3f2]: Received:
metadata {
  row_type {
    fields {
      name: "createdAt"
      type {
        code: TIMESTAMP
      }
    }
    fields {
      name: "updatedBy"
      type {
        code: STRING
      }
    }
    fields {
      name: "payload"
      type {
        code: STRING
      }
    }
    fields {
      name: "createdBy"
      type {
        code: STRING
      }
    }
    fields {
      name: "requestId"
      type {
        code: STRING
      }
    }
    fields {
      name: "updatedAt"
      type {
        code: TIMESTAMP
      }
    }
  }
}
values {
  string_value: "2021-12-08T01:29:45.013Z"
}
values {
  string_value: "some-user-id"
}
values {
  string_value: "\"test\""
}
values {
  string_value: "some-user-id"
}
values {
  string_value: "bc2a9941-db7e-4b8e-b4ce-1149b8f20753"
}
values {
  string_value: "2021-12-08T01:29:45.013Z"
}

2021-12-08 10:46:59.772 DEBUG 12360 --- [sportChannel-13] c.g.c.spanner.spi.v1.GapicSpannerRpc     : google.spanner.v1.Spanner/StreamingRead[6affe3f2]: Closed with status Status{code=OK, description=null, cause=null} and trailers Metadata(grpc-server-stats-bin=AACXBr4AAAAAAA)
2021-12-08 10:46:59.773 ERROR 12360 --- [ctor-http-nio-5] a.w.r.e.AbstractErrorWebExceptionHandler : [6441fce4-1]  500 Server Error for HTTP GET "/requests/bc2a9941-db7e-4b8e-b4ce-1149b8f20753"

com.google.cloud.spring.data.spanner.core.mapping.SpannerDataException: Column not found: entityHistory

Edit: added sample repository: https://github.com/mixefy/embedded-objects

The text was updated successfully, but these errors were encountered:

suztomo · 2021-12-08T16:33:15Z

Checking this...

I was able to reproduce the problem with the sample repository (https://github.com/mixefy/embedded-objects; thanks!):

12:05:09.663 [reactor-http-nio-3] ERROR o.s.b.a.w.r.e.AbstractErrorWebExceptionHandler - [eb38fea5-1, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:51485]  500 Server Error for HTTP GET "/requests/3e50e8c3-67a6-4621-a8a2-13dd09a8c4d4"
com.google.cloud.spring.data.spanner.core.mapping.SpannerDataException: Column not found: entityHistory
        at com.google.cloud.spring.data.spanner.core.convert.StructPropertyValueProvider.getPropertyValue(StructPropertyValueProvider.java:84)
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
        *__checkpoint ⇢ HTTP GET "/requests/3e50e8c3-67a6-4621-a8a2-13dd09a8c4d4" [ExceptionHandlingWebHandler]
Original Stack Trace:
                at com.google.cloud.spring.data.spanner.core.convert.StructPropertyValueProvider.getPropertyValue(StructPropertyValueProvider.java:84)
                at com.google.cloud.spring.data.spanner.core.convert.StructPropertyValueProvider.getPropertyValue(StructPropertyValueProvider.java:39)
                at org.springframework.data.mapping.model.PersistentEntityParameterValueProvider.getParameterValue(PersistentEntityParameterValueProvider.java:74)
                at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator.extractInvocationArguments(ClassGeneratingEntityInstantiator.java:276)
                at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator$EntityInstantiatorAdapter.createInstance(ClassGeneratingEntityInstantiator.java:248)
                at org.springframework.data.mapping.model.ClassGeneratingEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:89)
                at com.google.cloud.spring.data.spanner.core.convert.ConverterAwareMappingSpannerEntityReader.read(ConverterAwareMappingSpannerEntityReader.java:93)
                at com.google.cloud.spring.data.spanner.core.convert.ConverterAwareMappingSpannerEntityProcessor.mapToList(ConverterAwareMappingSpannerEntityProcessor.java:80)
                at com.google.cloud.spring.data.spanner.core.SpannerTemplate.mapToListAndResolveChildren(SpannerTemplate.java:576)
                at com.google.cloud.spring.data.spanner.core.SpannerTemplate.read(SpannerTemplate.java:207)
                at com.google.cloud.spring.data.spanner.core.SpannerTemplate.read(SpannerTemplate.java:187)
                at com.google.cloud.spring.data.spanner.core.SpannerTemplate.read(SpannerTemplate.java:166)
                at com.google.cloud.spring.data.spanner.repository.support.SimpleSpannerRepository.findById(SimpleSpannerRepository.java:101)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                at java.base/java.lang.reflect.Method.invoke(Method.java:566)
                at org.springframework.data.repository.core.support.RepositoryMethodInvoker$RepositoryFragmentMethodInvoker.lambda$new$0(RepositoryMethodInvoker.java:289)
                at org.springframework.data.repository.core.support.RepositoryMethodInvoker.doInvoke(RepositoryMethodInvoker.java:137)
                at org.springframework.data.repository.core.support.RepositoryMethodInvoker.invoke(RepositoryMethodInvoker.java:121)
                at org.springframework.data.repository.core.support.RepositoryComposition$RepositoryFragments.invoke(RepositoryComposition.java:529)
                at org.springframework.data.repository.core.support.RepositoryComposition.invoke(RepositoryComposition.java:285)
                at org.springframework.data.repository.core.support.RepositoryFactorySupport$ImplementationMethodExecutionInterceptor.invoke(RepositoryFactorySupport.java:599)
                at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
                at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.doInvoke(QueryExecutorMethodInterceptor.java:163)
                at org.springframework.data.repository.core.support.QueryExecutorMethodInterceptor.invoke(QueryExecutorMethodInterceptor.java:138)
                at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
                at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
                at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
                at org.springframework.data.repository.core.support.MethodInvocationValidator.invoke(MethodInvocationValidator.java:98)
                at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
                at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215)
                at com.sun.proxy.$Proxy72.findById(Unknown Source)
                at jp.mixefy.embeddedobjects.ApiController.getRequest(EmbeddedObjectsApplication.kt:61)

Memo

The doc (https://googlecloudplatform.github.io/spring-cloud-gcp/2.0.6/reference/html/index.html#embedded-objects) says the embedded objects can be "saved" and "stored", but it doesn't mention "read" or "retrieved.
Embedded Id support spring-attic/spring-cloud-gcp#588 (comment) states "Reading from Struct/row from Spanner"
SpannerRepositoryIntegrationTests uses this.tradeRepository.findById. The return type Trade uses @Embedded:
```
  @Embedded
  @PrimaryKey
  private TradeDetail tradeDetail;
```

Trying to do in Java to see any difference.

suztomo · 2021-12-08T19:37:33Z

Update: I tried in Java and it worked fine. https://github.com/suztomo/embedded-objects-java

    @GetMapping("/requests/{requestId}")
    String getRequest(@PathVariable String requestId)  {
        // This findById was failing with "SpannerDataException: Column not found: entityHistory"
        // in the Kotlin. In Java this just works fine.
        Optional<Request> r = requestRepository.findById(requestId);
        System.out.println("request isPresent? = " + r.isPresent());
        return r.get().payload;
    }

https://github.com/suztomo/embedded-objects-java/blob/773802cd056d43736fef67a84c24b0cfdc0afee4/src/main/java/com/example/embedded/ApiController.java#L22

Current status

https://github.com/mixefy/embedded-objects (Kotlin) doesn't work
https://github.com/suztomo/embedded-objects-java (Java) works.

We haven't figured out why.

Next Steps

Potential steps I'm thinking:

Compare debug-level log between Kotlin and Java
With debugger, step execution to see the different behavior of findById.

elefeint · 2021-12-08T23:44:34Z

@gaeshi The issue is with the default constructor in Kotlin -- data classes are immutable, so all fields have to be initialized at construction time. Spring Data's KotlinClassGeneratingEntityInstantiator looks at the constructor available, and tries to get each column's value from Spanner-specific code, failing when it gets to a synthetic "column" entityHistory, which does not exist in the underlying table.

In the Java flow, the object is constructed with Java's default constructor, which takes no arguments, allowing Spring Data's object instantiation code to work. The logic for filling in synthetic columns can then be applied.

The following entity structure works as expected in Kotlin -- note that this is no longer a data class, and that entityHistory is a settable field:

@Table(name = "Requests")
class Request (@PrimaryKey val requestId: String,
               var payload: String) {

    @Embedded var entityHistory: EntityHistory? = null
}

gaeshi · 2021-12-09T00:25:33Z

@suztomo @elefeint
Thank you very much for your help!

suztomo added priority: p4 priority: p3 labels Dec 8, 2021

elefeint closed this as completed Dec 9, 2021

suztomo mentioned this issue Dec 9, 2021

Document restriction on embedded objects #784

Merged

elefeint mentioned this issue Dec 10, 2021

Nested aggregations that used to work with 1.3.6 no longer work with 1.4.0 spring-projects/spring-data-r2dbc#695

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Spanner @Embedded annotation doesn't work with findById #774

Spanner @Embedded annotation doesn't work with findById #774

gaeshi commented Dec 8, 2021 •

edited

suztomo commented Dec 8, 2021 •

edited

suztomo commented Dec 8, 2021 •

edited

elefeint commented Dec 8, 2021

gaeshi commented Dec 9, 2021

Spanner @Embedded annotation doesn't work with findById #774

Spanner @Embedded annotation doesn't work with findById #774

Comments

gaeshi commented Dec 8, 2021 • edited

suztomo commented Dec 8, 2021 • edited

suztomo commented Dec 8, 2021 • edited

Current status

Next Steps

elefeint commented Dec 8, 2021

gaeshi commented Dec 9, 2021

gaeshi commented Dec 8, 2021 •

edited

suztomo commented Dec 8, 2021 •

edited

suztomo commented Dec 8, 2021 •

edited