org.springframework.dao.DataIntegrityViolationException - HELP NEEDED!

[BenNeighbour]

Hi, @dartartem @cer , I have just encountered this very odd issue. After a method is called by a saga (in-between steps), I am getting this exception:

org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [UPDATE eventuate.saga_instance SET state_name = ?, last_request_id = ?, saga_data_type = ?, saga_data_json = ?, end_state = ?, compensating = ? where saga_type = ? AND saga_id = ?]; ERROR: value too long for type character varying(1000); nested exception is org.postgresql.util.PSQLException: ERROR: value too long for type character varying(1000)
	at org.springframework.jdbc.support.SQLStateSQLExceptionTranslator.doTranslate(SQLStateSQLExceptionTranslator.java:104) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1443) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:862) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:917) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:927) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at io.eventuate.common.common.spring.jdbc.EventuateSpringJdbcStatementExecutor.update(EventuateSpringJdbcStatementExecutor.java:23) ~[eventuate-common-common-spring-jdbc-0.9.0.RELEASE.jar!/:na]
	at io.eventuate.tram.sagas.orchestration.SagaInstanceRepositoryJdbc.update(SagaInstanceRepositoryJdbc.java:146) ~[eventuate-tram-sagas-orchestration-0.13.0.RELEASE.jar!/:na]
	at io.eventuate.tram.sagas.orchestration.SagaManagerImpl.processActions(SagaManagerImpl.java:222) ~[eventuate-tram-sagas-orchestration-0.13.0.RELEASE.jar!/:na]
	at io.eventuate.tram.sagas.orchestration.SagaManagerImpl.create(SagaManagerImpl.java:105) ~[eventuate-tram-sagas-orchestration-0.13.0.RELEASE.jar!/:na]
	at io.eventuate.tram.sagas.orchestration.SagaManagerImpl.create(SagaManagerImpl.java:69) ~[eventuate-tram-sagas-orchestration-0.13.0.RELEASE.jar!/:na]
	at io.eventuate.tram.sagas.orchestration.SagaInstanceFactory.create(SagaInstanceFactory.java:21) ~[eventuate-tram-sagas-orchestration-0.13.0.RELEASE.jar!/:na]
	at com.allocationengine.AllocationControllerGrpc.allocate(AllocationControllerGrpc.java:52) ~[classes!/:na]
	at com.allocationServiceGrpc$MethodHandlers.invoke(AllocationServiceGrpc.java:209) ~[api-common-library-0.0.1-SNAPSHOT.jar!/:na]
	at io.grpc.stub.ServerCalls$UnaryServerCallHandler$UnaryServerCallListener.onHalfClose(ServerCalls.java:171) ~[grpc-stub-1.16.1.jar!/:1.16.1]
	at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.halfClosed(ServerCallImpl.java:283) ~[grpc-core-1.16.1.jar!/:1.16.1]
	at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1HalfClosed.runInContext(ServerImpl.java:710) ~[grpc-core-1.16.1.jar!/:1.16.1]
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.16.1.jar!/:1.16.1]
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[grpc-core-1.16.1.jar!/:1.16.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_282]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_282]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_282]
aused by: org.postgresql.util.PSQLException: ERROR: value too long for type character varying(1000)
	at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2476) ~[postgresql-42.1.1.jar!/:42.1.1]
	at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2189) ~[postgresql-42.1.1.jar!/:42.1.1]
	at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:300) ~[postgresql-42.1.1.jar!/:42.1.1]
	at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428) ~[postgresql-42.1.1.jar!/:42.1.1]
	at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354) ~[postgresql-42.1.1.jar!/:42.1.1]
	at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169) ~[postgresql-42.1.1.jar!/:42.1.1]
	at org.postgresql.jdbc.PgPreparedStatement.executeUpdate(PgPreparedStatement.java:136) ~[postgresql-42.1.1.jar!/:42.1.1]
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeUpdate(ProxyPreparedStatement.java:61) ~[HikariCP-3.4.5.jar!/:na]
	at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeUpdate(HikariProxyPreparedStatement.java) ~[HikariCP-3.4.5.jar!/:na]
	at org.springframework.jdbc.core.JdbcTemplate.lambda$update$0(JdbcTemplate.java:867) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:617) ~[spring-jdbc-5.2.9.RELEASE.jar!/:5.2.9.RELEASE]
	... 19 common frames omitted

Here is my saga definition too:

@Override
public SagaDefinition<AllocateSagaData> getSagaDefinition() {
    return step()
            .invokeLocal(allocationEngine::allocateTickets)
            .withCompensation(sagaData -> {
            })
            .step()
            .invokeParticipant(sagaData -> allocationEngine.updateStatus(sagaData.getId(), Common.Status.ALLOCATED))
            .withCompensation(sagaData -> allocationEngine.updateStatus(sagaData.getId(), Common.Status.OPEN))
            .step()
            .invokeParticipant(sagaData -> allocationEngine.allocateTickets(sagaData.getTickets()))
            .build();
}

It completes the first step, .invokeLocal(allocationEngine::allocateTickets)

but then it completely falls over in between in some eventuate classes?

Any help or pointers would be much appreciated, I'm trying to get this shipped as soon as possible!!

[cer]

I'm away from a computer but most likely the value inserted into saga_data_json is too long and you need to increase the size of the column.

[BenNeighbour]

@dartartem @eventuateio But I didn't think I'd have to do that? I thought that the Eventuate CDC and the gradle dependancies deal with all of that - let's say the database gets dropped, (in a Dev or production environment) I will have to reset it each time? I thought that those tables were only touched by the eventuate code. I may have got to the bottom of the issue, but that wasn't what I did, I just did something to a dogey bit of code in one of my service methods.

I appreciate that you are away from a computer right now, so if/when you get access to one again I would appreciate any help. I will comment later if my initial fix has worked. Thanks @cer for the help so far, I really like your framework, it makes it so easy for me to develop complex, reliable and scalable apps at a fast pace.

I would be super interested in contributing to the framework itself in the future once I've finished the project I'm currently working on, because I am enjoying developing with this and would love to improve it!

Anyways, thanks for now

[cer]

See https://github.com/eventuate-tram/eventuate-tram-sagas/blob/0a8ea3814948701023d8a6a3617714d9eedf7066/postgres/tram-saga-schema.sql#L22

The database schema is your responsibility. You will need to setup the tables for Eventuate along with the rest of the tables. You can copy/paste and perhaps adapt SQL scripts in the repositories.

Glad you like Eventuate. Contributions are always welcome!

[BenNeighbour]

@cer Right, I get that. Sorry I was just using the eventuate postgres docker image with the tables pre-created because I didn't know what it needed to have in it. My bad. Also, on what basis does the recieved_messages table get purged/wiped? I can see that hogging up a lot of storage? I'm just curious but I'm sure it does clean it up and everything.

Thanks for your help

[cer]

BTW There are other column types: e.g. TEXT. See https://github.com/eventuate-foundation/eventuate-common/blob/a691ea45909ff74f9c19d527402c0859931752ce/postgres/2.initialize-database.sql#L8

There is also the possibility of using JSON type too - https://github.com/eventuate-foundation/eventuate-common/blob/a691ea45909ff74f9c19d527402c0859931752ce/postgres-json/4.initialize-database.sql#L5

However, that work is in a development branch for Eventuate Tram and we need to review whether anything needs to be done for this framework.

[cer]

Also, on what basis does the recieved_messages table get purged/wiped? I can see that hogging up a lot of storage? I'm just curious but I'm sure it does clean it up and everything.

That currently does not happen automatically. See eventuate-tram/eventuate-tram-docs#3