Fix: MySQL/MariaDB TIMESTAMP precision for message ordering (#3442)

[seungwone]

Fixes #3442

The AddBatchPreparedStatement class uses AtomicLong.getAndIncrement() to create sequential millisecond timestamps, but MySQL/MariaDB's default TIMESTAMP precision (0) truncates this information, causing message ordering issues.

Added fractional seconds precision (TIMESTAMP(3)) to MySQL and MariaDB schemas to support millisecond-based message ordering in JdbcChatMemoryRepository.

Changes

• schema-mysql.sql: Changed TIMESTAMP to TIMESTAMP(3)
• schema-mariadb.sql: Changed TIMESTAMP to TIMESTAMP(3)

[sunyuhan1998]

I think this PR itself is fine, but as @markpollack previously mentioned:

#3154 (comment)

this field is actually being used as an auto-incrementing sequence to ensure message ordering, rather than representing time in a temporal sense. Perhaps we should consider renaming this field in the future.

[YunKuiLu]

There is a common belief that TIMESTAMP can only be used until 2038. Perhaps it should be changed to BIGINT or DATETIME.

https://stackoverflow.com/questions/409286/should-i-use-the-datetime-or-timestamp-data-type-in-mysql

[markpollack]

yea, I'm not sure what to do here in a few ways.

1. I think the 1.1 line should drop the timestamp completely.
2. if we change the schema 'just like that' in a 1.0.1 release, stuff breaks - though one can argue of course it is already broken.

As for point 2, a migration script can be provided to run to upgrade to 1.0.1 and advertised in the release notes etc.

I'm thinking for the 1.0.x line, we provide the migration script.

thoughts @YunKuiLu @sunyuhan1998 @seungwone @ThomasVitale ?

[YunKuiLu]

yea, I'm not sure what to do here in a few ways.

1. I think the 1.1 line should drop the timestamp completely.
2. if we change the schema 'just like that' in a 1.0.1 release, stuff breaks - though one can argue of course it is already broken.

As for point 2, a migration script can be provided to run to upgrade to 1.0.1 and advertised in the release notes etc.

I'm thinking for the 1.0.x line, we provide the migration script.

thoughts @YunKuiLu @sunyuhan1998 @seungwone @ThomasVitale ?

Maybe we can do something like this:

1. In the 1.0.1-SNAPSHOT version, provide a migration script and support both schemas at the same time. When constructing the JdbcChatMemoryRepository, we can check the current schema. If the table in the DB still uses TIMESTAMP as the column type, we log a warn, and include a link to the upgrade guide.

2. In the 1.0.x release version, we do the same check. If TIMESTAMP is detected, we log an error instead, and again include a link to the migration guide. Also, users would need to explicitly set the flag -Dspring.ai.chat.memory.repository.jdbc.compatible-version=1.0.0 to bypass the error — otherwise, the app won't start.

3. In 1.1.x, if the schema still has the timestamp column with type TIMESTAMP, we simply block startup and show an error message with a link to the upgrade guide.

These are just my initial thoughts though — not sure if we really need this many steps.

[markpollack]

I think I found an easier way, since the values of the timestamps are meaningless, they just need to increase, we make sure that when we are assigning the timestamp, it is always increases at an appropriate level of granulatirty - eg. one second or something like that.

[sobychacko]

@seungwone Thanks for working on this! After reviewing the discussion and based on Mark's suggestion about fixing the granularity in code, here is a more fine-grained approach that doesn't require any schema breaking changes in 1.1.x or 1.0.x:

Plan:

• Implement a code-only fix using second-level granularity (as Mark suggested)
• Apply this non-breaking change to both 1.1.x and 1.0.x
• No schema changes required
• We'll implement the fix in 1.1.x first, then backport to 1.0.x

This approach avoids migration complexity for existing users while solving the ordering issue across all databases.

Here is the new PR: #4739

Also closing this PR in preference to the new one.

I created an issue for tracking all the breaking changes we discussed above (dropping the timestamp column altogether and using a sequence_id column) for the 2.0.x timeline.