大批量插入数据(千万级)时,自动生成的id会有冲突

[guchengod]

No description provided.

[qmdx]

调用代码请提出，描述内容太简单了？

[FelixLiuSheng]

我遇到的问题和你的一样，希望尽快解决一下该问题吧！导致该问题的主要原因是机器位占用的比重太大，还有就是可以增加随机位或者增加本地自增位去保证，再增加id位的长度解决问题 @qmdx @guchengod
大致思想可以参考一下mycat的自增ID,例如在同一机器同一毫秒的自增位自动从0开始，该毫秒中的id都在该基础上去自增

[FelixLiuSheng]

@qmdx 我晚些时候提交一个这个bug的修复代码吧！大致的实现思路如我上诉的，如何？

[qmdx]

@FelixLiuSheng PR 过来看看

[guchengod]

@qmdx 顶级IService的 boolean insertBatch(List entityList, int batchSize);方法 每次插入5000数据 一千万数据重复大概几十条

[FelixLiuSheng]

我今天下午进行了一次调试，在单机多线程的情况下进行无数次测试都是OK的，但是在多机不指定机器编号的时候计算出来的是会出现问题的，再说对于docker这种镜像方式的部署，出现机器重复的可能很大，个人建议做一下更改即可或者做一个zookeeper之类的去维持机器的唯一性
位置在:
public Sequence() { this.datacenterId = getDatacenterId(maxDatacenterId); this.workerId = getMaxWorkerId(datacenterId, maxWorkerId); }

[guchengod]

@FelixLiuSheng 单机下确实有冲突的
java.sql.BatchUpdateException: ORA-00001: unique constraint (表A) violated

at oracle.jdbc.driver.DatabaseError.throwBatchUpdateException(DatabaseError.java:345)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:10844)
at com.alibaba.druid.filter.FilterChainImpl.statement_executeBatch(FilterChainImpl.java:2596)
at com.alibaba.druid.filter.FilterAdapter.statement_executeBatch(FilterAdapter.java:2474)
at com.alibaba.druid.filter.FilterEventAdapter.statement_executeBatch(FilterEventAdapter.java:279)
at com.alibaba.druid.filter.FilterChainImpl.statement_executeBatch(FilterChainImpl.java:2594)
at com.alibaba.druid.filter.FilterAdapter.statement_executeBatch(FilterAdapter.java:2474)
at com.alibaba.druid.filter.FilterEventAdapter.statement_executeBatch(FilterEventAdapter.java:279)
at com.alibaba.druid.filter.FilterChainImpl.statement_executeBatch(FilterChainImpl.java:2594)
at com.alibaba.druid.proxy.jdbc.StatementProxyImpl.executeBatch(StatementProxyImpl.java:192)
at com.alibaba.druid.pool.DruidPooledPreparedStatement.executeBatch(DruidPooledPreparedStatement.java:559)
at sun.reflect.GeneratedMethodAccessor819.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.logging.jdbc.PreparedStatementLogger.invoke(PreparedStatementLogger.java:78)
at com.sun.proxy.$Proxy478.executeBatch(Unknown Source)
at org.apache.ibatis.executor.BatchExecutor.doFlushStatements(BatchExecutor.java:122)
at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:129)
at org.apache.ibatis.executor.BaseExecutor.flushStatements(BaseExecutor.java:122)
at org.apache.ibatis.executor.CachingExecutor.flushStatements(CachingExecutor.java:114)
at sun.reflect.GeneratedMethodAccessor517.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63)
at com.sun.proxy.$Proxy475.flushStatements(Unknown Source)
at sun.reflect.GeneratedMethodAccessor517.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.ibatis.plugin.Plugin.invoke(Plugin.java:63)
at com.sun.proxy.$Proxy475.flushStatements(Unknown Source)
at org.apache.ibatis.session.defaults.DefaultSqlSession.flushStatements(DefaultSqlSession.java:253)
at com.baomidou.mybatisplus.service.impl.ServiceImpl.insertBatch(ServiceImpl.java:131)
at com.baomidou.mybatisplus.service.impl.ServiceImpl$$FastClassBySpringCGLIB$$3e2398a4.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
at com.dbgo.acme.sale.handle.domain.service.OpeninvoiceDetailsServiceImpl$$EnhancerBySpringCGLIB$$531d76e.insertBatch(<generated>)
at com.baomidou.mybatisplus.service.impl.ServiceImpl$$FastClassBySpringCGLIB$$3e2398a4.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:99)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:282)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
at com.dbgo.platform.appasset.interceptor.FuncComponentEnabledInterceptor.around(FuncComponentEnabledInterceptor.java:45)
at sun.reflect.GeneratedMethodAccessor190.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:97)
at com.dbgo.platform.appasset.interceptor.FuncOperationEnabledInterceptor.around(FuncOperationEnabledInterceptor.java:62)
at sun.reflect.GeneratedMethodAccessor189.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:656)
at sun.reflect.GeneratedMethodAccessor1748.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:81)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

[yjjdick]

有2个问题
1.generatre的@TableId注解只要不是自增id的情况下就不会生成
2.activeRecord不支持符合主键

[qmdx]

@yjjdick 不是 Id 你数据库要设置自增， mp 不支持复合主键

[yjjdick]

@qmdx 我们数据库表很多业务情况都是用了双主键形式的表结构，不知道plus的activeRecord我这边用什么办法可以支持双主键吗，我们另外个项目用jfinal的ORM的active record是支持的我相信plus也应该有办法可以做到请教一下有什么解决方案
或者活之后plus有往active record 加双主键的支持吗，我觉得这个还是比较重要的一项支持多谢

[qmdx]

@yjjdick 要支持这个很多 ById 的意味着有问题，所以建议每一张表都给予物理主键哪怕没有意义

[qmdx]

idWorker 雪花算法 与 机器码 进程 id 时间有关系，出现该问题，注意查看每个可能的因素是否干扰

[AaronHxm]

此问题有解决的吗，我的id 用的是默认的 雪花算法，而且我还是单机情况下 就没整明白为什么重复了