Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parallel VM creation might fail #121

Closed
youyk opened this issue Sep 3, 2015 · 2 comments
Closed

Parallel VM creation might fail #121

youyk opened this issue Sep 3, 2015 · 2 comments
Labels
bug
Milestone
0.9

Comments

@youyk
Copy link
Contributor

youyk commented Sep 3, 2015

The call trace is like:

2015-09-04 06:57:47,724 DEBUG SimpleFlowChain [FlowChain: create-vm-7bee9185169c4986bb4a20bb9b1c593c] successfully executed flow[VmImageSelectBackupStorageFlow]
2015-09-04 06:57:47,723 DEBUG HostAllocatorChain org.springframework.dao.DataIntegrityViolationException
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:193) ~[spring-orm-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:155) ~[spring-orm-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:519) ~[spring-orm-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:757) ~[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:726) ~[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:478) ~[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:272) ~[spring-tx-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:63) ~[spring-aspects-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.springframework.transaction.aspectj.AbstractTransactionAspect.ajc$around$org_springframework_transaction_aspectj_AbstractTransactionAspect$1$2a73e96c(AbstractTransactionAspect.aj:63) ~[spring-aspects-4.0.3.RELEASE.jar:4.0.3.RELEASE]
at org.zstack.compute.allocator.HostAllocatorChain.reserveCapacity(HostAllocatorChain.java:77) ~[compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.done(HostAllocatorChain.java:135) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:230) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.FilterFlow.allocate(FilterFlow.java:36) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:226) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.TagAllocatorFlow.allocate(TagAllocatorFlow.java:117) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:226) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.AvoidHostAllocatorFlow.allocate(AvoidHostAllocatorFlow.java:33) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:226) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.HostPrimaryStorageAllocatorFlow.allocate(HostPrimaryStorageAllocatorFlow.java:84) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:226) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.ImageBackupStorageAllocatorFlow.allocate(ImageBackupStorageAllocatorFlow.java:70) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:226) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.HostCapacityAllocatorFlow.allocate(HostCapacityAllocatorFlow.java:65) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:226) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.HostStateAndHypervisorAllocatorFlow.allocate(HostStateAndHypervisorAllocatorFlow.java:72) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.next(HostAllocatorChain.java:226) [compute-0.9.0.jar:?]
at org.zstack.header.allocator.AbstractHostAllocatorFlow.next(AbstractHostAllocatorFlow.java:56) [header-0.9.0.jar:?]
at org.zstack.compute.allocator.AttachedL2NetworkAllocatorFlow.allocate(AttachedL2NetworkAllocatorFlow.java:113) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.runFlow(HostAllocatorChain.java:172) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.start(HostAllocatorChain.java:195) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.allocate_aroundBody2(HostAllocatorChain.java:201) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain$AjcClosure3.run(HostAllocatorChain.java:1) [compute-0.9.0.jar:?]
at org.zstack.core.aspect.AsyncSafeAspect.ajc$around$org_zstack_core_aspect_AsyncSafeAspect$1$c4af5facproceed(AsyncSafeAspect.aj:1) [core-0.9.0.jar:?]
at org.zstack.core.aspect.AsyncSafeAspect.ajc$around$org_zstack_core_aspect_AsyncSafeAspect$1$c4af5fac(AsyncSafeAspect.aj:84) [core-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.allocate(HostAllocatorChain.java:198) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.allocate_aroundBody6(HostAllocatorChain.java:254) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain$AjcClosure7.run(HostAllocatorChain.java:1) [compute-0.9.0.jar:?]
at org.zstack.core.aspect.AsyncSafeAspect.ajc$around$org_zstack_core_aspect_AsyncSafeAspect$1$c4af5facproceed(AsyncSafeAspect.aj:1) [core-0.9.0.jar:?]
at org.zstack.core.aspect.AsyncSafeAspect.ajc$around$org_zstack_core_aspect_AsyncSafeAspect$1$c4af5fac(AsyncSafeAspect.aj:84) [core-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorChain.allocate(HostAllocatorChain.java:252) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorManagerImpl.handle(HostAllocatorManagerImpl.java:118) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorManagerImpl.handleLocalMessage(HostAllocatorManagerImpl.java:57) [compute-0.9.0.jar:?]
at org.zstack.compute.allocator.HostAllocatorManagerImpl.handleMessage(HostAllocatorManagerImpl.java:51) [compute-0.9.0.jar:?]
at org.zstack.core.cloudbus.CloudBusImpl2$11$1$1.call(CloudBusImpl2.java:1818) [core-0.9.0.jar:?]
at org.zstack.core.cloudbus.CloudBusImpl2$11$1$1.call(CloudBusImpl2.java:1) [core-0.9.0.jar:?]
at org.zstack.core.thread.ThreadFacadeImpl$Worker.call(ThreadFacadeImpl.java:110) [core-0.9.0.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:262) [?:1.7.0_85]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) [?:1.7.0_85]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) [?:1.7.0_85]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [?:1.7.0_85]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [?:1.7.0_85]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_85]
Caused by: org.hibernate.exception.DataException: could not execute statement
at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:71) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:124) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:189) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:58) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3236) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3138) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3468) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:140) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:395) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:387) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:304) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:349) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1159) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:404) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:75) ~[hibernate-entitymanager-4.2.11.Final.jar:4.2.11.Final]
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:515) ~[spring-orm-4.0.3.RELEASE.jar:4.0.3.RELEASE]
... 62 more
Caused by: com.mysql.jdbc.MysqlDataTruncation: Data truncation: Out of range value for column 'availableMemory' at row 1
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4224) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4158) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2840) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2082) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2334) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2262) ~[mysql-connector-java-5.1.31.jar:?]
at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2246) ~[mysql-connector-java-5.1.31.jar:?]
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:435) ~[c3p0-0.9.5-pre10.jar:0.9.5-pre10]
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:186) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.jdbc.batch.internal.NonBatchingBatch.addToBatch(NonBatchingBatch.java:58) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3236) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:3138) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:3468) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.action.internal.EntityUpdateAction.execute(EntityUpdateAction.java:140) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.spi.ActionQueue.execute(ActionQueue.java:395) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:387) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:304) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:349) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1159) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:404) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.transaction.internal.jdbc.JdbcTransaction.beforeTransactionCommit(JdbcTransaction.java:101) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.engine.transaction.spi.AbstractTransactionImpl.commit(AbstractTransactionImpl.java:175) ~[hibernate-core-4.2.11.Final.jar:4.2.11.Final]
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:75) ~[hibernate-entitymanager-4.2.11.Final.jar:4.2.11.Final]
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:515) ~[spring-orm-4.0.3.RELEASE.jar:4.0.3.RELEASE]
... 62 more
2015-09-04 06:57:47,724 TRACE CloudBusImpl2 [msg send]: {"org.zstack.header.allocator.AllocateHostReply":{"success":false,"error":{"code":"SYS.1000","description":"An internal error happened in system","details":"could not execute statement; SQL [n/a]; nested exception is org.hibernate.exception.DataException: could not execute statement"},"headers":{"isReply":"true","correlationId":"9d026c8091dd4e03bae4a73045fdeefa","schema":{}},"id":"6d8541d480e34726a826a1018601dfe5","serviceId":"zstack.message.cloudbus.fd71fa102f2e4e359b7bb7840a425fc4","creatingTime":1441321067631}}
@youyk youyk added the bug label Sep 3, 2015
@youyk youyk added this to the 0.9 milestone Sep 3, 2015
@youyk
Copy link
Contributor Author

youyk commented Sep 6, 2015

This issue happens when host resource is not enough for all VM to be created up.

zxwing added a commit that referenced this issue Sep 9, 2015
@zxwing
Parallel VM creation might fail #121
54e3f22 
#121
@zstackorg
Copy link
Contributor

fixed in 54e3f22

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
0.9
Development

No branches or pull requests
2 participants
@youyk @zstackorg