Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stale chains don't seem to be cleaned properly #527

Closed
longfin opened this issue Sep 24, 2019 · 4 comments

Comments

@longfin
Copy link
Member

commented Sep 24, 2019

When multiple miners running on the same network, staled chains don't seem to be cleaned properly. so nodes in that network generate errors as below.

Uploading Crash Report
LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <a587b8ea66d842adb6cba66621529de9>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <a587b8ea66d842adb6cba66621529de9>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <a587b8ea66d842adb6cba66621529de9>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass51_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:206 
Rethrow as AggregateException: One or more errors occurred.
  at Nekoyume.BlockChain.Agent+<CoSwarmRunner>d__51.MoveNext () [0x00303] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:269 
  at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00027] in <81d9c44631a841d48a9fc2f4d60ae504>:0 

planetarium.zip

@longfin longfin added the bug label Sep 24, 2019
@dahlia dahlia changed the title Staled chains don't seem to be cleaned properly Stale chains don't seem to be cleaned properly Sep 24, 2019
@earlbread

This comment has been minimized.

Copy link
Member

commented Sep 25, 2019

It seems to due to exceptions occurred in ProcessBlockHashes. In SyncPreviousBlocksAsync, we fork the canonical chain if needed. If an exception is occurred after forking, the forked chain remains and it occupies the collection names size.

@earlbread

This comment has been minimized.

Copy link
Member

commented Sep 26, 2019

Closed by #537.

@earlbread earlbread closed this Sep 26, 2019
@earlbread

This comment has been minimized.

Copy link
Member

commented Sep 26, 2019

It is reproduced again. 😢

2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [C3De50211804656366fa3c7c06d875ACE584F035]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:35 +09:00: Received [Libplanet.Net.Messages.Pong] from [C3De50211804656366fa3c7c06d875ACE584F035]...
2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [BffD802f1c8D75ceB9849F9Cb667dAdf8E66Ce40]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:35 +09:00: Received [Libplanet.Net.Messages.Pong] from [BffD802f1c8D75ceB9849F9Cb667dAdf8E66Ce40]...
2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [1BB85481E775e42f41e6c9C47363cdcAEA08A8ce]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:35 +09:00: Received [Libplanet.Net.Messages.Pong] from [1BB85481E775e42f41e6c9C47363cdcAEA08A8ce]...
2019-09-26 PM 5:16:35 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [035f7D8A5259fdB13892A8122b2094736aAc245B]...
2019-09-26 PM 5:16:35 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [035f7D8A5259fdB13892A8122b2094736aAc245B]...
2019-09-26 PM 5:16:36 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [4243b1A947E6a21B9b676a9bB0C7D16c13929199]...
2019-09-26 PM 5:16:36 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [4243b1A947E6a21B9b676a9bB0C7D16c13929199]...
2019-09-26 PM 5:16:36 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [72079f2741f20034b3973D2C3c9f8A4bF15dF4f5]...
2019-09-26 PM 5:16:36 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [72079f2741f20034b3973D2C3c9f8A4bF15dF4f5]...
2019-09-26 PM 5:16:36 +09:00: Trying to send [Libplanet.Net.Messages.Ping] to [7D1C386D345698D2f2934bcBDB8a38025348b072]...
2019-09-26 PM 5:16:36 +09:00: Message sent, waiting for reply...
2019-09-26 PM 5:16:36 +09:00: Received [Libplanet.Net.Messages.Pong] from [7D1C386D345698D2f2934bcBDB8a38025348b072]...
2019-09-26 PM 5:16:36 +09:00: initialTip? : e64bbb61d17b794abd9bebc580d4066691aeb50bbcd788868915a3171f0f0000
Preloading terminated with an exception: System.AggregateException: One or more errors occurred. ---> LiteDB.LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <d5c3afffefc14e2aae761f621a34a541>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass54_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:245 
   --- End of inner exception stack trace ---
---> (Inner Exception #0) LiteDB.LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <d5c3afffefc14e2aae761f621a34a541>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass54_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:245 <---
UnityEngine.DebugLogHandler:Internal_Log(LogType, LogOption, String, Object)
UnityEngine.DebugLogHandler:LogFormat(LogType, Object, String, Object[])
UnityEngine.Logger:LogFormat(LogType, String, Object[])
UnityEngine.Debug:LogErrorFormat(String, Object[])
Nekoyume.Debug:LogErrorFormat(String, Object[]) (at /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/Debug.cs:83)
Nekoyume.BlockChain.<CoSwarmRunner>d__54:MoveNext() (at /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:309)
UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
 
(Filename: /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/Debug.cs Line: 83)
Uploading Crash Report
LiteException: This database exceeded the maximum limit of collection names size: 3000 bytes
  at LiteDB.CollectionService.Add (System.String name) [0x00093] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.GetCollectionPage (System.String name, System.Boolean addIfNotExits) [0x00035] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Transaction[T] (System.String collection, System.Boolean addIfNotExists, System.Func`2[T,TResult] action) [0x0004e] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.Insert (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, LiteDB.BsonType autoId) [0x00041] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteEngine.InsertBulk (System.String collection, System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize, LiteDB.BsonType autoId) [0x00051] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at LiteDB.LiteCollection`1[T].InsertBulk (System.Collections.Generic.IEnumerable`1[T] docs, System.Int32 batchSize) [0x00026] in <0e7fe05a592849968cfe23dfc2a26585>:0 
  at Libplanet.Store.LiteDBStore.ForkBlockIndexes (System.Guid sourceChainId, System.Guid destinationChainId, Libplanet.HashDigest`1[T] branchPoint) [0x00033] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Blockchain.BlockChain`1[T].Fork (Libplanet.HashDigest`1[T] point) [0x00035] in <d5c3afffefc14e2aae761f621a34a541>:0 
  at Libplanet.Net.Swarm`1+<PreloadAsync>d__79[T].MoveNext () [0x00195] in <d5c3afffefc14e2aae761f621a34a541>:0 
--- End of stack trace from previous location where exception was thrown ---
  at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x0000c] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Threading.Tasks.Task task) [0x0003e] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Threading.Tasks.Task task) [0x00028] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd (System.Threading.Tasks.Task task) [0x00008] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <1f0c1ef1ad524c38bbc5536809c46b48>:0 
  at Nekoyume.BlockChain.Agent+<>c__DisplayClass54_0+<<CoSwarmRunner>b__2>d.MoveNext () [0x00085] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:245 
Rethrow as AggregateException: One or more errors occurred.
  at Nekoyume.BlockChain.Agent+<CoSwarmRunner>d__54.MoveNext () [0x00303] in /Users/earl/planetarium/nekoyume-unity/nekoyume/Assets/_Scripts/BlockChain/Agent.cs:309 
  at UnityEngine.SetupCoroutine.InvokeMoveNext (System.Collections.IEnumerator enumerator, System.IntPtr returnValueAddress) [0x00027] in <81d9c44631a841d48a9fc2f4d60ae504>:0 
 
(Filename: <0e7fe05a592849968cfe23dfc2a26585> Line: 0)
@earlbread earlbread reopened this Sep 26, 2019
@earlbread

This comment has been minimized.

Copy link
Member

commented Oct 7, 2019

This seems to be resolved by #540.

@earlbread earlbread closed this Oct 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.