Hbase can't start

[xiehurricane]

Thank you for sharing this project.
I have an error when docker-compose up.
Could i get some help?
output:
`
pinpoint-hbase | org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/hbase,42832,1492678149833
pinpoint-hbase | at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
pinpoint-hbase | at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
pinpoint-hbase | at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
pinpoint-hbase | at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:178)
pinpoint-hbase | at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1236)
pinpoint-hbase | at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1225)
pinpoint-hbase | at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1416)
pinpoint-hbase | at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1090)
pinpoint-hbase | at java.lang.Thread.run(Thread.java:745)
pinpoint-hbase | 2017-04-20 08:53:31,761 INFO [RS:0;hbase:42832] regionserver.HRegionServer: stopping server hbase,42832,1492678149833; zookeeper connection closed.
pinpoint-hbase | 2017-04-20 08:53:31,761 INFO [RS:0;hbase:42832] regionserver.HRegionServer: RS:0;hbase:42832 exiting
pinpoint-hbase | 2017-04-20 08:53:31,772 INFO [Thread-6] regionserver.ShutdownHook: Starting fs shutdown hook thread.
pinpoint-hbase | 2017-04-20 08:53:31,773 INFO [Thread-6] regionserver.ShutdownHook: Shutdown hook finished.

`

[minyk]

Hi @xiehurricane
It looks like hbase to zookeeper connection is lost.
can you share full log of hbase container? Maybe Paste it comment or attach as file.

[xiehurricane]

@minyk

2017-04-20 10:28:26,827 INFO [hbase,45891,1492683714564_ChoreService_1] regionserver.HRegionServer: hbase,45891,1492683714564-MemstoreFlusherChore requesting flush of hbase:meta,,1.1588230740 because info has an old edit so flush to free WALs after random delay 284889ms 2017-04-20 10:28:28,409 INFO [MemStoreFlusher.0] regionserver.HRegion: Flushing 1/1 column families, memstore=360.18 KB 2017-04-20 10:28:28,470 INFO [MemStoreFlusher.0] regionserver.DefaultStoreFlusher: Flushed, sequenceid=2131, memsize=360.2 K, hasBloomFilter=false, into tmp file file:/home/pinpoint/hbase/data/hbase/meta/1588230740/.tmp/07f5cf7cbed24a16b9ed3aef430a2556 2017-04-20 10:28:28,475 INFO [MemStoreFlusher.0] regionserver.HStore: Added file:/home/pinpoint/hbase/data/hbase/meta/1588230740/info/07f5cf7cbed24a16b9ed3aef430a2556, entries=1491, sequenceid=2131, filesize=181.7 K 2017-04-20 10:28:28,476 INFO [MemStoreFlusher.0] regionserver.HRegion: Finished memstore flush of ~360.18 KB/368824, currentsize=0 B/0 for region hbase:meta,,1.1588230740 in 67ms, sequenceid=2131, compaction requested=true 2017-04-20 10:28:28,489 INFO [RS:0;hbase:45891-shortCompactions-1492684108479] regionserver.HRegion: Starting compaction on info in region hbase:meta,,1.1588230740 2017-04-20 10:28:28,489 INFO [RS:0;hbase:45891-shortCompactions-1492684108479] regionserver.HStore: Starting compaction of 3 file(s) in info of hbase:meta,,1.1588230740 into tmpdir=file:/home/pinpoint/hbase/data/hbase/meta/1588230740/.tmp, totalSize=845.2 K 2017-04-20 10:28:28,667 INFO [RS:0;hbase:45891-shortCompactions-1492684108479] regionserver.HStore: Completed compaction of 3 (all) file(s) in info of hbase:meta,,1.1588230740 into 8282d38c12ed4f9c88db7ad4bfe7ae84(size=835.9 K), total size for store is 835.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2017-04-20 10:28:28,669 INFO [RS:0;hbase:45891-shortCompactions-1492684108479] regionserver.CompactSplitThread: Completed compaction: Request = regionName=hbase:meta,,1.1588230740, storeName=info, fileCount=3, fileSize=845.2 K, priority=7, time=3809014281124715; duration=0sec 2017-04-20 10:29:41,691 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /172.17.0.3:41426 2017-04-20 10:29:41,706 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /172.17.0.3:41426 2017-04-20 10:29:41,709 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x15b8ae1e1660006 with negotiated timeout 40000 for client /172.17.0.3:41426 2017-04-20 10:29:42,549 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /172.17.0.3:41428 2017-04-20 10:29:42,551 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to establish new session at /172.17.0.3:41428 2017-04-20 10:29:42,558 INFO [SyncThread:0] server.ZooKeeperServer: Established session 0x15b8ae1e1660007 with negotiated timeout 40000 for client /172.17.0.3:41428 2017-04-20 10:32:05,805 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=2.44 MB, freeSize=1.52 GB, max=1.53 GB, blockCount=101, accesses=605, hits=401, hitRatio=66.28%, , cachingAccesses=502, cachingHits=320, cachingHitsRatio=63.75%, evictions=59, evicted=81, evictedPerRun=1.3728814125061035 2017-04-20 10:32:13,744 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: Stopping HBase metrics system... 2017-04-20 10:32:13,749 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase metrics system stopped. 2017-04-20 10:32:14,353 INFO [HBase-Metrics2-1] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties 2017-04-20 10:32:14,356 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2017-04-20 10:32:14,357 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase metrics system started 2017-04-20 10:35:32,039 INFO [1842387105@qtp-1648232591-7] logs: Aliases are enabled 2017-04-20 10:37:05,805 INFO [LruBlockCacheStatsExecutor] hfile.LruBlockCache: totalSize=2.47 MB, freeSize=1.52 GB, max=1.53 GB, blockCount=104, accesses=707, hits=500, hitRatio=70.72%, , cachingAccesses=604, cachingHits=419, cachingHitsRatio=69.37%, evictions=89, evicted=81, evictedPerRun=0.9101123809814453 2017-04-20 10:37:13,744 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: Stopping HBase metrics system... 2017-04-20 10:37:13,747 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase metrics system stopped. 2017-04-20 10:37:14,304 INFO [HBase-Metrics2-1] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties 2017-04-20 10:37:14,307 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2017-04-20 10:37:14,307 INFO [HBase-Metrics2-1] impl.MetricsSystemImpl: HBase metrics system started 2017-04-20 10:37:50,000 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660001, timeout of 10000ms exceeded 2017-04-20 10:38:52,437 WARN [M:0;hbase:45249] util.Sleeper: We slept 48958ms instead of 3000ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2017-04-20 10:39:10,593 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660005, timeout of 10000ms exceeded 2017-04-20 10:39:11,133 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660004, timeout of 10000ms exceeded 2017-04-20 10:39:11,756 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660000, timeout of 10000ms exceeded 2017-04-20 10:39:22,954 INFO [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 6669ms for sessionid 0x15b8ae1e1660000, closing socket connection and attempting reconnect 2017-04-20 10:39:39,004 INFO [M:0;hbase:45249-SendThread(localhost:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 6669ms for sessionid 0x15b8ae1e1660002, closing socket connection and attempting reconnect 2017-04-20 10:39:38,993 INFO [RS:0;hbase:45891-SendThread(localhost:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 6978ms for sessionid 0x15b8ae1e1660003, closing socket connection and attempting reconnect 2017-04-20 10:39:39,738 WARN [M:0;hbase:45249] util.Sleeper: We slept 13156ms instead of 3000ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2017-04-20 10:39:26,553 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660003, timeout of 10000ms exceeded 2017-04-20 10:39:25,330 INFO [hbase:45249.activeMasterManager-SendThread(localhost:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 7031ms for sessionid 0x15b8ae1e1660005, closing socket connection and attempting reconnect 2017-04-20 10:39:25,331 INFO [hbase:45249.activeMasterManager-SendThread(localhost:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 6668ms for sessionid 0x15b8ae1e1660004, closing socket connection and attempting reconnect 2017-04-20 10:39:25,997 INFO [main-SendThread(localhost:2181)] zookeeper.ClientCnxn: Client session timed out, have not heard from server in 8377ms for sessionid 0x15b8ae1e1660001, closing socket connection and attempting reconnect 2017-04-20 10:39:43,291 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660002, timeout of 10000ms exceeded 2017-04-20 10:39:43,914 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660007, timeout of 40000ms exceeded 2017-04-20 10:39:43,914 INFO [SessionTracker] server.ZooKeeperServer: Expiring session 0x15b8ae1e1660006, timeout of 40000ms exceed 2017-04-20 10:40:20,578 WARN [M:0;hbase:45249] util.Sleeper: We slept 29719ms instead of 3000ms, this is likely due to a long garbage collecting pause and it's usually bad, see http://hbase.apache.org/book.html#trouble.rs.runtime.zkexpired 2017-04-20 10:40:21,581 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x15b8ae1e1660001 2017-04-20 10:40:21,581 INFO [hbase,45249,1492683712701_splitLogManager__ChoreService_1] hbase.ScheduledChore: Chore: SplitLogManager Timeout Monitor missed its start time 2017-04-20 10:40:31,416 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x15b8ae1e1660005 2017-04-20 10:40:32,530 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x15b8ae1e1660004 2017-04-20 10:40:42,062 INFO [ProcessThread(sid:0 cport:-1):] server.PrepRequestProcessor: Processed session termination for sessionid: 0x15b8ae1e1660000

2017-04-20 10:42:52,595 INFO [StoreCloserThread-SqlMetaData_Ver2,\x06\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695721670.88127bf1df3b7777c608e9810d46d4e6.-1] regionserver.HStore: Closed Sql 2017-04-20 10:42:52,596 INFO [StoreCloserThread-ApplicationMapStatisticsCaller_Ver2,\x1C\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695730600.97271c488694941bccd208c2554e108e.-1] regionserver.HStore: Closed C 2017-04-20 10:42:52,596 INFO [RS_CLOSE_REGION-hbase:45891-0] regionserver.HRegion: Closed SqlMetaData_Ver2,\x06\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695721670.88127bf1df3b7777c608e9810d46d4e6. 2017-04-20 10:42:52,596 INFO [RS_CLOSE_REGION-hbase:45891-2] regionserver.HRegion: Closed ApplicationMapStatisticsCaller_Ver2,\x1C\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695730600.97271c488694941bccd208c2554e108e. 2017-04-20 10:42:52,596 INFO [StoreCloserThread-SqlMetaData_Ver2,,1481695721670.6bb75a178d1a377355e98c6781d3e421.-1] regionserver.HStore: Closed Sql 2017-04-20 10:42:52,597 INFO [RS_CLOSE_REGION-hbase:45891-1] regionserver.HRegion: Closed SqlMetaData_Ver2,,1481695721670.6bb75a178d1a377355e98c6781d3e421. 2017-04-20 10:42:52,597 INFO [StoreCloserThread-AgentStatV2,\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695713326.a1df076901b889ab8a3027e16e593647.-1] regionserver.HStore: Closed S 2017-04-20 10:42:52,597 INFO [StoreCloserThread-TraceV2,U\x00\x00\x00\x00\x00\x00\x00,1481695725131.fcabba9fe12e8e5d7a320cbaa458e7cf.-1] regionserver.HStore: Closed S 2017-04-20 10:42:52,598 INFO [RS_CLOSE_REGION-hbase:45891-1] regionserver.HRegion: Closed TraceV2,U\x00\x00\x00\x00\x00\x00\x00,1481695725131.fcabba9fe12e8e5d7a320cbaa458e7cf. 2017-04-20 10:42:52,598 INFO [RS_CLOSE_REGION-hbase:45891-0] regionserver.HRegion: Closed AgentStatV2,\x0E\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695713326.a1df076901b889ab8a3027e16e593647. 2017-04-20 10:42:52,600 INFO [StoreCloserThread-Traces,\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695722897.8a3f016bb818aeb75700da09623a7f40.-1] regionserver.HStore: Closed A 2017-04-20 10:42:52,600 INFO [StoreCloserThread-Traces,\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695722897.8a3f016bb818aeb75700da09623a7f40.-1] regionserver.HStore: Closed S 2017-04-20 10:42:52,600 INFO [StoreCloserThread-Traces,\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695722897.8a3f016bb818aeb75700da09623a7f40.-1] regionserver.HStore: Closed T 2017-04-20 10:42:52,600 INFO [RS_CLOSE_REGION-hbase:45891-2] regionserver.HRegion: Closed Traces,\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00,1481695722897.8a3f016bb818aeb75700da09623a7f40. 2017-04-20 10:42:52,672 INFO [regionserver/hbase/172.17.0.2:0.leaseChecker] regionserver.Leases: regionserver/hbase/172.17.0.2:0.leaseChecker closing leases 2017-04-20 10:42:52,672 INFO [regionserver/hbase/172.17.0.2:0.leaseChecker] regionserver.Leases: regionserver/hbase/172.17.0.2:0.leaseChecker closed leases 2017-04-20 10:42:52,680 INFO [RS:0;hbase:45891] regionserver.HRegionServer: stopping server hbase,45891,1492683714564; all regions closed. 2017-04-20 10:42:52,700 INFO [RS:0;hbase:45891] regionserver.Leases: RS:0;hbase:45891 closing leases 2017-04-20 10:42:52,749 INFO [RS:0;hbase:45891] regionserver.Leases: RS:0;hbase:45891 closed leases 2017-04-20 10:42:52,780 INFO [RS:0;hbase:45891] hbase.ChoreService: Chore service for: hbase,45891,1492683714564 had [[ScheduledChore: Name: hbase,45891,1492683714564-MemstoreFlusherChore Period: 10000 Unit: MILLISECONDS], [ScheduledChore: Name: MovedRegionsCleaner for region hbase,45891,1492683714564 Period: 120000 Unit: MILLISECONDS]] on shutdown 2017-04-20 10:42:52,975 INFO [RS_OPEN_META-hbase:45891-0-MetaLogRoller] regionserver.LogRoller: LogRoller exiting. 2017-04-20 10:42:52,975 INFO [regionserver/hbase/172.17.0.2:0.logRoller] regionserver.LogRoller: LogRoller exiting. 2017-04-20 10:42:52,978 WARN [RS:0;hbase:45891] zookeeper.ZKUtil: regionserver:45891-0x15b8ae1e1660001, quorum=localhost:2181, baseZNode=/hbase Unable to list children of znode /hbase/replication/rs/hbase,45891,1492683714564 org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/replication/rs/hbase,45891,1492683714564 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getChildren(RecoverableZooKeeper.java:292) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:456) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchThem(ZKUtil.java:484) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenBFSAndWatchThem(ZKUtil.java:1476) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeRecursivelyMultiOrSequential(ZKUtil.java:1398) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeRecursively(ZKUtil.java:1280) at org.apache.hadoop.hbase.replication.ReplicationQueuesZKImpl.removeAllQueues(ReplicationQueuesZKImpl.java:188) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:310) at org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:180) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:172) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2151) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1082) at java.lang.Thread.run(Thread.java:745) 2017-04-20 10:42:52,979 ERROR [RS:0;hbase:45891] zookeeper.ZooKeeperWatcher: regionserver:45891-0x15b8ae1e1660001, quorum=localhost:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/replication/rs/hbase,45891,1492683714564 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1472) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getChildren(RecoverableZooKeeper.java:292) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchForNewChildren(ZKUtil.java:456) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenAndWatchThem(ZKUtil.java:484) at org.apache.hadoop.hbase.zookeeper.ZKUtil.listChildrenBFSAndWatchThem(ZKUtil.java:1476) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeRecursivelyMultiOrSequential(ZKUtil.java:1398) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNodeRecursively(ZKUtil.java:1280) at org.apache.hadoop.hbase.replication.ReplicationQueuesZKImpl.removeAllQueues(ReplicationQueuesZKImpl.java:188) at org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.join(ReplicationSourceManager.java:310) at org.apache.hadoop.hbase.replication.regionserver.Replication.join(Replication.java:180) at org.apache.hadoop.hbase.replication.regionserver.Replication.stopReplicationService(Replication.java:172) at org.apache.hadoop.hbase.regionserver.HRegionServer.stopServiceThreads(HRegionServer.java:2151) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1082) at java.lang.Thread.run(Thread.java:745) 2017-04-20 10:42:52,980 INFO [RS:0;hbase:45891] ipc.RpcServer: Stopping server on 45891 2017-04-20 10:42:53,005 INFO [RpcServer.listener,port=45891] ipc.RpcServer: RpcServer.listener,port=45891: stopping 2017-04-20 10:42:53,046 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopped 2017-04-20 10:42:53,046 INFO [RpcServer.responder] ipc.RpcServer: RpcServer.responder: stopping 2017-04-20 10:42:53,047 WARN [RS:0;hbase:45891] regionserver.HRegionServer: Failed deleting my ephemeral node org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /hbase/rs/hbase,45891,1492683714564 at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:178) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1236) at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1225) at org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(HRegionServer.java:1416) at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1090) at java.lang.Thread.run(Thread.java:745) 2017-04-20 10:42:53,047 INFO [RS:0;hbase:45891] regionserver.HRegionServer: stopping server hbase,45891,1492683714564; zookeeper connection closed. 2017-04-20 10:42:53,047 INFO [RS:0;hbase:45891] regionserver.HRegionServer: RS:0;hbase:45891 exiting 2017-04-20 10:42:53,200 INFO [Thread-6] regionserver.ShutdownHook: Shutdown hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@37e4d7bb 2017-04-20 10:42:53,202 INFO [Thread-6] regionserver.ShutdownHook: Starting fs shutdown hook thread. 2017-04-20 10:42:53,204 INFO [Thread-6] regionserver.ShutdownHook: Shutdown hook finished. [hxgy@localhost pinpoint]$ ^C [hxgy@localhost pinpoint]$ docker logs -f --tail 10 1af95a60c258 2017-04-20 10:43:05,859 INFO [Thread-2] server.NIOServerCnxn: Closed socket connection for client /127.0.0.1:36444 (no session established for client) 2017-04-20 10:43:05,947 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /172.17.0.3:42048 2017-04-20 10:43:06,185 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to renew session 0x15b8ae1e1660007 at /172.17.0.3:42048 2017-04-20 10:43:06,187 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Invalid session 0x15b8ae1e1660007 for client /172.17.0.3:42048, probably expired 2017-04-20 10:43:06,390 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /172.17.0.3:42048 which had sessionid 0x15b8ae1e1660007 2017-04-20 10:43:07,260 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxnFactory: Accepted socket connection from /172.17.0.3:42052 2017-04-20 10:43:07,261 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Client attempting to renew session 0x15b8ae1e1660006 at /172.17.0.3:42052 2017-04-20 10:43:07,262 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.ZooKeeperServer: Invalid session 0x15b8ae1e1660006 for client /172.17.0.3:42052, probably expired 2017-04-20 10:43:07,265 INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] server.NIOServerCnxn: Closed socket connection for client /172.17.0.3:42052 which had sessionid 0x15b8ae1e1660006 2017-04-20 10:43:07,719 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

[xiehurricane]

logs.zip

[minyk]

@xiehurricane Sorry for late.

IMO, HBase could not handle the data size. Are you running the container for some production environments? This repo's configuration is for test purpose, so if you want to run on production, you should build HBase cluster separately.

If this is not production env(and data doesn't need to be preserved), wiped out hbase data dir, and start things like first up.

Thanks.

[xiehurricane]

@minyk Thank you for reply.
I never start the testApp yet.After collector and WebUI started, errors came out.
I will move this project to other machine and try again.

[minyk]

@xiehurricane hmm, I see. I'll seek the log more.

[minyk]

@xiehurricane Can you send full console log from hbase container? The log.zip inside of container is only for table creation at image build time, not runtime. The log on comment is missing time. Between 2017-04-20 10:40:42,062 and 2017-04-20 10:42:52,595 something happen. At 2017-04-20 10:42:52,595, the regions on regionserver are already starting to close.

[xiehurricane]

@minyk Sorry I got a little busy.Working in weekend you know o(╯□╰)o.
Searching about how to export docker logs now.

[xiehurricane]

@minyk
Old log is missing and it's ok runing on my mac. I have to give up digging this error, working on some hurry-up project.
But your answer is meaningful to me. �It's warm. 😁

[minyk]

@xiehurricane ok, maybe next time ;-)