Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Expose bookkeeperDiskWeightBasedPlacementEnabled in broker.conf #5254

Merged
merged 1 commit into from
Oct 2, 2019
Merged

Expose bookkeeperDiskWeightBasedPlacementEnabled in broker.conf #5254

merged 1 commit into from
Oct 2, 2019
+23 −1

Conversation

codelipenghui
Copy link
Contributor

Motivation

Allow users to enable bookkeeper DiskWeightBasedPlacement in broker.conf

Does this pull request potentially affect one of the following parts:

If yes was chosen, please highlight the changes

  • Dependencies (does it add or upgrade a dependency): (no)
  • The public API: (no)
  • The schema: (no)
  • The default values of configurations: (no)
  • The wire protocol: (no)
  • The rest endpoints: (no)
  • The admin cli options: (no)
  • Anything that affects deployment: (no)

Documentation

  • Does this pull request introduce a new feature? (no)

@codelipenghui codelipenghui self-assigned this Sep 23, 2019
@codelipenghui codelipenghui added this to the 2.5.0 milestone Sep 23, 2019
@codelipenghui
Copy link
Contributor Author

codelipenghui commented Sep 24, 2019 •
edited

When i test enable diskWeightBasedPlacement locally, standalone can't startup successful, the output log stoped at:

10:22:07.615 [main-EventThread] INFO  org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now.
10:22:07.618 [main] INFO  org.apache.distributedlog.impl.BKNamespaceDriver - Created shared client builder bk:distributedlog://127.0.0.1:2181/stream/storage:factory_writer_shared : zkServers = 127.0.0.1:2181, ledgersPath = /ledgers, numIOThreads = 6
10:22:07.623 [main] INFO  org.apache.distributedlog.impl.metadata.BKDLConfig - Propagate BKDLConfig to DLConfig : encodeRegionID = false, firstLogSegmentSequenceNumber = 1, createStreamIfNotExists = true, isFederated = false.
10:22:07.632 [main] INFO  org.apache.distributedlog.impl.BKNamespaceDriver - Initialized BK namespace driver: clientId = storage-server, regionId = 0, federated = false.
10:22:07.652 [main] INFO  org.apache.distributedlog.logsegment.LogSegmentMetadataCache - Log segment cache is enabled = true
10:22:07.652 [main] INFO  org.apache.bookkeeper.stream.server.service.DLNamespaceProviderService - Provided distributedlog namespace at distributedlog://127.0.0.1:2181/stream/storage.
10:22:07.801 [main] INFO  org.apache.bookkeeper.stream.storage.impl.sc.ZkStorageContainerManager - Watched cluster assignment data.
10:22:07.801 [main] INFO  org.apache.bookkeeper.stream.storage.impl.sc.ZkStorageContainerManager - Scheduled storage container probe task at every 15000 ms
10:22:07.822 [zk-storage-container-manager] INFO  org.apache.bookkeeper.stream.storage.impl.sc.ZkStorageContainerManager - Servers joined : [hostname: "10.2.4.175"
port: 4181
]
10:22:07.833 [main] INFO  org.apache.bookkeeper.stream.storage.impl.grpc.GrpcMetaRangeService - Created MetaRange service
10:22:07.838 [main] INFO  org.apache.bookkeeper.stream.storage.impl.grpc.GrpcTableService - Created Table service
10:22:07.871 [main] INFO  org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=10000 watcher=org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase@5e8e02a3
10:22:07.874 [main-SendThread(localhost:2181)] INFO  org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
10:22:07.875 [main-SendThread(localhost:2181)] INFO  org.apache.zookeeper.ClientCnxn - Socket connection established to localhost/127.0.0.1:2181, initiating session
10:22:07.875 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxnFactory - Accepted socket connection from /127.0.0.1:56997
10:22:07.876 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.ZooKeeperServer - Client attempting to establish new session at /127.0.0.1:56997
10:22:07.876 [SyncThread:0] INFO  org.apache.zookeeper.server.ZooKeeperServer - Established session 0x100294ba83d0007 with negotiated timeout 10000 for client /127.0.0.1:56997
10:22:07.877 [main-SendThread(localhost:2181)] INFO  org.apache.zookeeper.ClientCnxn - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x100294ba83d0007, negotiated timeout = 10000
10:22:07.877 [main-EventThread] INFO  org.apache.bookkeeper.zookeeper.ZooKeeperWatcherBase - ZooKeeper client is connected now.
10:22:07.880 [main] INFO  org.apache.bookkeeper.stream.server.service.RegistrationStateService - Successfully register myself under registration path /stream/servers/172.20.10.2:4181
10:22:07.891 [main] INFO  org.apache.bookkeeper.stream.server.service.ClusterControllerService - Successfully started the cluster controller.
10:22:08.121 [client-scheduler-OrderedScheduler-0-0] INFO  org.apache.bookkeeper.clients.impl.channel.StorageServerChannelManager - Added range server (hostname: "10.2.4.175"
port: 4181
) into the channel manager.
10:22:14.180 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a50000, timeout of 6000ms exceeded
10:22:14.180 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a50000
10:22:17.182 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a50001, timeout of 10000ms exceeded
10:22:17.182 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a50007, timeout of 10000ms exceeded
10:22:17.183 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a50001
10:22:17.183 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a50007
10:22:38.187 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a50009, timeout of 30000ms exceeded
10:22:38.188 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a50008, timeout of 30000ms exceeded
10:22:38.188 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a5000b, timeout of 30000ms exceeded
10:22:38.188 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a50006, timeout of 30000ms exceeded
10:22:38.188 [SessionTracker] INFO  org.apache.zookeeper.server.ZooKeeperServer - Expiring session 0x1002940a6a5000a, timeout of 30000ms exceeded
10:22:38.188 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a50009
10:22:38.188 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a50008
10:22:38.188 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a5000b
10:22:38.188 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a50006
10:22:38.188 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x1002940a6a5000a
^C10:23:00.961 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x100294ba83d0004 type:delete cxid:0xe zxid:0xab txntype:-1 reqpath:n/a Error Path:/stream/controller/_c_37b58eec-c0ec-4a1b-8383-8597ca137d77-lock-0000000001 Error:KeeperErrorCode = NoNode for /stream/controller/_c_37b58eec-c0ec-4a1b-8383-8597ca137d77-lock-0000000001
10:23:00.964 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x100294ba83d0007
10:23:00.964 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x100294ba83d0007
10:23:00.964 [Thread-1] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x100294ba83d0007 closed
10:23:00.965 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56997 which had sessionid 0x100294ba83d0007
10:23:00.966 [Thread-1] INFO  org.apache.distributedlog.BookKeeperClient - BookKeeper Client closed bk:distributedlog://127.0.0.1:2181/stream/storage:factory_writer_shared
10:23:00.966 [Thread-1] INFO  org.apache.distributedlog.ZooKeeperClient - Close zookeeper client dlzk:distributedlog://127.0.0.1:2181/stream/storage:factory_writer_shared.
10:23:00.966 [Thread-1] INFO  org.apache.distributedlog.ZooKeeperClient - Closing zookeeper client dlzk:distributedlog://127.0.0.1:2181/stream/storage:factory_writer_shared.
10:23:00.967 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x100294ba83d0006
10:23:00.967 [Thread-1] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x100294ba83d0006 closed
10:23:00.967 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x100294ba83d0006
10:23:00.967 [Thread-1] INFO  org.apache.distributedlog.ZooKeeperClient - Closed zookeeper client dlzk:distributedlog://127.0.0.1:2181/stream/storage:factory_writer_shared.
10:23:00.967 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56996 which had sessionid 0x100294ba83d0006
10:23:00.968 [Thread-1] INFO  org.apache.distributedlog.impl.BKNamespaceDriver - Release external resources used by channel factory.
10:23:00.968 [Thread-1] INFO  org.apache.distributedlog.impl.BKNamespaceDriver - Stopped request timer
10:23:00.970 [Thread-1] INFO  org.apache.distributedlog.BKDistributedLogNamespace - Executor Service Stopped.
10:23:00.970 [Curator-Framework-0] INFO  org.apache.curator.framework.imps.CuratorFrameworkImpl - backgroundOperationsLoop exiting
10:23:00.971 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x100294ba83d0004
10:23:00.972 [Thread-1] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x100294ba83d0004 closed
10:23:00.972 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x100294ba83d0004
10:23:00.972 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56994 which had sessionid 0x100294ba83d0004
10:23:00.972 [Thread-1] INFO  org.apache.bookkeeper.proto.BookieServer - Shutting down BookieServer
10:23:00.972 [Thread-1] INFO  org.apache.bookkeeper.proto.BookieNettyServer - Shutting down BookieNettyServer
10:23:00.978 [Thread-1] INFO  org.apache.bookkeeper.bookie.Bookie - Shutting down Bookie-3181 with exitCode 0
10:23:00.978 [Thread-1] INFO  org.apache.bookkeeper.bookie.Bookie - Turning bookie to read only during shut down
10:23:00.978 [Thread-1] INFO  org.apache.bookkeeper.bookie.SyncThread - Shutting down SyncThread
10:23:00.980 [SyncThread-7-1] INFO  org.apache.bookkeeper.bookie.SyncThread - Flush ledger storage at checkpoint CheckpointList{checkpoints=[LogMark: logFileId - 0 , logFileOffset - 0]}.
10:23:00.980 [Thread-1] INFO  org.apache.bookkeeper.bookie.Journal - Shutting down Journal
10:23:00.981 [ForceWriteThread] INFO  org.apache.bookkeeper.bookie.Journal - ForceWrite thread interrupted
10:23:00.981 [BookieJournal-3181] INFO  org.apache.bookkeeper.bookie.Journal - Journal exits when shutting down
10:23:00.982 [BookieJournal-3181] INFO  org.apache.bookkeeper.bookie.Journal - Journal exited loop!
10:23:00.982 [Thread-1] INFO  org.apache.bookkeeper.bookie.Journal - Finished Shutting down Journal thread
10:23:00.982 [Bookie-3181] INFO  org.apache.bookkeeper.bookie.Bookie - Journal thread(s) quit.
10:23:00.983 [Thread-1] INFO  org.apache.bookkeeper.bookie.GarbageCollectorThread - Shutting down GarbageCollectorThread
10:23:00.984 [Thread-1] INFO  org.apache.bookkeeper.bookie.EntryLogger - Stopping EntryLogger
10:23:00.984 [Thread-1] INFO  org.apache.bookkeeper.bookie.EntryLoggerAllocator - Stopped entry logger preallocator.
10:23:00.985 [Thread-1] INFO  org.apache.bookkeeper.bookie.LedgerDirsMonitor - Shutting down LedgerDirsMonitor
10:23:00.986 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x100294ba83d0001
10:23:00.987 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x100294ba83d0001
10:23:00.987 [Thread-1] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x100294ba83d0001 closed
10:23:00.987 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56987 which had sessionid 0x100294ba83d0001
10:23:00.987 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Processed session termination for sessionid: 0x100294ba83d0000
10:23:00.988 [Thread-1] INFO  org.apache.zookeeper.ZooKeeper - Session: 0x100294ba83d0000 closed
10:23:00.988 [main-EventThread] INFO  org.apache.zookeeper.ClientCnxn - EventThread shut down for session: 0x100294ba83d0000
10:23:00.988 [Thread-1] INFO  org.apache.zookeeper.server.ZooKeeperServer - shutting down
10:23:00.988 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxn - Closed socket connection for client /127.0.0.1:56986 which had sessionid 0x100294ba83d0000
10:23:00.988 [Thread-1] INFO  org.apache.zookeeper.server.SessionTrackerImpl - Shutting down
10:23:00.988 [Thread-1] INFO  org.apache.zookeeper.server.PrepRequestProcessor - Shutting down
10:23:00.988 [Thread-1] INFO  org.apache.zookeeper.server.SyncRequestProcessor - Shutting down
10:23:00.988 [ProcessThread(sid:0 cport:2181):] INFO  org.apache.zookeeper.server.PrepRequestProcessor - PrepRequestProcessor exited loop!
10:23:00.988 [SyncThread:0] INFO  org.apache.zookeeper.server.SyncRequestProcessor - SyncRequestProcessor exited!
10:23:00.988 [Thread-1] INFO  org.apache.zookeeper.server.FinalRequestProcessor - shutdown of request processor complete
10:23:00.989 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181] INFO  org.apache.zookeeper.server.NIOServerCnxnFactory - NIOServerCnxn factory exited run method```

@sijie
Copy link
Member

sijie commented Sep 24, 2019

@codelipenghui weight based placement policy requires configuring bookies to use v3 protocol. there are two options:

  1. make changes to managed ledger to allow using v3 protocol. so that you can use both.

or

  1. wait until we release bookkeeper 4.10. 4.10 will support a mixed mode of both v2 and v3. the release of bookkeeper 4.10 is under discussion.

@codelipenghui
Copy link
Contributor Author

@sijie Thanks, broker start succeed after set bookkeeperUseV2WireProtocol=false in broker.conf

@sijie sijie merged commit 6542c65 into apache:master Oct 2, 2019
@codelipenghui codelipenghui deleted the config_disk_weight branch May 19, 2021 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sijie sijie sijie approved these changes

@jiazhai jiazhai jiazhai approved these changes

@merlimat merlimat Awaiting requested review from merlimat

@rdhabalia rdhabalia Awaiting requested review from rdhabalia

Assignees

@codelipenghui codelipenghui

Labels
area/broker area/config
Projects
None yet
Milestone
2.5.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@codelipenghui @sijie @jiazhai