Skip to content

Conversation

@chia7712
Copy link
Contributor

允許使用者將host folders掛載給broker containers,如此可達到下列兩個效果

  1. 保存資料
  2. 同時使用多顆硬碟作為資料存取

@chia7712 chia7712 requested a review from qoo332001 November 25, 2021 11:43
@chia7712
Copy link
Contributor Author

@qoo332001 麻煩協助測試,謝謝

@chia7712 chia7712 changed the title Enable to mound host folder to broker container Enable to mount host folder to broker container Nov 25, 2021
@chia7712 chia7712 changed the title Enable to mount host folder to broker container Enable to mount host folders to broker container Nov 25, 2021
@chia7712 chia7712 merged commit ca07214 into opensource4you:main Nov 25, 2021
@qoo332001
Copy link
Collaborator

@chia7712 在使用docker創建broker的時候在local可以在指定的log.dir生成log folders,但這些folders的存取權限是只有root才能使用,所以會導致沒辦法在裡面放log檔
執行指令如下:

env LOG_FOLDERS=/home/sean/Documents/log1,/home/sean/Documents/log2,/home/sean/Documents/log3 ./docker/start_broker.sh zookeeper.connect=192.168.103.39:16878

其錯誤訊息如下:

[2021-11-26 09:17:07,951] ERROR Disk error while locking directory /tmp/kafka-logs1 (kafka.server.LogDirFailureChannel)
java.nio.file.AccessDeniedException: /tmp/kafka-logs1/.lock
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:182)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)
	at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)
	at kafka.utils.FileLock.<init>(FileLock.scala:31)
	at kafka.log.LogManager.$anonfun$lockLogDirs$1(LogManager.scala:231)
	at scala.collection.StrictOptimizedIterableOps.flatMap(StrictOptimizedIterableOps.scala:117)
	at scala.collection.StrictOptimizedIterableOps.flatMap$(StrictOptimizedIterableOps.scala:104)
	at scala.collection.mutable.ArraySeq.flatMap(ArraySeq.scala:37)
	at kafka.log.LogManager.lockLogDirs(LogManager.scala:229)
	at kafka.log.LogManager.<init>(LogManager.scala:106)
	at kafka.log.LogManager$.apply(LogManager.scala:1238)
	at kafka.server.KafkaServer.startup(KafkaServer.scala:249)
	at kafka.Kafka$.main(Kafka.scala:109)
	at kafka.Kafka.main(Kafka.scala)

權限:

drwxr-xr-x  2 root root  4096 十一 26 17:22  log1
drwxr-xr-x  2 root root  4096 十一 26 17:22  log2
drwxr-xr-x  2 root root  4096 十一 26 17:22  log3

@chia7712
Copy link
Contributor Author

@qoo332001 感謝回報。這是因為host folder不存在就會變成由docker來建立,也就變成root權限,原本的用意是使用者必須先建立好資料夾並給予權限。不過這樣可能不夠有銅鋰鋅,我會發一隻PR來處理資料夾不存在時的問題。

@chia7712
Copy link
Contributor Author

@qoo332001 see #126

@chia7712 chia7712 deleted the config_folders branch December 29, 2021 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants