KAFKA-3940 Log should check the return value of dir.mkdirs() #1748
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This commit changes all the occurrences of dir.mkdirs() with Files.createDirectory(dir.toPath())
…ng in Selector Author: Manikumar Reddy O <manikumar.reddy@gmail.com> Reviewers: Jun Rao <junrao@gmail.com> Closes apache#1750 from omkreddy/KAFKA-4044-LOG
Author: Jason Gustafson <jason@confluent.io> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io>, Ismael Juma <ismael@juma.me.uk>, Guozhang Wang <wangguoz@gmail.com> Closes apache#1627 from hachikuji/KAFKA-3888
Fixed the failing errors in the previous commit that changed occurrences of dir.mkdirs() to Files.createDirectory()
…eteTopicRequests Author: Grant Henke <granthenke@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes apache#1737 from granthenke/transient-delete
Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes apache#1733 from ewencp/rest-api-retries
Author: Vahid Hashemian <vahidhashemian@us.ibm.com> Reviewers: Dana Powers, Gwen Shapira Closes apache#1732 from vahidhashemian/minor/clarify_producer_config_documentation
On Windows, the following output is seen when starting Zookeeper and Kafka servers: ``` '#' is not recognized as an internal or external command, operable program or batch file. ``` This pull request makes a minor correction to the Windows `kafka-run-class.bat` script to replace the use of `#` with `rem`. Author: P. Thorpe <paul.thorpe@uk.ibm.com> Reviewers: Vahid Hashemian, Gwen Shapira Closes apache#1740 from p-thorpe/trunk
Author: Ewen Cheslack-Postava <me@ewencp.org> Reviewers: Shikhar Bhushan, Gwen Shapira Closes apache#1721 from ewencp/kafka-3845-per-connector-converters
| try { | ||
| Files.createDirectory(dir.toPath()) | ||
| } catch { | ||
| case e: IOException => throw new KafkaException("Error in creating new directory.", e) |
There was a problem hiding this comment.
Since we are wrapping the IOException in a KafkaException I think it may make sense to add the Kafka context to the error message. Something like:
throw new KafkaException("Error creating log directory ${dir.toPath}", e)Same for the other occurrences in this PR.
|
In the Jira @ijuma mentioned replacing |
|
Maybe we can do that in a separate PR to make it easier to review and merge. |
| } catch { | ||
| case e: IOException => throw new KafkaException("Error in creating new directory '%s'".format(csvDir), e) | ||
| } | ||
| Files.createDirectory(csvDir.toPath()) |
There was a problem hiding this comment.
My bad, will fix it
This patch is for KIP-33. https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index Author: Jiangjie Qin <becket.qin@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Jun Rao <junrao@gmail.com>, Liquan Pei <liquanpei@gmail.com> Closes apache#1215 from becketqin/KAFKA-3163
Add an optional configuration for the SecureRandom PRNG implementation, with the default behavior being the same (use the default implementation in the JDK/JRE). Author: Todd Palino <Todd Palino> Reviewers: Grant Henke <granthenke@gmail.com>, Ismael Juma <ismael@juma.me.uk>, Joel Koshy <jjkoshy@gmail.com>, Jiangjie Qin <becket.qin@gmail.com>, Rajini Sivaram <rajinisivaram@googlemail.com> Closes apache#1747 from toddpalino/trunk
In case of unknown configs, only list the name without the value Author: Mickael Maison <mickael.maison@gmail.com> Reviewers: Jaikiran, Gwen Shapira, Grant Henke, Ryan Pridgeon, Dustin Cote Closes apache#1759 from mimaison/KAFKA-4056
Author: Shuai Zhang <shuai.xyz@gmail.com> Reviewers: Gwen Shapira Closes apache#1751 from sh-z/KAFKA-4053
Author: Eno Thereska <eno.thereska@gmail.com> Reviewers: Ismael Juma, Damian Guy, Guozhang Wang Closes apache#1700 from enothereska/join-benchmarks
…balance Author: Jason Gustafson <jason@confluent.io> Reviewers: Vahid Hashemian, Guozhang Wang Closes apache#1762 from hachikuji/KAFKA-3949
junrao Could you take a look when get a chance? Thanks. Author: Jiangjie Qin <becket.qin@gmail.com> Reviewers: Jun Rao <junrao@gmail.com> Closes apache#1769 from becketqin/KAFKA-3163-follow-up
As discussed in https://issues.apache.org/jira/browse/KAFKA-3894, this PR makes the log cleaner do a "partial" clean on a segment, whereby it builds a partial offset map up to a particular offset in a segment. Once cleaning resumes again, we will continue from the next dirty offset, which can now be located in the middle of a segment. Prior to this PR, segments with overly numerous keys could crash the log cleaner thread, as it was required that the log cleaner had to fit at least a single segment in the offset map. Author: Tom Crayford <tcrayford@googlemail.com> Reviewers: Jun Rao <junrao@gmail.com> Closes apache#1725 from tcrayford/dont_crash_log_cleaner_thread_if_segment_overflows_buffer
Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes apache#1763 from rajinisivaram/KAFKA-4066
1. I think the instructions in step 2 of the security section which describe adding the CA to server/client truststores are swapped. That is, the instruction that says to add the CA to the server truststore adds it to the client truststore (and vice versa).
2. "clients keys" should be possessive ("clients' keys").
This contribution is my original work, and I license the work to the project under the project's open source license.
Author: Samuel Taylor <staylor@square-root.com>
Reviewers: Ismael Juma <ismael@juma.me.uk>
Closes apache#1651 from ssaamm/trunk
…n is down Author: Matthias J. Sax <matthias@confluent.io> Reviewers: Guozhang Wang <wangguoz@gmail.com> Closes apache#1764 from mjsax/improveResetTool
Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes apache#1731 from guozhangwang/Kminor-log4j-streams-examples
Use System.nanoseconds instead of System.currentTimeMillis in broker timer tasks to cope with changes to wall-clock time. Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Gwen Shapira Closes apache#1768 from rajinisivaram/KAFKA-4051
Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Ismael Juma <ismael@juma.me.uk>, Ewen Cheslack-Postava <me@ewencp.org> Closes apache#1746 from guozhangwang/K4049-RegexSourceIntegrationTest-failure
handled by adding a catch all for any unhandled exception. Because the jira specifically mentions the InvalidReplicationFactor exception, a test was added for that specific case. Author: Grant Henke <granthenke@gmail.com> Reviewers: Jason Gustafson <jason@confluent.io>, Ismael Juma <ismael@juma.me.uk> Closes apache#1739 from granthenke/create-errors
…ntroller Author: Jason Gustafson <jason@confluent.io> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes apache#1734 from hachikuji/KAFKA-3916
Author: Guozhang Wang <wangguoz@gmail.com> Reviewers: Damian Guy <damian.guy@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io> Closes apache#1736 from guozhangwang/Kminor-topology-applicationID
…ectly Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Jun Rao <junrao@gmail.com> Closes apache#1773 from ijuma/kafka-4073-mirror-maker-timestamps
Configure default classes using class objects instead of class names, enable configurable lists of classes to be specified as class objects, add tests for different classloader configurations. Author: Rajini Sivaram <rajinisivaram@googlemail.com> Reviewers: Sriharsha Chintalapani <harsha@hortonworks.com>, Ismael Juma <ismael@juma.me.uk> Closes apache#1421 from rajinisivaram/KAFKA-3680
Author: Konstantine Karantasis <k.karantasis@gmail.com> Reviewers: Ewen Cheslack-Postava <ewen@confluent.io> Closes apache#1771 from kkonstantine/KAFKA-2894-rewind-offsets-on-rebalance
…ith Compressed Messages ijuma - Making the change against trunk based on your suggestions to have the stream closing handled in the private RecordIterator constructor which I understand is only to be used only if the block of message(s) are compressed. Author: William Yu <wyu@unified.com> Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Ismael Juma <ismael@juma.me.uk> Closes apache#1760 from wiyu/compressor_memory_leak_in_fetcher
Also upgrade scoverage (required for compatibility) and remove usage of `useAnt` which doesn't exist in Gradle 3.0 It turns out that one cannot even run `gradle` to download the project Gradle version if `useAnt` is used in the build. This is unfortunate (the SBT launcher has much saner behaviour). Release notes: https://docs.gradle.org/3.0/release-notes Author: Ismael Juma <ismael@juma.me.uk> Reviewers: Grant Henke <granthenke@gmail.com>, Ewen Cheslack-Postava <ewen@confluent.io>, Sriharsha Chintalapani <harsha@hortonworks.com> Closes apache#1774 from ijuma/kafka-4082-support-gradle-3.0
This commit changes all the occurrences of dir.mkdirs() with Files.createDirectory(dir.toPath())
Fixed the failing errors in the previous commit that changed occurrences of dir.mkdirs() to Files.createDirectory()
|
Closing this PR and creating a new one as I made a mistake somewhere in the process of rebasing. |
|
Redirecting to PR #1787 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This commit changes all the occurrences of dir.mkdirs() with Files.createDirectory(dir.toPath())