Permalink
Commits on Jun 8, 2018
  1. FLUME-3142: Adding HBase2 sink

    szaboferee committed Jun 8, 2018
    HBase2Sink is the equivalent of HBaseSink for HBase version 2.
    HBaseSink used some API calls which were deprecated in HBase 1.x
    and they are not available in HBase 2.x any more.
    
    HBase2Sink has been implemented by copying the existing
    flume-ng-hbase-sink module to the new flume-ng-hbase2-sink module,
    then adjusting the incompatible API calls to HBase 2.
    The package and class names have also been modified to have
    the hbase2/HBase2 tag. "Hbase" typos have been fixed too.
    
    The functionality provided by HBase2Sink and the configuration parameters
    are the same as in case of HBaseSink (except the hbase2 tag in the sink type
    and the package/class names).
    
    HBaseSink has not been modified, so it works with HBase 1.x as before.
    
    This closes #209
    
    Reviewers: Denes Arvay, Endre Major, Ferenc Szabo
    
    (Peter Turcsanyi via Ferenc Szabo)
Commits on Jun 5, 2018
  1. FLUME-3222 Fix for NoSuchFileException thrown when files are being de…

    szaboferee committed Jun 5, 2018
    …leted
    
    from the TAILDIR source
    
    We fetch file names from a directory and later we fetch inodes.
    If there is a delete between these operations this problem occurs.
    Reproduced from unit test.
    Added exception handling to handle this case.
    It is enough to ignore the NoSuchFileException and continue.
    
    this closes #208
    
    reviewers: Ferenc Szabo, Peter Turcsanyi
    
    (Endre Major via Ferenc Szabo)
Commits on May 30, 2018
  1. FLUME-2976 Exception when JMS source tries to connect to a Weblogic s…

    szaboferee committed May 30, 2018
    …erver without authentication
    
    changing the default "" value of the password to null
    
    Reviewers: Bessenyei Balazs Donat, Peter Turcsanyi, Ferenc Szabo
    
    (Denes Arvay via Ferenc Szabo)
Commits on May 3, 2018
  1. FLUME-2894: Flume components should stop in the correct order (gracef…

    laxman-ch authored and szaboferee committed May 3, 2018
    …ul shutdown)
    
    This closes #101
    
    Reviewers: Bessenyei Balazs Donat, Ferenc Szabo
    
    (Laxman Ch via Ferenc Szabo)
Commits on Mar 9, 2018
  1. FLUME-2786 FLUME-3056 FLUME-3117 Application enters a deadlock when s…

    andrasbeni authored and szaboferee committed Mar 9, 2018
    …topped while handleConfigurationEvent
    
    Adding better locking mechanism to Application class to prevent deadlock.
    
    this closes #108
    this closes #144
    
    Revievers: Denes Arvay, Attila Simon, Benedict Jin, Ferenc Szabo
    
    (Andras Beni, Yan Jian via Ferenc Szabo)
Commits on Feb 23, 2018
  1. FLUME-3218 fixing executable permission in test case in a more portab…

    szaboferee committed Feb 23, 2018
    … way
    
    This closes #199
    
    Reviewers: Denes Arvay
    
    (Ferenc Szabo via Ferenc Szabo)
Commits on Feb 22, 2018
  1. FLUME-2442 Need an alternative to providing clear text passwords in f…

    szaboferee authored and adenes committed Feb 22, 2018
    …lume config
    
    Adding an interface with 3 implementations to provide functionality at the
    configuration level to replace variables/keys from external sources. This
    component is capable of hiding sensitive information or injecting generated
    data into the configuration.
    
    The implementation affects only the configuration layer so existing components
    (sinks/sources/channels/etc) do not have to change and new components can
    already have it through the configuration.
    
    New custom implementations can be easily added even in plugin form.
    
    Each implementation has unit tests in their module and an integration test in
    the flume-ng-tests module.
    
    This closes #197
    
    Reviewers: Miklos Csanady, Denes Arvay
    
    (Ferenc Szabo via Denes Arvay)
Commits on Feb 6, 2018
  1. FLUME-2957. Remove Guava from our public API

    adenes authored and szaboferee committed Feb 6, 2018
    The only Guava class used in public API is the ImmutableMap in the
    
    org.apache.flume.Context,
    org.apache.flume.node.MaterializedConfiguration and
    org.apache.flume.node.SimpleMaterializedConfiguration
    
    classes.
    
    This commit replaces these with java.util.Map.
    
    Note: this is a breaking change as it modifies
    the return type of a method on a public interface.
    
    This closes #195
    
    Reviewers: Ferenc Szabo, Mike Percy
    
    (Denes Arvay via Ferenc Szabo)
Commits on Jan 26, 2018
  1. FLUME-3115. Update netty library

    szaboferee authored and adenes committed Aug 18, 2017
    This change upgrades the netty library to 3.10.6 and also adds netty-all 4.1.17
    
    This closes #155
    
    Reviewers: Miklos Csanady
    
    (Ferenc Szabo via Denes Arvay)
  2. FLUME-3201. Fix SyslogUtil to handle RFC3164 format in December corre…

    szaboferee authored and adenes committed Dec 6, 2017
    …ctly
    
    This closes #188
    
    Reviewers: Tristan Stevens, Miklos Csanady
    
    (Ferenc Szabo via Denes Arvay)
  3. FLUME-3194 upgrade derby to the latest version

    Miklos Csanady authored and adenes committed Dec 5, 2017
    Current derby version contains security vulnerabilities.
    This update will upgrade to a later version.
    
    This closes #184
    
    Reviewers: Ferenc Szabo
    
    (Miklos Csanady via Ferenc Szabo)
  4. FLUME-3158: Upgrade surefire version and config

    Miklos Csanady authored and adenes committed Nov 21, 2017
    Set surefire version to the latest and configure it
    to rerun failed tests.
    This way the flaky tests will not break the build as often.
    
    This closes #182
    
    Reviewers: Ferenc Szabo
    
    (Miklos Csanady via Ferenc Szabo)
  5. FLUME-3195. Split the KafkaChannelTest to avoid timeouts

    Viktor Somogyi authored and adenes committed Nov 16, 2017
    KafkaChannelTest had quite a few test methods so during the build sometimes it
    caused a timeout. Refactoring it to smaller test classes decrease the chance of
    timeouts.
    
    This closes #183
    
    Reviewers: Miklos Csanady, Ferenc Szabo
    
    (Viktor Somogyi via Ferenc Szabo)
  6. FLUME-2050. Upgrade to Log4j 2.10.0

    rgoers authored and adenes committed Nov 3, 2017
    This closes #181
    
    Reviewers: Ferenc Szabo
    
    (Ralph Goers via Denes Arvay)
Commits on Oct 26, 2017
  1. FLUME-199. Remove hardcoded ports from unit tests

    andrasbeni authored and adenes committed Oct 26, 2017
    Many unit tests use hardcoded port numbers which leads to flakiness and causes
    problems when running builds in parallel.
    This patch fixes this issue by searching for available ports instead of the
    hardcoded ones.
    
    This closes #124
    
    Reviewers: Miklos Csanady, Ferenc Szabo
    
    (Andras Beni via Denes Arvay)
Commits on Oct 25, 2017
  1. FLUME-2698. Upgrade Jetty Version

    tmgstevens authored and adenes committed Oct 25, 2017
    Update Jetty version to 9.4.6.
    Beside the version upgrade this patch exposes the new Jetty's configuration
    variables on the HTTPSource to provide users the possibility of fine-tuning
    the HTTPSource.
    
    This closes #158
    
    Reviewers: Ferenc Szabo, Mike Percy, Miklos Csanady, Denes Arvay
    
    (Tristan Stevens via Denes Arvay)
Commits on Oct 19, 2017
  1. FLUME-3186. Make asyncHbaseClient config parameters available from Fl…

    Miklos Csanady authored and adenes committed Oct 19, 2017
    …ume config
    
    This patch adds the ability to set the asyncHbaseClient's config parameters via
    the Flume configuration.
    
    This closes #178
    
    Reviewers: Ferenc Szabo, Denes Arvay
    
    (Miklos Csanady via Denes Arvay)
Commits on Oct 5, 2017
  1. FLUME-3159. Setting JdbcChannel test connection to in-memory derby

    szaboferee authored and adenes committed Oct 1, 2017
    This patch changes the JdbcChannel tests to use in-memory derby to make parallel
    test run possible.
    
    This closes #177
    
    Reviewers: Denes Arvay
    
    (Ferenc Szabo via Denes Arvay)
Commits on Sep 15, 2017
  1. Fix source assembly

    adenes committed Sep 15, 2017
    flume-checkstyle breaks the assembly because its parent is not the flume-parent
    Removing the moduleSets definition from the src assembly solved the issue.
    Files are added based on fileSets, the resulting tarball's content equals
    to the result of the dev-support/generate-source-release.sh in a clean
    working directory.
    
    This closes #175
    
    Reviewers: Miklos Csanady, Ferenc Szabo
    
    (Denes Arvay via Denes Arvay)
  2. Add README.md and doap_Flume.rdf to the generated release artifacts

    bessbd authored and adenes committed Oct 12, 2016
    This closes #69
    
    (Cherry-picked from flume-1.7 branch by Denes Arvay)
Commits on Sep 13, 2017
  1. FLUME-3166. Update LICENSE

    adenes committed Sep 13, 2017
    Adding geronimo-jms_1.1_spec jar to the Apache licensed list.
  2. Changing javax.jms to geronimo jms spec

    szaboferee authored and adenes committed Sep 13, 2017
    javax.jms has GNU GPL licence which is not compatible with Apache
    
    This closes #172
    
    Reviewers: Miklos Csanady, Denes Arvay
    
    (Ferenc Szabo via Denes Arvay)
  3. Set the copyright date dynamically in documentation footer

    adenes committed Sep 12, 2017
    Display "2009-current year" instead of the hardcoded "2009-2012".
    
    This closes #171
    
    Reviewers: Ferenc Szabo
    
    (Denes Arvay via Denes Arvay)
  4. FLUME-3175. Turn off doclint for javadoc generation

    adenes committed Sep 11, 2017
    With Java 8's new doclint feature the javadoc generation breaks due to the
    strict checking so adding -Xdoclint:none to the maven-javadoc-plugin's
    configuration.
    
    This closes #168
    
    Reviewers: Bessenyei Balázs Donát, Ferenc Szabo
    
    (Denes Arvay via Denes Arvay)
Commits on Sep 12, 2017
  1. FLUME-3046. Kafka Sink and Source Configuration Improvements

    tmgstevens authored and adenes committed Sep 12, 2017
    This patch fixes the infinite loop between Kafka source and Kafka sink
    by introducing the following configuration parameters in those components:
    - topicHeader in Kafka source to specify the name of the header where it
      stores the topic name where the event comes from.
    - setTopicHeader in Kafka source to control whether the topic name is stored
      in the given header.
    - topicHeader in Kafka sink to configure the name of the header which
      is used to specify in which topic to send the event.
    - allowTopicOverride in Kafka sink to control whether the target topic's name
      can be overridden by the specified header.
    
    This closes #105
    
    Reviewers: Attila Simon
    
    (Tristan Stevens via Denes Arvay)
  2. Fix NetCat UDP Source table in FlumeUserGuide.rst

    adenes committed Sep 12, 2017
    This closes #170
    
    Reviewers: Ferenc Szabo
    
    (Denes Arvay via Denes Arvay)
  3. FLUME-3173. Upgrade joda-time to 2.9.9

    Miklos Csanady authored and adenes committed Sep 12, 2017
    This closes #169
    
    Reviewers: Marcell Hegedus
    
    (Miklos Csanady via Denes Arvay)
Commits on Sep 11, 2017
  1. FLUME-1520. Timestamp interceptor should support custom headers

    adenes committed Sep 11, 2017
    This change adds a configuration parameter to the TimestampInterceptor
    for the user to be able to define the name of the timestamp header.
    
    Reviewers: Tristan Stevens, Attila Simon
    
    (Hari Shreedharan, Tristan Stevens, Attila Simon via Denes Arvay)
  2. FLUME-3127. Upgrade libfb303 library dependency

    szaboferee authored and adenes committed Sep 1, 2017
    - thrift version changed to 0.9.3
    - thrift objects generated with the new version
    - deprecated method calls updated in ScribeSource
    - cross-compatibility tested with Flume 1.6 and 1.7
    
    This closes #163
    
    Reviewers: Denes Arvay
    
    (Ferenc Szabo via Denes Arvay)
Commits on Sep 4, 2017
  1. FLUME-3156. Update example configuration of Kafka Sink

    Robin021 authored and adenes committed Sep 2, 2017
    Change ki to k1 in Kafka Sink configuration example.
    (a1.sinks.ki.kafka.producer.compression.type)
    
    This closes #165
    
    Reviewers: Bessenyei Balázs Donát, Denes Arvay
    
    (Robin Wang via Denes Arvay)
  2. Fix an infinite recursion in SourceConfiguration

    TeddyBear1314 authored and adenes committed Sep 3, 2017
    Minor change to fix an infinite recursion in
    org.apache.flume.conf.source.SourceConfiguration.getSourceConfigurationType()
    
    This closes #166
    
    Reviewers: Denes Arvay
    
    (TeddyBear1314 via Denes Arvay)
  3. FLUME-3157. Refactor TestHDFSEventSinkOnMiniCluster to not use LeaseM…

    szaboferee authored and adenes committed Sep 1, 2017
    …anager private API
    
    This fix removes LeaseManager private API usages from tests in
    TestHDFSEventSinkOnMiniCluster as these methods are expected to be
    removed/changed in future versions of Hadoop/HDFS.
    
    Test refactored, it has the same result and behaviour.
    
    This closes #164
    
    Reviewers: Denes Arvay
    
    (Ferenc Szabo via Denes Arvay)