output_format , just simple

[jiaqianjing]

Now I have a problem,

format whole data as serialized JSON, single attribute or separated multi attributes
or LTSV, labeled-TSV (see http://ltsv.org/ )

However,I just want raw data ,like input equal output
neither {“message”:"......"} or message : .....

[tagomoris]

This plugin doesn't support standard formatter plugins... It'll be supported when this plugin supports Fluentd v0.14 plugin API (and formatter_single_value included in Fluentd v0.14).

[jiaqianjing]

thx,guy

[jiaqianjing]

Hi, @tagomoris ,I need your help
Now,I have upgraded to the fluentd-v0.14 version... But how to achieve my needs

[tagomoris]

@jiaqianjing it's needed to release a new version which uses v0.14 APIs... code exists here #44, so I'll do it right after I finished some more work about Fluentd v0.14 core.

[jiaqianjing]

@tagomoris thx a lot . I got it. But I mean can I merge it ( local ) and install in my fluentd(v 0.14.11) ?
However ,I have no idea about suorce code #44 installed . Just the following operations:

1. git clone fluent-plugin-webhdfs(V0.52)
2. git fetch origin pull/44/head:migrate-to-v0.14-api
3. mv SOURCE_CODE_DIR /opt/td-agent/embedded/lib/ruby/gems/2.4.0/gems/

hdfs.conf

<source>
...
</source>
<match>
  ...
 output_data_type single_value
  ...
</match>

[jiaqianjing]

@tagomoris now I have installed #44 by RubyGems

gem uninstall fluent-plugin-webhdfs
gem build fluent-plugin-webhdfs.gemspec
gem install fluent-plugin-webhdfs-0.5.1.gem --local

But something has been wrong! It didn't work correctly...
And how can I write a configuration file,which one param, format single_value (info: not use) or output_data_type single_value( seems like wrong +_+)

[tagomoris]

I re-checked #44 and confirmed that it doesn't enable APIs about formatting (and some others expected in v0.14).
So I'll work about it and release new version for Fluentd v0.14 at early next week. Please wait a little.

[jiaqianjing]

3q , happy new year !!!

[tagomoris]

I've just released 0.6.0rc1 version, which fully supports Fluentd v0.14 including <format> section like below:

<match mytag>
  @type webhdfs
  host my.namenode.hostname.local
  path /my/path/on/hdfs
  <format>
    @type single_value
    message_key my_whole_data_in_a_field
  </format>
</match>

It can be installed via command gem install fluent-plugin-webhdfs --prerelease.
@jiaqianjing Can you try this version? I want to release this version as a stable one after someone confirms that it works fine (and update README).

[jiaqianjing]

66666666666666666 @tagomoris I mean good job Dutch.
And it works gracefully . Thanks a lot

[tagomoris]

Thank you to check it! I'll release the new version soon.

[tagomoris]

I've just released v1.0.0 for Fluentd v0.14.