Skip to content
Branch: master
Find file History
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
Makefile.am
README.md
omhttpfs.c

README.md

OmHTTPFS

Author: sskaje (sskaje@gmail.com, http://sskaje.me/)

OmHTTPFS is an Rsyslog plugin writting data to HDS via Hadoop HDFS over HTTP.

Hadoop HDFS over HTTP

Official site: Hadoop HDFS over HTTP

HTTPFS is not well documented. I tried to read its source and write an intro with examples, until I found Administering the file system by using HttpFS REST APIs by IBM.

OmHDFS for Rsyslog

Rsyslog provides a plugin named omHDFS which requires lots of work compiling and configuring, and it's not that usable. Here is what I tried before writting this omhttpfs: Build omhdfs for Rsyslog.

Rsyslog config

Legacy config NOT supported.

Example:

module(load="omhttpfs")
template(name="hdfs_tmp_file" type="string" string="/tmp/%$YEAR%/test.log")
template(name="hdfs_tmp_filecontent" type="string" string="%$YEAR%-%$MONTH%-%$DAY% %MSG% ==\n")
local4.*    action(type="omhttpfs" host="10.1.1.161" port="14000" https="off" file="hdfs_tmp_file" isDynFile="on")
local5.*    action(type="omhttpfs" host="10.1.1.161" port="14000" https="off" file="hdfs_tmp_file" isDynFile="on" template="hdfs_tmp_filecontent")

Tested with CDH 5.2.0 + Rsyslog 8.6.0 on CentOS 7

# EOF

You can’t perform that action at this time.