A Hadoop input format for sending lists of files as keys to a mapper. Set the list of files, and an input split will be created per file. Each map task gets only one input key: the filename for its split.
Scribe is a server for aggregating log data streamed in real time from a large number of servers. It is designed to be scalable, extensible without client-side modification, and robust to failure of the network or any specific machine.