Skip to content


Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

379 lines (286 sloc) 11.212 kb
package POE::Filter::Line;
use strict;
use POE::Filter;
use vars qw($VERSION @ISA);
$VERSION = '1.289'; # NOTE - Should be #.### (three decimal places)
@ISA = qw(POE::Filter);
use Carp qw(carp croak);
sub DEBUG () { 0 }
sub FRAMING_BUFFER () { 0 }
sub INPUT_REGEXP () { 1 }
sub OUTPUT_LITERAL () { 2 }
sub AUTO_STATE_DONE () { 0x00 }
sub AUTO_STATE_FIRST () { 0x01 }
sub AUTO_STATE_SECOND () { 0x02 }
sub new {
my $type = shift;
croak "$type requires an even number of parameters" if @_ and @_ & 1;
my %params = @_;
croak "$type cannot have both Regexp and Literal line endings" if (
defined $params{Regexp} and defined $params{Literal}
my ($input_regexp, $output_literal);
my $autodetect = AUTO_STATE_DONE;
# Literal newline for both incoming and outgoing. Every other known
# parameter conflicts with this one.
if (defined $params{Literal}) {
croak "A defined Literal must have a nonzero length"
unless defined($params{Literal}) and length($params{Literal});
$input_regexp = quotemeta $params{Literal};
$output_literal = $params{Literal};
if (
exists $params{InputLiteral} or # undef means something
defined $params{InputRegexp} or
defined $params{OutputLiteral}
) {
croak "$type cannot have Literal with any other parameter";
# Input and output are specified separately, then.
else {
# Input can be either a literal or a regexp. The regexp may be
# compiled or not; we don't rightly care at this point.
if (exists $params{InputLiteral}) {
$input_regexp = $params{InputLiteral};
# InputLiteral is defined. Turn it into a regexp and be done.
# Otherwise we will autodetect it.
if (defined($input_regexp) and length($input_regexp)) {
$input_regexp = quotemeta $input_regexp;
else {
$autodetect = AUTO_STATE_FIRST;
$input_regexp = '';
croak "$type cannot have both InputLiteral and InputRegexp"
if defined $params{InputRegexp};
elsif (defined $params{InputRegexp}) {
$input_regexp = $params{InputRegexp};
croak "$type cannot have both InputLiteral and InputRegexp"
if defined $params{InputLiteral};
else {
$input_regexp = "(\\x0D\\x0A?|\\x0A\\x0D?)";
if (defined $params{OutputLiteral}) {
$output_literal = $params{OutputLiteral};
else {
$output_literal = "\x0D\x0A";
delete @params{qw(Literal InputLiteral OutputLiteral InputRegexp)};
carp("$type ignores unknown parameters: ", join(', ', sort keys %params))
if scalar keys %params;
my $self = bless [
$input_regexp, # INPUT_REGEXP
$output_literal, # OUTPUT_LITERAL
$autodetect, # AUTODETECT_STATE
], $type;
DEBUG and warn join ':', @$self;
# get() is inherited from POE::Filter.
# 2001-07-27 RCC: Add get_one_start() and get_one() to correct filter
# changing and make input flow control possible.
sub get_one_start {
my ($self, $stream) = @_;
DEBUG and do {
my $temp = join '', @$stream;
$temp = unpack 'H*', $temp;
warn "got some raw data: $temp\n";
$self->[FRAMING_BUFFER] .= join '', @$stream;
# TODO There is a lot of code duplicated here. What can be done?
sub get_one {
my $self = shift;
# Process as many newlines an we can find.
LINE: while (1) {
# Autodetect is done, or it never started. Parse some buffer!
unless ($self->[AUTODETECT_STATE]) {
DEBUG and warn unpack 'H*', $self->[INPUT_REGEXP];
last LINE
unless $self->[FRAMING_BUFFER] =~ s/^(.*?)$self->[INPUT_REGEXP]//s;
DEBUG and warn "got line: <<", unpack('H*', $1), ">>\n";
return [ $1 ];
# Waiting for the first line ending. Look for a generic newline.
last LINE
unless $self->[FRAMING_BUFFER] =~ s/^(.*?)(\x0D\x0A?|\x0A\x0D?)//;
my $line = $1;
# The newline can be complete under two conditions. First: If
# it's two characters. Second: If there's more data in the
# framing buffer. Loop around in case there are more lines.
if ( (length($2) == 2) or
(length $self->[FRAMING_BUFFER])
) {
DEBUG and warn "detected complete newline after line: <<$1>>\n";
$self->[INPUT_REGEXP] = $2;
# The regexp has matched a potential partial newline. Save it,
# and move to the next state. There is no more data in the
# framing buffer, so we're done.
else {
DEBUG and warn "detected suspicious newline after line: <<$1>>\n";
$self->[INPUT_REGEXP] = $2;
return [ $line ];
# Waiting for the second line beginning. Bail out if we don't
# have anything in the framing buffer.
return [ ] unless length $self->[FRAMING_BUFFER];
# Test the first character to see if it completes the previous
# potentially partial newline.
if (
substr($self->[FRAMING_BUFFER], 0, 1) eq
( $self->[INPUT_REGEXP] eq "\x0D" ? "\x0A" : "\x0D" )
) {
# Combine the first character with the previous newline, and
# discard the newline from the buffer. This is two statements
# for backward compatibility.
DEBUG and warn "completed newline after line: <<$1>>\n";
$self->[INPUT_REGEXP] .= substr($self->[FRAMING_BUFFER], 0, 1);
substr($self->[FRAMING_BUFFER], 0, 1) = '';
elsif (DEBUG) {
warn "decided prior suspicious newline is okay\n";
# Regardless, whatever is in INPUT_REGEXP is now a complete
# newline. End autodetection, post-process the found newline,
# and loop to see if there are other lines in the buffer.
$self->[INPUT_REGEXP] = $self->[INPUT_REGEXP];
next LINE;
die "consistency error: AUTODETECT_STATE = $self->[AUTODETECT_STATE]";
return [ ];
# New behavior. First translate system newlines ("\n") into whichever
# newlines are supposed to be sent. Second, add a trailing newline if
# one doesn't already exist. Since the referenced output list is
# supposed to contain one line per element, we also do a split and
# join. Bleah. ... why isn't the code doing what the comment says?
sub put {
my ($self, $lines) = @_;
my @raw;
foreach (@$lines) {
push @raw, $_ . $self->[OUTPUT_LITERAL];
sub get_pending {
my $self = shift;
return [ $self->[FRAMING_BUFFER] ] if length $self->[FRAMING_BUFFER];
return undef;
=head1 NAME
POE::Filter::Line - serialize and parse terminated records (lines)
use POE qw(Wheel::FollowTail Filter::Line);
inline_states => {
_start => sub {
$_[HEAP]{tailor} = POE::Wheel::FollowTail->new(
Filename => "/var/log/system.log",
InputEvent => "got_log_line",
Filter => POE::Filter::Line->new(),
got_log_line => sub {
print "Log: $_[ARG0]\n";
POE::Filter::Line parses stream data into terminated records. The
default parser interprets newlines as the record terminator, and the
default serializer appends network newlines (CR/LF, or "\x0D\x0A") to
outbound records.
Record terminators are removed from the data POE::Filter::Line
POE::Filter::Line supports a number of other ways to parse lines.
Constructor parameters may specify literal newlines, regular
expressions, or that the filter should detect newlines on its own.
POE::Filter::Line's new() method has some interesting parameters.
=head2 new
new() accepts a list of named parameters.
In all cases, the data interpreted as the record terminator is
stripped from the data POE::Filter::Line returns.
C<InputLiteral> may be used to parse records that are terminated by
some literal string. For example, POE::Filter::Line may be used to
parse and emit C-style lines, which are terminated with an ASCII NUL:
my $c_line_filter = POE::Filter::Line->new(
InputLiteral => chr(0),
OutputLiteral => chr(0),
C<OutputLiteral> allows a filter to put() records with a different
record terminator than it parses. This can be useful in applications
that must translate record terminators.
C<Literal> is a shorthand for the common case where the input and
output literals are identical. The previous example may be written
my $c_line_filter = POE::Filter::Line->new(
Literal => chr(0),
An application can also allow POE::Filter::Line to figure out which
newline to use. This is done by specifying C<InputLiteral> to be
my $whichever_line_filter = POE::Filter::Line->new(
InputLiteral => undef,
OutputLiteral => "\n",
C<InputRegexp> may be used in place of C<InputLiteral> to recognize
line terminators based on a regular expression. In this example,
input is terminated by two or more consecutive newlines. On output,
the paragraph separator is "---" on a line by itself.
my $paragraph_filter = POE::Filter::Line->new(
InputRegexp => "([\x0D\x0A]{2,})",
OutputLiteral => "\n---\n",
POE::Filter::Line has no additional public methods.
=head1 SEE ALSO
Please see L<POE::Filter> for documentation regarding the base
The SEE ALSO section in L<POE> contains a table of contents covering
the entire POE distribution.
=head1 BUGS
The default input newline parser is a regexp that has an unfortunate
race condition. First the regular expression:
While it quickly recognizes most forms of newline, it can sometimes
detect an extra blank line. This happens when a two-byte newline
character is broken between two reads. Consider this situation:
some stream dataCR
LFother stream data
The regular expression will see the first CR without its corresponding
LF. The filter will properly return "some stream data" as a line.
When the next packet arrives, the leading "LF" will be treated as the
terminator for a 0-byte line. The filter will faithfully return this
empty line.
B<It is advised to specify literal newlines or use the autodetect
feature in applications where blank lines are significant.>
Please see L<POE> for more information about authors and contributors.
# rocco // vim: ts=2 sw=2 expandtab
# TODO - Edit.
Jump to Line
Something went wrong with that request. Please try again.