Skip to content

Commit

Permalink
Various Pod corrections
Browse files Browse the repository at this point in the history
  • Loading branch information
hinrik committed Feb 12, 2010
1 parent 6d8ecb4 commit 7dc1c02
Show file tree
Hide file tree
Showing 15 changed files with 104 additions and 112 deletions.
3 changes: 3 additions & 0 deletions Changes
@@ -1,3 +1,6 @@
0.11
- Corrected outdated documentation in some places

0.10 Fri Feb 12 02:31:34 GMT 2010
- Normalized the SQL schema some more. This breaks compatability with old
brains of course, but training/learning is quite a bit faster now.
Expand Down
70 changes: 36 additions & 34 deletions README
Expand Up @@ -2,28 +2,34 @@ NAME
Hailo - A pluggable Markov engine analogous to MegaHAL

SYNOPSIS
use Hailo;
use strict;
use warnings;
use Hailo;

my $hailo = Hailo->new(
# Or Pg, or Perl ...
storage_class => 'SQLite',
brain_resource => 'brain.db'
);
my $hailo = Hailo->new(
# Or Pg, or Perl ...
storage_class => 'SQLite',
brain_resource => 'brain.db'
);

while (<>) {
$hailo->learn($_);
print $hailo->reply($_), "\n";
}
while (<>) {
$hailo->learn($_);
print $hailo->reply($_), "\n";
}

DESCRIPTION
Hailo is a fast and lightweight markov engine intended to replace
AI::MegaHAL. It has a lightweight Moose-based core with pluggable
storage, tokenizer and engine backends.
storage and tokenizer backends.

It's faster than MegaHAL and can handle huge brains easily with the
default SQLite backend. It can be used, amongst other things, to
implement IRC chat bots with POE::Component::IRC. In fact, there exists
a POE::Component::IRC plugin for just that purpose.
It is similar to MegaHAL in functionality, the main difference being
(with the default storage/tokenizer backends) better scalability,
drastically less memory usage, and an improved tokenizer.

With this distribution, you can create, modify, and query Hailo brains.
To use Hailo in asynchronous applications, you can use the
POE::Component::Hailo wrapper. One example is
POE::Component::IRC::Plugin::Hailo, which implements and IRC chat bot.

Etymology
*Hailo* is a portmanteau of *HAL* (as in MegaHAL) and failo
Expand All @@ -44,31 +50,27 @@ ATTRIBUTES
This gives you an idea of approximately how the backends compare in
speed:

s/iter CHI::File CHI::BerkeleyDB PostgreSQL MySQL CHI::Memory SQLite Perl Perl::Flat
CHI::File 15.1 -- -51% -72% -75% -82% -91% -95% -95%
CHI::BerkeleyDB 7.43 103% -- -42% -49% -64% -81% -90% -91%
PostgreSQL 4.30 252% 73% -- -11% -37% -67% -83% -84%
MySQL 3.83 295% 94% 12% -- -29% -63% -81% -82%
CHI::Memory 2.70 460% 176% 59% 42% -- -48% -73% -75%
SQLite 1.41 972% 427% 205% 171% 91% -- -48% -52%
Perl 0.735 1957% 911% 485% 420% 267% 92% -- -7%
Perl::Flat 0.683 2114% 988% 529% 460% 295% 106% 8% --
s/iter CHI::File CHI::BerkeleyDB PostgreSQL MySQL CHI::Memory SQLite Perl Perl::Flat
CHI::File 15.1 -- -51% -72% -75% -82% -91% -95% -95%
CHI::BerkeleyDB 7.43 103% -- -42% -49% -64% -81% -90% -91%
PostgreSQL 4.30 252% 73% -- -11% -37% -67% -83% -84%
MySQL 3.83 295% 94% 12% -- -29% -63% -81% -82%
CHI::Memory 2.70 460% 176% 59% 42% -- -48% -73% -75%
SQLite 1.41 972% 427% 205% 171% 91% -- -48% -52%
Perl 0.735 1957% 911% 485% 420% 267% 92% -- -7%
Perl::Flat 0.683 2114% 988% 529% 460% 295% 106% 8% --

To run your own test try running utils/hailo-benchmark in the Hailo
distribution.

"tokenizer_class"
The tokenizer to use. Default: 'Words';

"engine_class"
The engine to use. Default: 'Default';

"engine_class"
"ui_class"
The UI to use. Default: 'ReadLine';

"storage_args"
"tokenizer_args"
"engine_args"
"ui_args"
A "HashRef" of arguments storage/tokenizer/engine/ui backends. See the
documentation for the backends for what sort of arguments they accept.
Expand All @@ -80,7 +82,7 @@ ATTRIBUTES

METHODS
"new"
This is the constructor. It accept the attributes specified in
This is the constructor. It accepts the attributes specified in
"ATTRIBUTES".

"run"
Expand All @@ -90,16 +92,16 @@ METHODS
Takes a line of UTF-8 encoded text as input and learns from it.

"train"
Takes a filename and calls "learn" on all its lines. Lines are expected
Takes a filename and calls "learn" on all its lines. The file is assumed
to be UTF-8 encoded.

"reply"
Takes a line of text and generates a reply (UTF-8 encoded) that might be
Takes an optional line of text and generates a reply that might be
relevant.

"learn_reply"
Takes a line of text, learns from it, and generates a reply (UTF-8
encoded) that might be relevant.
Takes a line of text, learns from it, and generates a reply that might
be relevant.

"save"
Tells the underlying storage backend to save its state.
Expand Down
82 changes: 40 additions & 42 deletions lib/Hailo.pm
Expand Up @@ -467,32 +467,37 @@ Hailo - A pluggable Markov engine analogous to MegaHAL
=head1 SYNOPSIS
use Hailo;
use strict;
use warnings;
use Hailo;
my $hailo = Hailo->new(
# Or Pg, or Perl ...
storage_class => 'SQLite',
brain_resource => 'brain.db'
);
my $hailo = Hailo->new(
# Or Pg, or Perl ...
storage_class => 'SQLite',
brain_resource => 'brain.db'
);
while (<>) {
$hailo->learn($_);
print $hailo->reply($_), "\n";
}
while (<>) {
$hailo->learn($_);
print $hailo->reply($_), "\n";
}
=head1 DESCRIPTION
Hailo is a fast and lightweight markov engine intended to replace
L<AI::MegaHAL>. It has a lightweight L<Moose>-based core with
pluggable L<storage|Hailo::Role::Storage>,
L<tokenizer|Hailo::Role::Tokenizer> and L<engine|Hailo::Role::Engine>
backends.
L<AI::MegaHAL|AI::MegaHAL>. It has a lightweight L<Moose|Moose>-based core
with pluggable L<storage|Hailo::Role::Storage> and
L<tokenizer|Hailo::Role::Tokenizer> backends.
It's faster than MegaHAL and can handle huge brains easily with the
default L<SQLite backend|Hailo::Storage::DBD::SQLite>. It can be used,
amongst other things, to implement IRC chat bots with
L<POE::Component::IRC>. In fact, there exists a L<POE::Component::IRC>
L<plugin|POE::Component::IRC::Plugin::Hailo> for just that purpose.
It is similar to MegaHAL in functionality, the main difference being (with
the default storage/tokenizer backends) better scalability, drastically less
memory usage, and an improved tokenizer.
With this distribution, you can create, modify, and query Hailo brains. To
use Hailo in asynchronous applications, you can use the
L<POE::Component::Hailo|POE::Component::Hailo> wrapper. One example is
L<POE::Component::IRC::Plugin::Hailo|POE::Component::IRC::Plugin::Hailo>,
which implements and IRC chat bot.
=head2 Etymology
Expand All @@ -518,15 +523,15 @@ The storage backend to use. Default: 'SQLite'.
This gives you an idea of approximately how the backends compare in
speed:
s/iter CHI::File CHI::BerkeleyDB PostgreSQL MySQL CHI::Memory SQLite Perl Perl::Flat
CHI::File 15.1 -- -51% -72% -75% -82% -91% -95% -95%
CHI::BerkeleyDB 7.43 103% -- -42% -49% -64% -81% -90% -91%
PostgreSQL 4.30 252% 73% -- -11% -37% -67% -83% -84%
MySQL 3.83 295% 94% 12% -- -29% -63% -81% -82%
CHI::Memory 2.70 460% 176% 59% 42% -- -48% -73% -75%
SQLite 1.41 972% 427% 205% 171% 91% -- -48% -52%
Perl 0.735 1957% 911% 485% 420% 267% 92% -- -7%
Perl::Flat 0.683 2114% 988% 529% 460% 295% 106% 8% --
s/iter CHI::File CHI::BerkeleyDB PostgreSQL MySQL CHI::Memory SQLite Perl Perl::Flat
CHI::File 15.1 -- -51% -72% -75% -82% -91% -95% -95%
CHI::BerkeleyDB 7.43 103% -- -42% -49% -64% -81% -90% -91%
PostgreSQL 4.30 252% 73% -- -11% -37% -67% -83% -84%
MySQL 3.83 295% 94% 12% -- -29% -63% -81% -82%
CHI::Memory 2.70 460% 176% 59% 42% -- -48% -73% -75%
SQLite 1.41 972% 427% 205% 171% 91% -- -48% -52%
Perl 0.735 1957% 911% 485% 420% 267% 92% -- -7%
Perl::Flat 0.683 2114% 988% 529% 460% 295% 106% 8% --
To run your own test try running F<utils/hailo-benchmark> in the Hailo
distribution.
Expand All @@ -535,20 +540,14 @@ distribution.
The tokenizer to use. Default: 'Words';
=head2 C<engine_class>
The engine to use. Default: 'Default';
=head2 C<engine_class>
=head2 C<ui_class>
The UI to use. Default: 'ReadLine';
=head2 C<storage_args>
=head2 C<tokenizer_args>
=head2 C<engine_args>
=head2 C<ui_args>
A C<HashRef> of arguments storage/tokenizer/engine/ui backends. See
Expand All @@ -565,7 +564,7 @@ this is C<"\t">.
=head2 C<new>
This is the constructor. It accept the attributes specified in
This is the constructor. It accepts the attributes specified in
L</ATTRIBUTES>.
=head2 C<run>
Expand All @@ -578,18 +577,17 @@ Takes a line of UTF-8 encoded text as input and learns from it.
=head2 C<train>
Takes a filename and calls L<C<learn>|/learn> on all its lines. Lines are
expected to be UTF-8 encoded.
Takes a filename and calls L<C<learn>|/learn> on all its lines. The file is
assumed to be UTF-8 encoded.
=head2 C<reply>
Takes a line of text and generates a reply (UTF-8 encoded) that might be
relevant.
Takes an optional line of text and generates a reply that might be relevant.
=head2 C<learn_reply>
Takes a line of text, learns from it, and generates a reply (UTF-8 encoded)
that might be relevant.
Takes a line of text, learns from it, and generates a reply that might be
relevant.
=head2 C<save>
Expand Down
8 changes: 1 addition & 7 deletions lib/Hailo/Role/Storage.pm
Expand Up @@ -76,14 +76,8 @@ Learns from a sequence of tokens. Takes an array ref of strings.
=head2 C<make_reply>
Takes a single token as an argument and returns a randomly picked expression
which contains it.
=head2 C<make_reply>
Takes an array reference of tokens and returns a reply (arrayref of tokens)
that might be relevant. If none of the supplied tokens are known, nothing is
returned.
that might be relevant.
=head2 C<start_learning>
Expand Down
4 changes: 2 additions & 2 deletions lib/Hailo/Storage/CHI/BerkeleyDB.pm
Expand Up @@ -22,7 +22,8 @@ __PACKAGE__->meta->make_immutable;
=head1 NAME
Hailo::Storage::CHI::BerkeleyDB - A L<BerkeleyDB|CHI::Driver::BerkeleyDB> storage backend for L<Hailo|Hailo> using L<CHI>
Hailo::Storage::CHI::BerkeleyDB - A L<BerkeleyDB|CHI::Driver::BerkeleyDB>
storage backend for L<Hailo|Hailo> using L<CHI>
=head1 AUTHOR
Expand All @@ -36,4 +37,3 @@ This program is free software, you can redistribute it and/or modify
it under the same terms as Perl itself.
=cut

4 changes: 2 additions & 2 deletions lib/Hailo/Storage/CHI/File.pm
Expand Up @@ -24,7 +24,8 @@ __PACKAGE__->meta->make_immutable;
=head1 NAME
Hailo::Storage::CHI::File - A L<file|CHI::Driver::File> storage backend for L<Hailo|Hailo> using L<CHI>
Hailo::Storage::CHI::File - A L<file|CHI::Driver::File> storage backend
for L<Hailo|Hailo> using L<CHI>
=head1 AUTHOR
Expand All @@ -38,4 +39,3 @@ This program is free software, you can redistribute it and/or modify
it under the same terms as Perl itself.
=cut

4 changes: 2 additions & 2 deletions lib/Hailo/Storage/CHI/Memory.pm
Expand Up @@ -22,7 +22,8 @@ __PACKAGE__->meta->make_immutable;
=head1 NAME
Hailo::Storage::CHI::Memory - A L<memory|CHI::Driver::Memory> storage backend for L<Hailo|Hailo> using L<CHI>
Hailo::Storage::CHI::Memory - A L<memory|CHI::Driver::Memory> storage
backend for L<Hailo|Hailo> using L<CHI>
=head1 AUTHOR
Expand All @@ -36,4 +37,3 @@ This program is free software, you can redistribute it and/or modify
it under the same terms as Perl itself.
=cut

18 changes: 10 additions & 8 deletions lib/Hailo/Storage/DBD/SQLite.pm
Expand Up @@ -52,22 +52,24 @@ L<DBD::SQLite|DBD::SQLite>
=head1 DESCRIPTION
This backend maintains information in an SQLite database. It can handle
pretty large datasets.
This backend maintains information in an SQLite database. It is the default
storage backend.
For some example numbers, I have a 5th-order database built from a 204k line
(7.2MB) IRC channel log file (7.2MB). On my laptop (Core 2 Duo 2.53 GHz) it
took 10 minutes and 42 seconds (317 lines/sec) to create the 290MB database.
Furthermore, it can generate 166 replies per second from it. Since this is
just an SQL database, there is very little RAM usage.
For some example numbers, I have a 5th-order database built from a ~210k line
(7.4MB) IRC channel log file. On my laptop (Core 2 Duo 2.53 GHz) it took 8
minutes and 50 seconds (~400 lines/sec) to create the 229MB database.
Furthermore, it can generate about 90 replies per second from it.
=head1 AUTHOR
E<AElig>var ArnfjE<ouml>rE<eth> Bjarmason <avar@cpan.org>
Hinrik E<Ouml>rn SigurE<eth>sson, hinrik.sig@gmail.com
=head1 LICENSE AND COPYRIGHT
Copyright 2010 E<AElig>var ArnfjE<ouml>rE<eth> Bjarmason.
Copyright 2010 E<AElig>var ArnfjE<ouml>rE<eth> Bjarmason and
Hinrik E<Ouml>rn SigurE<eth>sson
This program is free software, you can redistribute it and/or modify
it under the same terms as Perl itself.
Expand Down
4 changes: 2 additions & 2 deletions lib/Hailo/Storage/Mixin/CHI.pm
Expand Up @@ -113,7 +113,8 @@ __PACKAGE__->meta->make_immutable;
=head1 NAME
Hailo::Storage::Mixin::CHI - A mixin class for L<Hailo> L<storage|Hailo::Role::Storage> backends using L<CHI>
Hailo::Storage::Mixin::CHI - A mixin class for L<Hailo>
L<storage|Hailo::Role::Storage> backends using L<CHI>
=head1 AUTHOR
Expand All @@ -127,4 +128,3 @@ This program is free software, you can redistribute it and/or modify
it under the same terms as Perl itself.
=cut

3 changes: 2 additions & 1 deletion lib/Hailo/Storage/Mixin/Hash.pm
Expand Up @@ -195,7 +195,8 @@ __PACKAGE__->meta->make_immutable;
=head1 NAME
Hailo::Storage::Mixin::Hash - A mixin class for L<storage|Hailo::Role::Storage> classes using a C<HashRef>
Hailo::Storage::Mixin::Hash - A mixin class for
L<storage|Hailo::Role::Storage> classes using a C<HashRef>
=head1 DESCRIPTION
Expand Down
4 changes: 2 additions & 2 deletions lib/Hailo/Storage/Mixin/Hash/Flat.pm
Expand Up @@ -158,7 +158,8 @@ __PACKAGE__->meta->make_immutable;
=head1 NAME
Hailo::Storage::Mixin::Hash::Flat - A mixin class for L<storage|Hailo::Role::Storage> classes using a flat C<HashRef>
Hailo::Storage::Mixin::Hash::Flat - A mixin class for
L<storage|Hailo::Role::Storage> classes using a flat C<HashRef>
=head1 DESCRIPTION
Expand All @@ -178,4 +179,3 @@ This program is free software, you can redistribute it and/or modify
it under the same terms as Perl itself.
=cut

0 comments on commit 7dc1c02

Please sign in to comment.