README for Perl module String::Tokenizer

This module provides a simple string tokenizer which takes a string
and splits it on whitespace. It also optionally takes a string of
characters to use as delimiters, and returns them with the token set
as well. This allows for splitting the string in many different ways.

You can read a nicely formatted version of the documentation for
this module online:

    https://metacpan.org/pod/String::Tokenizer

You should be able to install this using your usual method for installing
modules from CPAN. If you don't have one, have a look at:

    http://www.cpan.org/modules/INSTALL.html

This module was written by Stevan Little <stevan@cpan.org>

Copyright (c) 2004 Infinity Interactive, Inc.