Skip to content

Commit

Permalink
Link updates.
Browse files Browse the repository at this point in the history
  • Loading branch information
@scop
scop committed Apr 30, 2011
1 parent 25f86ba commit 7683c5f
Show file tree
Hide file tree
Showing 3 changed files with 36 additions and 34 deletions.
4 changes: 2 additions & 2 deletions lib/LWP.pm
View file
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ The libwww-perl library is based on HTTP style communication. This
section tries to describe what that means.
Let us start with this quote from the HTTP specification document
<URL:http://www.w3.org/pub/WWW/Protocols/>:
<URL:http://www.w3.org/Protocols/>:
=over 3
Expand Down Expand Up @@ -274,7 +274,7 @@ headers from the E<lt>head> section of HTML documents.
=item *
The B<proxy> and B<no_proxy> attributes specify if and when to go through
a proxy server. <URL:http://www.w3.org/pub/WWW/Proxies/>
a proxy server. <URL:http://www.w3.org/History/1994/WWW/Proxies/>
=item *
Expand Down
30 changes: 16 additions & 14 deletions lwpcook.pod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,11 @@ net. The LWP::Simple module provides the get() function that return
the document specified by its URL argument:

use LWP::Simple;
$doc = get 'http://www.linpro.no/lwp/';
$doc = get 'http://search.cpan.org/dist/libwww-perl/';

or, as a perl one-liner using the getprint() function:

perl -MLWP::Simple -e 'getprint "http://www.linpro.no/lwp/"'
perl -MLWP::Simple -e 'getprint "http://search.cpan.org/dist/libwww-perl/"'

or, how about fetching the latest perl by running this command:

Expand All @@ -35,7 +35,7 @@ or, how about fetching the latest perl by running this command:
You will probably first want to find a CPAN site closer to you by
running something like the following command:

perl -MLWP::Simple -e 'getprint "http://www.perl.com/perl/CPAN/CPAN.html"'
perl -MLWP::Simple -e 'getprint "http://www.cpan.org/SITES.html"'

Enough of this simple stuff! The LWP object oriented interface gives
you more control over the request sent to the server. Using this
Expand All @@ -47,7 +47,8 @@ handle the response returned.
$ua->agent("$0/0.1 " . $ua->agent);
# $ua->agent("Mozilla/8.0") # pretend we are very capable browser

$req = HTTP::Request->new(GET => 'http://www.linpro.no/lwp');
$req = HTTP::Request->new(
GET => 'http://search.cpan.org/dist/libwww-perl/');
$req->header('Accept' => 'text/html');

# send request
Expand Down Expand Up @@ -95,9 +96,10 @@ operation is to access a WWW form application:
use LWP::UserAgent;
$ua = LWP::UserAgent->new;

my $req = HTTP::Request->new(POST => 'http://www.perl.com/cgi-bin/BugGlimpse');
my $req = HTTP::Request->new(
POST => 'http://rt.cpan.org/Public/Dist/Display.html');
$req->content_type('application/x-www-form-urlencoded');
$req->content('match=www&errors=0');
$req->content('Status=Active&Name=libwww-perl');

my $res = $ua->request($req);
print $res->as_string;
Expand All @@ -110,8 +112,8 @@ suitable default for the content_type:
use LWP::UserAgent;
$ua = LWP::UserAgent->new;

my $req = POST 'http://www.perl.com/cgi-bin/BugGlimpse',
[ search => 'www', errors => 0 ];
my $req = POST 'http://rt.cpan.org/Public/Dist/Display.html',
[ Status => 'Active', Name => 'libwww-perl' ];

print $ua->request($req)->as_string;

Expand Down Expand Up @@ -193,7 +195,7 @@ and respond to cookie requests if you set up a cookie jar.
autosave => 1));

# and then send requests just as you used to do
$res = $ua->request(HTTP::Request->new(GET => "http://www.yahoo.no"));
$res = $ua->request(HTTP::Request->new(GET => "http://no.yahoo.com/"));
print $res->status_line, "\n";

As you visit sites that send you cookies to keep, then the file
Expand Down Expand Up @@ -230,10 +232,10 @@ code similar to this at regular intervals:
use LWP::Simple;

%mirrors = (
'http://www.sn.no/' => 'sn.html',
'http://www.perl.com/' => 'perl.html',
'http://www.sn.no/libwww-perl/' => 'lwp.html',
'gopher://gopher.sn.no/' => 'gopher.html',
'http://www.sn.no/' => 'sn.html',
'http://www.perl.com/' => 'perl.html',
'http://search.cpan.org/distlibwww-perl/' => 'lwp.html',
'gopher://gopher.sn.no/' => 'gopher.html',
);

while (($url, $localfile) = each(%mirrors)) {
Expand All @@ -259,7 +261,7 @@ name):
$ua = LWP::UserAgent->new;

my $req = HTTP::Request->new(GET =>
'http://www.linpro.no/lwp/libwww-perl-5.46.tar.gz');
'http://www.cpan.org/authors/Gisle_Aas/libwww-perl-6.02.tar.gz');
$res = $ua->request($req, "libwww-perl.tar.gz");
if ($res->is_success) {
print "ok\n";
Expand Down
36 changes: 18 additions & 18 deletions lwptut.pod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ In a Perl program, you can call its C<get($url)> function. It will try
getting that URL's content. If it works, then it'll return the
content; but if there's some error, it'll return undef.

my $url = 'http://freshair.npr.org/dayFA.cfm?todayDate=current';
my $url = 'http://www.npr.org/programs/fa/?todayDate=current';
# Just an example: the URL for the most recent /Fresh Air/ show

use LWP::Simple;
Expand All @@ -47,14 +47,14 @@ The handiest variant on C<get> is C<getprint>, which is useful in Perl
one-liners. If it can get the page whose URL you provide, it sends it
to STDOUT; otherwise it complains to STDERR.

% perl -MLWP::Simple -e "getprint 'http://cpan.org/RECENT'"
% perl -MLWP::Simple -e "getprint 'http://www.cpan.org/RECENT'"

That is the URL of a plain text file that lists new files in CPAN in
the past two weeks. You can easily make it part of a tidy little
shell command, like this one that mails you the list of new
C<Acme::> modules:

% perl -MLWP::Simple -e "getprint 'http://cpan.org/RECENT'" \
% perl -MLWP::Simple -e "getprint 'http://www.cpan.org/RECENT'" \
| grep "/by-module/Acme" | mail -s "New Acme modules! Joy!" $USER

There are other useful functions in LWP::Simple, including one function
Expand Down Expand Up @@ -99,7 +99,7 @@ illustrated:
...

# Then later, whenever you need to make a get request:
my $url = 'http://freshair.npr.org/dayFA.cfm?todayDate=current';
my $url = 'http://www.npr.org/programs/fa/?todayDate=current';

my $response = $browser->get( $url );
die "Can't get $url -- ", $response->status_line
Expand Down Expand Up @@ -297,22 +297,22 @@ the HTML the report of the number of matches:
use warnings;
use LWP 5.64;
my $browser = LWP::UserAgent->new;

my $word = 'tarragon';
my $url = 'http://www.altavista.com/sites/search/web';

my $url = 'http://search.yahoo.com/yhs/search';
my $response = $browser->post( $url,
[ 'q' => $word, # the Altavista query string
'pg' => 'q', 'avkw' => 'tgz', 'kl' => 'XX',
'fr' => 'altavista', 'pg' => 'q', 'avkw' => 'tgz', 'kl' => 'XX',
]
);
die "$url error: ", $response->status_line
unless $response->is_success;
die "Weird content type at $url -- ", $response->content_type
unless $response->content_is_html;

if( $response->decoded_content =~ m{AltaVista found ([0-9,]+) results} ) {
# The substring will be like "AltaVista found 2,345 results"
if( $response->decoded_content =~ m{([0-9,]+)(?:<.*?>)? results for} ) {
# The substring will be like "996,000</strong> results for"
print "$word: $1\n";
}
else {
Expand All @@ -331,23 +331,23 @@ the HTML the report of the number of matches:
Some HTML forms convey their form data not by sending the data
in an HTTP POST request, but by making a normal GET request with
the data stuck on the end of the URL. For example, if you went to
C<imdb.com> and ran a search on "Blade Runner", the URL you'd see
C<www.imdb.com> and ran a search on "Blade Runner", the URL you'd see
in your browser window would be:

http://us.imdb.com/Tsearch?title=Blade%20Runner&restrict=Movies+and+TV
http://www.imdb.com/find?s=all&q=Blade+Runner

To run the same search with LWP, you'd use this idiom, which involves
the URI class:

use URI;
my $url = URI->new( 'http://us.imdb.com/Tsearch' );
my $url = URI->new( 'http://www.imdb.com/find' );
# makes an object representing the URL

$url->query_form( # And here the form data pairs:
'title' => 'Blade Runner',
'restrict' => 'Movies and TV',
'q' => 'Blade Runner',
's' => 'all',
);

my $response = $browser->get($url);

See chapter 5 of I<Perl & LWP> for a longer discussion of HTML forms
Expand Down Expand Up @@ -815,7 +815,7 @@ L<HTML::LinkExtor> -- class for finding links in HTML documents
=item *

The book I<Perl & LWP> by Sean M. Burke. O'Reilly & Associates,
2002. ISBN: 0-596-00178-9, L<http://www.oreilly.com/catalog/perllwp/>. The
2002. ISBN: 0-596-00178-9, L<http://oreilly.com/catalog/perllwp/>. The
whole book is also available free online:
L<http://lwp.interglacial.com>.

Expand Down

0 comments on commit 7683c5f

Please sign in to comment.