(perl) Queryable web-based CPAN Index
Perl Perl 6
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
lib/App
t
tools
.gitignore
MANIFEST
Makefile.PL
README

README

NAME
    App::CPANIDX - Queryable web-based CPAN Index

SYNOPSIS
      # Generate the index database

      $ cpanidx-gendb --config cpanidx.ini

      # Run the FastCGI script

      $ cpanidx-fcgi --config cpanidx.ini

DESCRIPTION
    App::CPANIDX provides a number of scripts to build a queryable web-based
    CPAN index.

CONFIGURATION
    Configuration is dealt with by a Config::Tiny based configuration file.

    There are a number of parameters which can be specified

    "dsn"
        The DBI dsn string of the database that the scripts will use. This
        is a mandatory requirement.

    "user"
        The username for the supplied "dsn".

    "pass"
        The password for the supplied "dsn".

    "url"
        The "cpanidx-gendb" script will poll this url when it has finished
        its update. It should be the root url of your CPANIDX site

          url=http://my.cpanidx.site/cpanidx/

    "mirror"
        The url of a CPAN mirror site where "cpanidx-gendb" will obtain its
        index files from. If not supplied it defaults to the Funet site
        <ftp://ftp.funet.fi/pub/CPAN/>.

    "socket"
        This is the socket that FCGI should listen on for requests. It is a
        mandatory requirement for the "cpanidx-fcgi" script.

    "skipcore"
        Applicable to the "cpanidx-gendb" script, will skip the generation
        of the Module::CoreList based tables.

    "skipmirrors"
        Applicable to the "cpanidx-gendb" script, will skip the generation
        of the mirrorlist based tables.

    "skipperms"
        Applicable to the "cpanidx-gendb" script, will skip the generation
        of the CPAN permissions based tables.

SCRIPTS
    Both the scripts will by default look for a "cpanidx.ini" file in the
    current working directory unless you specify an alternative with the
    "--config" command line option.

    "cpanidx-gendb"
        Generates the CPANIDX database. It will retrieve the CPAN index
        files from a CPAN mirror and parse them to build the database.

        The CPAN indexes are downloaded to "~/.cpanidx" by default. You may
        override this location by setting the "PERL5_CPANIDX_DIR"
        environment variable to a different location to use.

        In tests a DBD::SQLite database took over 3 minutes to generate and
        a DBD::mysql database took 30 seconds.

        It is recommended that one uses cron or some such scheduler to run
        this script every hour to ensure freshness of the CPAN index.

    "cpanidx-fcgi"
        Presents the CPAN index to web clients via FastCGI. Specify a socket
        that the script should listen for requests on and configure your
        webserver accordingly.

        The following is an example for Lighttpd:

          fastcgi.server = (
                "/cpanidx/" =>
                 ( "localhost" => (
                    "host" => "127.0.0.1",
                    "port" => 1027,
                    "check-local" => "disable",
                    )
                 ),
          )

        The interface that clients can query is described below.

INTERFACE
    The "cpanidx-fcgi" provides a number of ways that clients can access
    information from the CPAN Index.

    The information is provided in a number of different formats: YAML,
    JSON, XML and HTML.

    Information is requested by using a special URL

      http://name.of.website/<prefix>/<format>/<cmd>/<search_term>

    We will assume that <prefix> is "cpanidx" for the purposes of this
    documentation.

    "<format">
        The format may be one of "yaml", "json", "xml" or "html".

    "<cmd">
        The command may be one of the following:

        "mod"
            Takes a search term which is a module name to search for.
            Returns information relating to that module if it exists.

              curl -i http://name.of.website/cpanidx/yaml/mod/LWP

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Thu, 04 Mar 2010 11:34:07 GMT
              Server: lighttpd/1.4.25

              ---
              -
                cpan_id: GAAS
                dist_file: G/GA/GAAS/libwww-perl-5.834.tar.gz
                dist_name: libwww-perl
                dist_vers: 5.834
                mod_name: LWP
                mod_vers: 5.834

        "dist"
            Takes a search term which is a distribution name to search for.
            Returns information relating to that distribution if it exists.

              curl -i http://name.of.website/cpanidx/yaml/dist/CPANPLUS-Dist-Build

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Mon, 06 Sep 2010 14:02:23 GMT
              Server: lighttpd/1.4.25

              ---
              -
                cpan_id: BINGOS
                dist_file: B/BI/BINGOS/CPANPLUS-Dist-Build-0.48.tar.gz
                dist_name: CPANPLUS-Dist-Build
                dist_vers: 0.48

        "auth"
            Takes a search term which is the CPAN ID of an author to search
            for. Returns information relating to that author if they exist.

              curl -i http://name.of.website/cpanidx/yaml/auth/BINGOS

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Thu, 04 Mar 2010 11:36:13 GMT
              Server: lighttpd/1.4.25

              ---
              -
                cpan_id: BINGOS
                email: chris@bingosnet.co.uk
                fullname: 'Chris Williams'

        "dists"
            Takes a search term which is the CPAN ID of an author. Returns a
            list of distributions that author has on CPAN.

              curl -i http://name.of.website/cpanidx/yaml/dists/BINGOS

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Thu, 04 Mar 2010 11:39:14 GMT
              Server: lighttpd/1.4.25

              ---
              -
                cpan_id: BINGOS
                dist_file: B/BI/BINGOS/POE-Filter-LZO-1.70.tar.gz
                dist_name: POE-Filter-LZO
                dist_vers: 1.70
              -
                cpan_id: BINGOS
                dist_file: B/BI/BINGOS/POE-Component-Server-SimpleSMTP-1.44.tar.gz
                dist_name: POE-Component-Server-SimpleSMTP
                dist_vers: 1.44
              -
                cpan_id: BINGOS
                dist_file: B/BI/BINGOS/POE-Component-Server-RADIUS-1.02.tar.gz
                dist_name: POE-Component-Server-RADIUS
                dist_vers: 1.02
              -
                cpan_id: BINGOS
                dist_file: B/BI/BINGOS/Archive-Extract-0.38.tar.gz
                dist_name: Archive-Extract
                dist_vers: 0.38
              -
                cpan_id: BINGOS
                dist_file: B/BI/BINGOS/POE-Component-IRC-Plugin-URI-Find-1.08.tar.gz
                dist_name: POE-Component-IRC-Plugin-URI-Find
                dist_vers: 1.08
              -
                cpan_id: BINGOS
                dist_file: B/BI/BINGOS/POE-Component-SmokeBox-Dists-1.00.tar.gz
                dist_name: POE-Component-SmokeBox-Dists
                dist_vers: 1.00

            etc, etc.

        "perms"
            Takes a search term which is a module name to search for.
            Returns CPAN permissions relating to that module if it exists.

            The permission is one of "m" for "modulelist", "f" for
            "first-come" and "c" for "co-maint".

              curl -i http://name.of.website/cpanidx/yaml/perms/POE::Component::SmokeBox::Dists

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Thu, 04 Mar 2010 11:39:14 GMT
              Server: lighttpd/1.4.25

              ---
              -
                mod_name: POE::Component::SmokeBox::Dists
                cpan_id: BINGOS
                perms: f

        "timestamp"
            Does not take a search term. Returns a timestamp of when the
            CPAN Index Database was last updated and when the packages file
            that was used was last updated. Both values are in epoch time.

              curl -i http://name.of.website/cpanidx/yaml/timestamp

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Wed, 09 Jun 2010 10:16:15 GMT
              Server: lighttpd/1.4.25

              ---
              -
                lastupdated: 1276075625
                timestamp: 1276077865

        "topten"
            Does not take a search term. Returns a list of the authors with
            the most distributions. This is not the most accurate, try
            <http://thegestalt.org/simon/perl/wholecpan.html> for a more
            accurate leaderboard.

              curl -i http://name.of.website/cpanidx/yaml/topten

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Thu, 04 Mar 2010 11:44:44 GMT
              Server: lighttpd/1.4.25

              ---
              -
                cpan_id: ADAMK
                dists: 237
              -
                cpan_id: RJBS
                dists: 215
              -
                cpan_id: ZOFFIX
                dists: 212
              -
                cpan_id: MIYAGAWA
                dists: 190
              -
                cpan_id: SMUELLER
                dists: 130
              -
                cpan_id: NUFFIN
                dists: 122
              -
                cpan_id: TOKUHIROM
                dists: 121
              -
                cpan_id: BINGOS
                dists: 121
              -
                cpan_id: GUGOD
                dists: 118
              -
                cpan_id: MARCEL
                dists: 114

        "mirrors"
            Does not take a search term. Returns a list of CPAN mirror sites
            as listed in the "MIRRORED.BY" file.

              curl -i http://name.of.website/cpanidx/yaml/mirrors

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Mon, 19 Apr 2010 14:52:52 GMT
              Server: lighttpd/1.4.25

              ---
              -
                dst_bandwidth: STM-1
                dst_contact: tenet.ac.za^aa
                dst_ftp: ftp://cpan.mirror.ac.za/
                dst_http: http://cpan.mirror.ac.za/
                dst_location: 'Cape Town, South Africa, Africa (-33.93 18.47)'
                dst_notes: ''
                dst_organisation: TENET
                dst_rsync: mirror.ac.za::cpan
                dst_src: rsync://www.cpan.org/CPAN/
                dst_timezone: '+2'
                frequency: '12 Hourly'
                hostname: mirror.ac.za
              -
                dst_bandwidth: 50MB
                dst_contact: is.co.za|ftpadmin
                dst_ftp: ftp://ftp.is.co.za/pub/cpan/
                dst_http: http://mirror.is.co.za/pub/cpan/
                dst_location: 'Johannesburg, Gauteng, South Africa, Africa (-26.17 28.03)'
                dst_notes: 'Limit to 4 simultaneous connections.'
                dst_organisation: 'Internet Solutions'
                dst_rsync: ftp.is.co.za::IS-Mirror/ftp.cpan.org/
                dst_src: rsync.nic.funet.fi
                dst_timezone: '+2'
                frequency: daily
                hostname: is.co.za
              -
                dst_bandwidth: T3
                dst_contact: saix.net=ftp
                dst_ftp: ftp://ftp.saix.net/pub/CPAN/
                dst_http: ''
                dst_location: 'Parow, Western Cape, South Africa, Africa (-33.9064 18.5631)'
                dst_notes: ''
                dst_organisation: 'South African Internet eXchange (SAIX)'
                dst_rsync: ''
                dst_src: ftp.funet.fi
                dst_timezone: '+2'
                frequency: daily
                hostname: saix.net
              -

            etc. etc.

        "corelist"
            Takes a search term which is a module name to search for.
            Returns information if that module is shipped with perl core.

              curl -i http://name.of.website/cpanidx/yaml/corelist/Class::ISA

              HTTP/1.1 200 OK
              Content-type: application/x-yaml; charset=utf-8
              Transfer-Encoding: chunked
              Date: Thu, 06 May 2010 09:31:25 GMT
              Server: lighttpd/1.4.25

              ---
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.007003
                released: 2002-03-05
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.008
                released: 2002-07-19
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.008001
                released: 2003-09-25
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.008002
                released: 2003-11-05
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.008003
                released: 2004-01-14
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.008004
                released: 2004-04-21
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.008005
                released: 2004-07-19
              -
                deprecated: 0
                mod_vers: 0.32
                perl_ver: 5.008006
                released: 2004-11-27

            etc etc.

AUTHOR
    Chris "BinGOs" Williams <chris@bingosnet.co.uk>

LICENSE
    Copyright © Chris Williams

    This module may be used, modified, and distributed under the same terms
    as Perl itself. Please see the license that came with your Perl
    distribution for details.

SEE ALSO
    Config::Tiny

    DBI

    FCGI