A free, open API for everything you want to know about CPAN
Perl Perl6 Shell
Latest commit 1ab4491 Jul 27, 2016 @oalders oalders committed on GitHub Merge pull request #507 from metacpan/mickey/2_birds
mickey/2 birds
Failed to load latest commit information.
.github Adds link to volunteer needed issues from CONTRIBUTING.md Mar 8, 2016
bin update links for new org and repo names Jun 28, 2016
docs update links for new org and repo names Jun 28, 2016
elasticsearch Delete obviated (and probably never used) cpanratings script. Mar 29, 2016
etc Keep test data out of t directory. Apr 25, 2016
git Avoid weirdness with git stash when running pre-commit hook. Apr 2, 2016
lib script/release: correct order of document updates Jul 27, 2016
t update test server/controller/author Jul 11, 2016
test-data/fakecpan update links for new org and repo names Jun 28, 2016
.editorconfig Keep indentation consistent in travis yml Oct 15, 2014
.gitignore Added script which fetches and indexes river data Apr 23, 2016
.perlcriticrc Cleanup PC policy, fix mirrors to delete first #463 Apr 26, 2016
.perltidyrc Tweak perltidy to drastically reduce my annoyance level Feb 10, 2014
.tidyallrc Move Perl::Critic checks to TidyAll. Oct 29, 2015
.travis.yml Stop testing on Perls < 5.22. May 27, 2016
LICENSE License under same terms as Perl (GPL/Artistic) Jan 25, 2014
README.md update links for new org and repo names Jun 28, 2016
app.psgi Temporarily comment out some stash interactions. Nov 10, 2015
cpanfile t/model/archive: replace size check with content check May 27, 2016
cpanfile.snapshot snapshot updates May 27, 2016
metacpan_server.conf Get minion DSN from config file. Apr 18, 2016
metacpan_server_testing.conf Ensure METACPAN_SERVER_CONFIG_LOCAL_SUFFIX is set on Travis and in lo… Apr 29, 2016

README.md

Build Status Coverage Status

A Web Service for the CPAN

MetaCPAN aims to provide a free, open web service which provides metadata for CPAN modules.

REST API

MetaCPAN is based on Elasticsearch, so it provides a RESTful interface as well as the option to create complex queries. The wiki provides a good starting point for REST access to MetaCPAN.

Expanding Your Author Info

MetaCPAN allows authors to add custom metadata about themselves to the index. Log in to MetaCPAN to add more information about yourself.

Installing Your Own MetaCPAN:

If you want to run MetaCPAN locally, we encourage you to start with a VM: Metacpan Developer VM However, you may still find some info here:

Troubleshooting Elasticsearch

You can restart Elasticsearch (ES) manually if you need to troubleshoot.

sudo service elasticsearch restart

If you are unable to access [http://localhost:9200] you should kill the Elasticsearch process and run it in foreground to see the debug output

sudo service elasticsearch stop
cd /opt/elasticsearch
sudo bin/elasticsearch -f

If you get a "Can't start up: not enough memory" error when trying to start Elasticsearch, you likely need to update your JRE. On Ubuntu:

# fixes "not enough memory" errors
sudo apt-get install openjdk-6-jre

(Note: If you intend to try indexing a full MiniCPAN, you may find that Elasticsearch wants to use more open filehandles than your system allows by default. This script can be used to start ES with the appropriate ulimit adjustment).

Run the test suite

The test suite accesses Elasticsearch on port 9900. The developer VM should have a dedicated test instance running in the background already, but if you want to run it manually:

cd /opt/elasticsearch
sudo bin/elasticsearch -f -Des.http.port=9900 -Des.cluster.name=testing

Then run the test suite:

cd /home/metacpan/api.metacpan.org
./bin/prove t

The test suite has to pass all tests.

Create the ElasticSearch Index

./bin/run bin/metacpan mapping --delete

--delete will drop all indices first to clear the index from test data.

Begin Indexing Your Modules

./bin/run bin/metacpan release /path/to/cpan/authors/id/

You should note that you can index either your CPAN mirror or a minicpan mirror. You can even index just parts of a mirror:

./bin/run bin/metacpan release /path/to/cpan/authors/id/{A,B}

Tag the Latest Releases

./bin/run bin/metacpan latest --cpan /path/to/cpan/

Index Author Data

./bin/run bin/metacpan author --cpan /path/to/cpan/

Note that minicpan doesn't provide the 00whois.xml file which is used to generate the index; you will have to download it manually (it is in the authors/ directory) in order to index authors.

wget -O /path/to/cpan/authors/00whois.xml cpan.cpantesters.org/authors/00whois.xml

It also doesn't include author.json files, so that data will also be missing unless you get it from somewhere else.

Set Up Proxy in Front of ElasticSearch

Start API server on port 5000

./bin/run plackup -p 5000 -r

This will start a single-threaded test server. If you need extra performance, use Starman instead.

Notes

For a full list of options:

./bin/run bin/metacpan release --help

Contributing:

If you'd like to get involved, find us at #metacpan or irc.perl.org or join our mailing list (see below) and let us know what you'd like to start working on.

IRC

You can find us at #metacpan on irc.perl.org Access it via web interface: https://chat.mibbit.com/?channel=%23metacpan&server=irc.perl.org

IRC logs can be found here: http://irclog.perlgeek.de/metacpan/today (Thanks to Moritz Lenz for making this service available)

Mailing List

Our mailing list is open to all: http://groups.google.com/group/cpan-api