A free, open API for everything you want to know about CPAN
oalders Merge pull request #877 from metacpan/oalders/changes
Add CHANGES.md to list of possible changes files. (See Perl::Tidy)
Latest commit c7f57d3 Nov 20, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github Adds link to volunteer needed issues from CONTRIBUTING.md Mar 9, 2016
bin Update documentation for api.pl Nov 11, 2018
docs Referenced URL unresponsive Aug 6, 2018
elasticsearch Delete obviated (and probably never used) cpanratings script. Mar 30, 2016
etc Keep test data out of t directory. Apr 25, 2016
git Revert "different version of tidy all fix!" Jun 21, 2018
improve-search-results Add a README to explain reasoning and setup Feb 4, 2017
lib Add CHANGES.md to list of possible changes files. (See Perl::Tidy) Nov 19, 2018
root/static Add comments for Release tag Nov 11, 2018
t move new tests into t/api Nov 11, 2018
templates Added Model::User + filled search_identities Nov 11, 2018
test-data/fakecpan Added mirrors data indexing to tests Jul 10, 2017
xt always use lib t/lib Nov 17, 2017
.dockerignore Dockerize MetaCPAN API Jul 5, 2017
.editorconfig Keep indentation consistent in travis yml Oct 15, 2014
.gitignore add log4perl logging to server web app Nov 17, 2017
.mailmap add mailmap Nov 19, 2017
.perlcriticrc only run core Perl::Critic policies Jan 23, 2018
.perltidyrc Tweak perltidy to drastically reduce my annoyance level Feb 11, 2014
.tidyallrc Bulk commit various permission indexing changes. Apr 2, 2017
.travis.yml Add ES configuration Nov 15, 2018
Dockerfile 💄 put `&&` at the start of RUN subcommands Jul 5, 2017
LICENSE License under same terms as Perl (GPL/Artistic) Jan 25, 2014
README.md add kritika badge Nov 18, 2017
app.psgi allow mojo to serve the static assets from root/ Nov 11, 2018
cpanfile start using the mojo application for routes in the openapi spec Nov 11, 2018
cpanfile.snapshot Upgrade Mojo from 8.05 to 8.06 Nov 11, 2018
log4perl.conf add log4perl logging to server web app Nov 17, 2017
metacpan_server.conf Revert "use new Search::Elasticsearch and ElasticSearchX::Model" Jun 27, 2018
metacpan_server_testing.conf Move github credentials and Mojo secret to config file Nov 11, 2018

README.md

Build Status Coverage Status Kritika Analysis Status

A Web Service for the CPAN

MetaCPAN aims to provide a free, open web service which provides metadata for CPAN modules.

REST API

MetaCPAN is based on Elasticsearch, so it provides a RESTful interface as well as the option to create complex queries. The wiki provides a good starting point for REST access to MetaCPAN.

Expanding Your Author Info

MetaCPAN allows authors to add custom metadata about themselves to the index. Log in to MetaCPAN to add more information about yourself.

Installing Your Own MetaCPAN:

If you want to run MetaCPAN locally, we encourage you to start with a VM: Metacpan Developer VM However, you may still find some info here:

Troubleshooting Elasticsearch

You can restart Elasticsearch (ES) manually if you need to troubleshoot.

sudo service elasticsearch restart

If you are unable to access [[http://localhost:9200]] (give it a few seconds) you should kill the Elasticsearch process and run it in foreground to see the debug output

sudo service elasticsearch stop
cd /opt/elasticsearch
sudo bin/elasticsearch -f

If you get a "Can't start up: not enough memory" error when trying to start Elasticsearch, you likely need to update your JRE. On Ubuntu:

# fixes "not enough memory" errors
sudo apt-get install openjdk-6-jre

(Note: If you intend to try indexing a full MiniCPAN, you may find that Elasticsearch wants to use more open filehandles than your system allows by default. This script can be used to start ES with the appropriate ulimit adjustment).

Run the test suite

The test suite accesses Elasticsearch on port 9900. The developer VM should have a dedicated test instance running in the background already, but if you want to run it manually:

cd /opt/elasticsearch
sudo bin/elasticsearch -f -Des.http.port=9900 -Des.cluster.name=testing

Then run the test suite:

cd /home/metacpan/metacpan-api
./bin/prove t

The test suite has to pass all tests.

Create the ElasticSearch Index

./bin/run bin/metacpan mapping --delete

--delete will drop all indices first to clear the index from test data.

Begin Indexing Your Modules

./bin/run bin/metacpan release /path/to/cpan/authors/id/

You should note that you can index either your CPAN mirror or a minicpan mirror. You can even index just parts of a mirror:

./bin/run bin/metacpan release /path/to/cpan/authors/id/{A,B}

Tag the Latest Releases

./bin/run bin/metacpan latest --cpan /path/to/cpan/

Index Author Data

./bin/run bin/metacpan author --cpan /path/to/cpan/

Note that minicpan doesn't provide the 00whois.xml file which is used to generate the index; you will have to download it manually (it is in the authors/ directory) in order to index authors.

wget -O /path/to/cpan/authors/00whois.xml cpan.cpantesters.org/authors/00whois.xml

It also doesn't include author.json files, so that data will also be missing unless you get it from somewhere else.

Set Up Proxy in Front of ElasticSearch

Start API server on port 5000

./bin/run plackup -p 5000 -r

This will start a single-threaded test server. If you need extra performance, use Starman instead.

Notes

For a full list of options:

./bin/run bin/metacpan release --help

Contributing:

If you'd like to get involved, find us at #metacpan or irc.perl.org or join our mailing list (see below) and let us know what you'd like to start working on.

IRC

You can find us at #metacpan on irc.perl.org Access it via web interface: https://chat.mibbit.com/?channel=%23metacpan&server=irc.perl.org

IRC logs can be found here: http://irclog.perlgeek.de/metacpan/today (Thanks to Moritz Lenz for making this service available)