Skip to content

Commit

Permalink
Merge branch 'devel'
Browse files Browse the repository at this point in the history
  • Loading branch information
dvarrazzo committed Sep 29, 2012
2 parents 71951c9 + 96d0010 commit 3e33bb1
Show file tree
Hide file tree
Showing 24 changed files with 845 additions and 261 deletions.
27 changes: 27 additions & 0 deletions AUTHORS
@@ -0,0 +1,27 @@
Who has contributed to the PGXN client?
=======================================

Daniele Varrazzo

He rushed to implement a client before David could do it in Perl!

David Wheeler

He is the PGXN mastermind: a lot of helpful design discussions.

Peter Eisentraut

First implementation of tarball support. Auto-sudo is not a good idea, I
got it.

Hitoshi Harada

Tricky installation corner cases.

Andrey Popp

Make selection. Helped the program not to suck on BSD!

Also thank you everybody for the useful discussions on the PGXN mailing list,
bug reports, proofreading the docs and the general support to the project.

11 changes: 11 additions & 0 deletions CHANGES
Expand Up @@ -3,6 +3,17 @@
PGXN Client changes log
-----------------------

pgxnclient 1.2
==============

- Packages can be downloaded, installed, loaded specifying an URL
(ticket #15).
- Added support for ``.tar`` files (ticket #17).
- Use ``gmake`` in favour of ``make`` for platforms where the two are
distinct, such as BSD (ticket #14).
- Added ``--make`` option to select the make executable (ticket #16).


pgxnclient 1.1
==============

Expand Down
2 changes: 1 addition & 1 deletion MANIFEST.in
@@ -1,4 +1,4 @@
include CHANGES COPYING MANIFEST.in README.rst setup.py Makefile
include AUTHORS CHANGES COPYING MANIFEST.in README.rst setup.py Makefile
include bin/pgxn bin/pgxnclient
recursive-include pgxnclient *.py
recursive-include testdata *
Expand Down
6 changes: 3 additions & 3 deletions docs/index.rst
Expand Up @@ -20,16 +20,16 @@ which would load the extension in one of the databases of the server.
The client interacts with the PGXN web service and a ``Makefile`` provided by
the extension. The best results are achieved with makefiles using the
PostgreSQL `Extension Building Infrastructure`__; however the client tries to
degrade gracefully in presence of any package hosted on PGXN.
degrade gracefully in presence of any package hosted on PGXN and any package
available outside the extension network.

.. _semver: http://pgxn.org/dist/semver
.. __: http://www.postgresql.org/docs/9.1/static/extend-pgxs.html
.. __: http://www.postgresql.org/docs/current/static/extend-pgxs.html

- Home page: http://pgxnclient.projects.postgresql.org/
- Downloads: http://pypi.python.org/pypi/pgxnclient/
- Discussion group: http://groups.google.com/group/pgxn-users/
- Source repository: https://github.com/dvarrazzo/pgxnclient/
- PgFoundry project: http://pgfoundry.org/projects/pgxnclient/


Contents:
Expand Down
80 changes: 51 additions & 29 deletions docs/usage.rst
Expand Up @@ -60,10 +60,17 @@ Whenever a command takes a specification in input, it also accepts options
``--stable``, ``--testing`` and ``--unstable`` to specify the minimum release
status accepted. The default is "stable".

A few commands also allow specifying a local ``.zip`` package or a local
directory containing a distribution: in this case the specification should
contain at least a path separator to disambiguate it from a distribution name,
for instance ``pgxn install ./foo.zip``.
A few commands also allow specifying a local archive or local directory
containing a distribution: in this case the specification should contain at
least a path separator to disambiguate it from a distribution name (for
instance ``pgxn install ./foo.zip``) or it should be specified as an URL with
``file://`` schema.

A few commands also allow specifying a remote package with a URL. Currently
the schemas ``http://`` and ``https://`` are supported.

Currently the client supports ``.zip`` and ``.tar`` archives (eventually with
*gzip* and *bz2* compression).


.. _install:
Expand All @@ -79,12 +86,14 @@ Usage:
:class: pgxn-install
pgxn install [--help] [--stable | --testing | --unstable]
[--pg_config *PATH*] [--sudo [*PROG*] | --nosudo]
[--pg_config *PROG*] [--make *PROG*]
[--sudo [*PROG*] | --nosudo]
*SPEC*
The program takes a `package specification`_ identifying the distribution to
work with. The download phase is skipped if the distribution specification
refers to a local directory or package.
refers to a local directory or package. The package may be specified with an
URL.

Note that the built extension is not loaded in any database: use the command
`load`_ for this purpose.
Expand All @@ -95,13 +104,20 @@ then will perform ``make all`` and ``make install``. It is assumed that the
but this is not enforced: you may provide any Makefile as long as the expected
commands are implemented.

.. _PGXS: http://www.postgresql.org/docs/9.1/static/extend-pgxs.html
.. _PGXS: http://www.postgresql.org/docs/current/static/extend-pgxs.html

If there are many PostgreSQL installations on the system, the extension will
be built and installed against the instance whose :program:`pg_config` is
first found on the :envvar:`PATH`. A different instance can be specified using
the option :samp:`--pg_config {PATH}`.

The PGXS_ build system relies on a presence of `GNU Make`__: in many systems
it is installed as :program:`gmake` or :program:`make` executable. The program
will use the first of them on the path. You can specify an alternative program
using ``--make`` option.

.. __: http://www.gnu.org/software/make/

If the extension is being installed into a system PostgreSQL installation, the
install phase will likely require root privileges to be performed. In this
case either run the command under :program:`sudo` or specify the ``--sudo``
Expand Down Expand Up @@ -131,12 +147,14 @@ Usage:
:class: pgxn-check
pgxn check [--help] [--stable | --testing | --unstable]
[--pg_config *PATH*] [-d *DBNAME*] [-h *HOST*] [-p *PORT*] [-U *NAME*]
[--pg_config *PROG*] [--make *PROG*]
[-d *DBNAME*] [-h *HOST*] [-p *PORT*] [-U *NAME*]
*SPEC*
The command takes a `package specification`_ identifying the distribution to
work with, which can also be a local file or directory. The distribution is
unpacked if required and the ``installcheck`` make target is run.
work with, which can also be a local file or directory or an URL. The
distribution is unpacked if required and the ``installcheck`` make target is
run.

.. note::
The command doesn't run ``make all`` before ``installcheck``: if any file
Expand All @@ -159,6 +177,8 @@ The database connection options are similar to the ones in load_, with the
difference that the variable :envvar:`PGDATABASE` doesn't influence the
database name.

See the install_ command for details about the command arguments.

.. warning::
At the time of writing, :program:`pg_regress` on Debian and derivatives is
affected by `bug #554166`__ which makes *HOST* selection impossible.
Expand All @@ -179,7 +199,8 @@ Usage:
:class: pgxn-uninstall
pgxn uninstall [--help] [--stable | --testing | --unstable]
[--pg_config *PATH*] [--sudo [*PROG*] | --nosudo]
[--pg_config *PROG*] [--make *PROG*]
[--sudo [*PROG*] | --nosudo]
*SPEC*
The command does the opposite of the install_ command, removing a
Expand Down Expand Up @@ -212,11 +233,11 @@ Usage:
*SPEC* [*EXT* [*EXT* ...]]
The distribution is specified according to the `package specification`_ and
can refer to a local directory or file. No consistency check is performed
between the packages specified in the ``install`` and ``load`` command: the
specifications should refer to compatible packages. The specified distribution
is only used to read the metadata: only installed files are actually used to
issue database commands.
can refer to a local directory or file or to an URL. No consistency check is
performed between the packages specified in the ``install`` and ``load``
command: the specifications should refer to compatible packages. The specified
distribution is only used to read the metadata: only installed files are
actually used to issue database commands.

The database to install into can be specified using options
``-d``/``--dbname``, ``-h``/``--host``, ``-p``/``--port``,
Expand All @@ -235,8 +256,8 @@ extension specifies a ``.control`` file, it will be loaded using the `CREATE
EXTENSION`_ command, otherwise it will be loaded as a loose set of objects.
For more information see the `extensions documentation`__.

.. _CREATE EXTENSION: http://www.postgresql.org/docs/9.1/static/sql-createextension.html
.. __: http://www.postgresql.org/docs/9.1/static/extend-extensions.html
.. _CREATE EXTENSION: http://www.postgresql.org/docs/current/static/sql-createextension.html
.. __: http://www.postgresql.org/docs/current/static/extend-extensions.html

The command is based on the `'provides' section`_ of the distribution's
``META.json``: if a SQL file is specified, that file will be used to load the
Expand All @@ -249,7 +270,7 @@ confirmation.

If the distribution provides more than one extension, the extensions are
loaded in the order in which they are specified in the ``provides`` section of
the ``META.json`` file. It is also possilbe to load only a few of the
the ``META.json`` file. It is also possible to load only a few of the
extensions provided, specifying them after *SPEC*: the extensions will be
loaded in the order specified.

Expand Down Expand Up @@ -297,11 +318,11 @@ itself, so the option will be ignored.

If the distribution specifies more than one extension, they are unloaded in
reverse order respect to the order in which they are specified in the
``META.json`` file. It is also possilbe to unload only a few of the
``META.json`` file. It is also possible to unload only a few of the
extensions provided, specifying them after *SPEC*: the extensions will be
unloaded in the order specified.

.. _DROP EXTENSION: http://www.postgresql.org/docs/9.1/static/sql-dropextension.html
.. _DROP EXTENSION: http://www.postgresql.org/docs/current/static/sql-dropextension.html

See the load_ command for details about the command arguments.

Expand All @@ -322,11 +343,12 @@ Usage:
[--target *PATH*]
*SPEC*
The distribution is specified according to the `package specification`_. The
file is saved in the current directory with name usually
:samp:`{distribution}-{version}.zip`. If a file with the same name exists, a
suffix ``-1``, ``-2`` etc. is added to the name, before the extension. A
different directory or name can be specified using the ``--target`` option.
The distribution is specified according to the `package specification`_ and
can be represented by an URL. The file is saved in the current directory with
name usually :samp:`{distribution}-{version}.zip`. If a file with the same
name exists, a suffix ``-1``, ``-2`` etc. is added to the name, before the
extension. A different directory or name can be specified using the
``--target`` option.


.. _pgxn-search:
Expand Down Expand Up @@ -407,9 +429,9 @@ Usage:
[--details | --meta | --readme | --versions]
*SPEC*
The distribution is specified according to the `package specification`_.
The command output is a list of values obtained by the distribution's
``META.json`` file, for example:
The distribution is specified according to the `package specification`_. It
cannot be a local dir or file nor an URL. The command output is a list of
values obtained by the distribution's ``META.json`` file, for example:

.. code-block:: console
Expand Down
2 changes: 1 addition & 1 deletion pgxnclient/__init__.py
Expand Up @@ -6,7 +6,7 @@

# This file is part of the PGXN client

__version__ = '1.1'
__version__ = '1.2'

# Paths where to find the command executables.
# If relative, it's from the `pgxnclient` package directory.
Expand Down
6 changes: 3 additions & 3 deletions pgxnclient/api.py
Expand Up @@ -10,9 +10,9 @@

from urllib import urlencode

from pgxnclient import network
from pgxnclient.utils import load_json
from pgxnclient.errors import NetworkError, NotFound, ResourceNotFound
from pgxnclient.network import get_file
from pgxnclient.utils.uri import expand_template


Expand Down Expand Up @@ -78,7 +78,7 @@ def user(self, username):

def call(self, meth, args=None, query=None):
url = self.get_url(meth, args, query)
return get_file(url)
return network.get_file(url)

def get_url(self, meth, args=None, query=None):
tmpl = self.get_template(meth)
Expand All @@ -98,7 +98,7 @@ def get_index(self):
if self._api_index is None:
url = self.mirror.rstrip('/') + '/index.json'
try:
with get_file(url) as f:
with network.get_file(url) as f:
self._api_index = load_json(f)
except ResourceNotFound:
raise NetworkError("API index not found at '%s'" % url)
Expand Down
99 changes: 99 additions & 0 deletions pgxnclient/archive.py
@@ -0,0 +1,99 @@
"""
pgxnclient -- archives handling
"""

# Copyright (C) 2011-2012 Daniele Varrazzo

# This file is part of the PGXN client

import os

from pgxnclient.i18n import _
from pgxnclient.utils import load_jsons
from pgxnclient.errors import PgxnClientException

def from_spec(spec):
"""Return an `Archive` instance to handle the file requested by *spec*
"""
assert spec.is_file()
return from_file(spec.filename)

def from_file(filename):
"""Return an `Archive` instance to handle the file *filename*
"""
from pgxnclient.zip import ZipArchive
from pgxnclient.tar import TarArchive

for cls in (ZipArchive, TarArchive):
a = cls(filename)
if a.can_open():
return a

raise PgxnClientException(
_("can't open archive '%s': file type not recognized")
% filename)


class Archive(object):
"""Base class to handle archives."""
def __init__(self, filename):
self.filename = filename

def can_open(self):
"""Return `!True` if the `!filename` can be opened by the obect."""
raise NotImplementedError

def open(self):
"""Open the archive for usage.
Raise PgxnClientException if the archive can't be open.
"""
raise NotImplementedError

def close(self):
"""Close the archive after usage."""
raise NotImplementedError

def list_files(self):
"""Return an iterable with the list of file names in the archive."""
raise NotImplementedError

def read(self, fn):
"""Return a file's data from the archive."""
raise NotImplementedError

def unpack(self, destdir):
raise NotImplementedError

def get_meta(self):
filename = self.filename

self.open()
try:
# Return the first file with the expected name
for fn in self.list_files():
if fn.endswith('META.json'):
return load_jsons(self.read(fn).decode('utf8'))
else:
raise PgxnClientException(
_("file 'META.json' not found in archive '%s'") % filename)
finally:
self.close()

def _find_work_directory(self, destdir):
"""
Choose the directory where to work.
Because we are mostly a wrapper for pgxs, let's look for a makefile.
The tar should contain a single base directory, so return the first
dir we found containing a Makefile, alternatively just return the
unpacked dir
"""
for dir in os.listdir(destdir):
for fn in ('Makefile', 'makefile', 'GNUmakefile', 'configure'):
if os.path.exists(os.path.join(destdir, dir, fn)):
return os.path.join(destdir, dir)

return destdir


3 changes: 3 additions & 0 deletions pgxnclient/cli.py
Expand Up @@ -122,3 +122,6 @@ def _get_exec(cmd):

return fn

if __name__ == '__main__':
script()

0 comments on commit 3e33bb1

Please sign in to comment.