Skip to content

Commit

Permalink
Merge pull request #59 from jaraco/feature/57-infer-script
Browse files Browse the repository at this point in the history
Infer python args based on extant script.
  • Loading branch information
jaraco committed Dec 10, 2022
2 parents 6d1f22b + d71f44f commit daa7a93
Show file tree
Hide file tree
Showing 6 changed files with 156 additions and 44 deletions.
106 changes: 71 additions & 35 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ Following the parameters to ``pip install``, one may optionally
include a ``--`` after which any parameters will be passed
to a Python interpreter in the context.

See ``pip-run --help`` for more details.

Examples
========

Expand Down Expand Up @@ -121,16 +123,6 @@ executable modules and packages via
.. image:: docs/cowsay.svg
:alt: cowsay example animation

Interactive Interpreter
-----------------------

``pip-run`` also offers a painless way to run a Python interactive
interpreter in the context of certain dependencies::

$ /clean-install/python -m pip-run boto
>>> import boto
>>>


Command Runner
--------------
Expand All @@ -152,36 +144,47 @@ or on Windows::

$ py -2.7 -m pip-run ...

Experiments and Testing
-----------------------
Script Runner
-------------

Because ``pip-run`` provides a single-command invocation, it
is great for experiments and rapid testing of various package
specifications.
``pip-run`` can run a Python file with indicated dependencies. Because
arguments after ``--`` are passed directly to the Python interpreter
and because the Python interpreter will run any script, invoking a script
with dependencies is easy. Consider this script "myscript.py":

Consider a scenario in which one wishes to create an environment
where two different versions of the same package are installed,
such as to replicate a broken real-world environment. Stack two
invocations of pip-run to get two different versions installed::
.. code-block:: python
$ pip-run keyring==21.8.0 -- -m pip-run keyring==22.0.0 -- -c "import importlib.metadata, pprint; pprint.pprint([dist._path for dist in importlib.metadata.distributions() if dist.metadata['name'] == 'keyring'])"
[PosixPath('/var/folders/03/7l0ffypn50b83bp0bt07xcch00n8zm/T/pip-run-a3xvd267/keyring-22.0.0.dist-info'),
PosixPath('/var/folders/03/7l0ffypn50b83bp0bt07xcch00n8zm/T/pip-run-1fdjsgfs/keyring-21.8.0.dist-info')]
#!/usr/bin/env python
.. todo: illustrate example here
import requests
Script Runner
-------------
req = requests.get('https://pypi.org/project/pip-run')
print(req.status_code)
To invoke it while making sure requests is present:

$ pip-run requests -- myscript.py

Let's say you have a script that has a one-off purpose. It's either not
part of a library, where dependencies are normally declared, or it is
normally executed outside the context of that library. Still, that script
probably has dependencies, say on `requests
<https://pypi.org/project/requests>`_. Here's how you can use pip-run to
declare the dependencies and launch the script in a context where
those dependencies have been resolved.
``pip-run`` will make sure that requests is installed then invoke
the script in a Python interpreter configured with requests and its
dependencies.

First, add a ``__requires__`` directive at the head of the script:
For added convenience when running scripts, ``pip-run`` will infer
the beginning of Python parameters if it encounters a filename
of a Python script that exists, allowing for omission of the ``--``
for script invocation:

$ pip-run requests myscript.py

Script-declared Dependencies
----------------------------

Building on Script Runner above, ``pip-run`` also allows
dependencies to be declared in the script itself so that
the user need not specify them at each invocation.

To declare dependencies in a script, add a ``__requires__``
variable to the script:

.. code-block:: python
Expand All @@ -194,13 +197,17 @@ First, add a ``__requires__`` directive at the head of the script:
req = requests.get('https://pypi.org/project/pip-run')
print(req.status_code)
Then, simply invoke that script with pip-run::
With that declaration in place, one can now invoke ``pip-run`` without
declaring any parameters to pip::

$ python -m pip-run -- myscript.py
$ pip-run myscript.py
200

The format for requirements must follow `PEP 508 <https://www.python.org/dev/peps/pep-0508/>`_.

Other Script Directives
-----------------------

``pip-run`` also recognizes a global ``__index_url__`` attribute. If present,
this value will supply ``--index-url`` to pip with the attribute value,
allowing a script to specify a custom package index:
Expand Down Expand Up @@ -235,6 +242,35 @@ the same technique works for pipenv::

$ pipenv install $(python -m pip_run.read-deps script.py)

Interactive Interpreter
-----------------------

``pip-run`` also offers a painless way to run a Python interactive
interpreter in the context of certain dependencies::

$ /clean-install/python -m pip-run boto
>>> import boto
>>>

Experiments and Testing
-----------------------

Because ``pip-run`` provides a single-command invocation, it
is great for experiments and rapid testing of various package
specifications.

Consider a scenario in which one wishes to create an environment
where two different versions of the same package are installed,
such as to replicate a broken real-world environment. Stack two
invocations of pip-run to get two different versions installed::

$ pip-run keyring==21.8.0 -- -m pip-run keyring==22.0.0 -- -c "import importlib.metadata, pprint; pprint.pprint([dist._path for dist in importlib.metadata.distributions() if dist.metadata['name'] == 'keyring'])"
[PosixPath('/var/folders/03/7l0ffypn50b83bp0bt07xcch00n8zm/T/pip-run-a3xvd267/keyring-22.0.0.dist-info'),
PosixPath('/var/folders/03/7l0ffypn50b83bp0bt07xcch00n8zm/T/pip-run-1fdjsgfs/keyring-21.8.0.dist-info')]

.. todo: illustrate example here
How Does It Work
================

Expand Down
2 changes: 1 addition & 1 deletion examples/test-mongodb-covered-query.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
simple execution of complex tasks with their
dependencies.
Run this example with ``pip-run -- $script``.
Run this example with ``pip-run $script``.
It creates a MongoDB instance, and then runs some
assertions against it.
Expand Down
80 changes: 72 additions & 8 deletions pip_run/commands.py
Original file line number Diff line number Diff line change
@@ -1,22 +1,81 @@
import os
import textwrap
import pathlib
import contextlib
import warnings

from more_itertools import split_before


def _separate_script(args):
"""
Inject a double-dash before the first arg that appears to be an
extant Python script.
>>> _separate_script(['foo', 'bar'])
[['foo', 'bar'], []]
>>> _separate_script(['foo', 'pip-run.py', 'bar'])
[['foo'], ['pip-run.py', 'bar']]
>>> _separate_script(['path.py', 'pip-run.py'])
[['path.py'], ['pip-run.py']]
>>> _separate_script(['README.rst'])
[['README.rst'], []]
"""

def is_extant_path(item: 'os.PathLike[str]'):
path = pathlib.Path(item)
return path.is_file() and path.suffix == '.py'

groups = split_before(args, is_extant_path, maxsplit=1)
return [next(groups), next(groups, [])]


def _separate_dash(args):
"""
Separate args based on a dash separator.
>>> _separate_dash(['foo', '--', 'bar'])
(['foo'], ['bar'])
>>> _separate_dash(['foo', 'bar', '--'])
(['foo', 'bar'], [])
>>> _separate_dash(['foo', 'bar'])
Traceback (most recent call last):
...
ValueError: '--' is not in list
"""
pivot = args.index('--')
return args[:pivot], args[pivot + 1 :]


def parse_script_args(args):
"""
Separate the command line arguments into arguments for pip
and arguments to Python.
"""
with contextlib.suppress(ValueError):
return _separate_dash(args)

>>> parse_script_args(['foo', '--', 'bar'])
return _separate_script(args)


def separate_dash(args):
"""
Separate args based on dash separator.
Deprecated; retained for compatibility.
>>> separate_dash(['foo', '--', 'bar'])
(['foo'], ['bar'])
>>> parse_script_args(['foo', 'bar'])
(['foo', 'bar'], [])
>>> separate_dash(['foo', 'bar'])
[['foo', 'bar'], []]
"""
try:
pivot = args.index('--')
except ValueError:
pivot = len(args)
return args[:pivot], args[pivot + 1 :]
warnings.warn("separate_dash is deprecated", DeprecationWarning)
with contextlib.suppress(ValueError):
return _separate_dash(args)
return [args, []]


help_doc = textwrap.dedent(
Expand All @@ -37,6 +96,11 @@ def parse_script_args(args):
pip-run -- script.py
For simplicity, the ``--`` may be omitted and Python arguments will
be inferred starting with the first Python file that exists:
pip-run script.py
If the `--` is ommitted or nothing is passed, the python interpreter
will be launched in interactive mode:
Expand Down
8 changes: 8 additions & 0 deletions py38compat.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
import sys
import os


if sys.version_info < (3, 9):
PathLike_str_type = 'os.PathLike[str]'
else:
PathLike_str_type = os.PathLike[str]
3 changes: 3 additions & 0 deletions pytest.ini
Original file line number Diff line number Diff line change
Expand Up @@ -26,3 +26,6 @@ filterwarnings=

# jupyter/nbformat#232
ignore:Passing a schema to Validator.iter_errors is deprecated

# known internal deprecations
ignore:separate_dash is deprecated
1 change: 1 addition & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ install_requires =
path >= 15.1
importlib_metadata; python_version < "3.8"
packaging
more_itertools

[options.packages.find]
exclude =
Expand Down

0 comments on commit daa7a93

Please sign in to comment.