-
Notifications
You must be signed in to change notification settings - Fork 4
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
4 changed files
with
179 additions
and
11 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
Changes | ||
======= | ||
|
||
0.1 | ||
--- | ||
|
||
* Initial release |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,14 +1,172 @@ | ||
arca | ||
==== | ||
|
||
.. image:: https://travis-ci.org/mikicz/arca.svg?branch=master | ||
:target: https://travis-ci.org/mikicz/arca | ||
.. image:: https://img.shields.io/travis/mikicz/arca.svg | ||
:target: https://travis-ci.org/mikicz/arca | ||
|
||
.. image:: https://codecov.io/gh/mikicz/arca/branch/master/graph/badge.svg | ||
:target: https://codecov.io/gh/mikicz/arca | ||
.. image:: https://img.shields.io/codecov/c/github/mikicz/arca.svg | ||
:target: https://codecov.io/gh/mikicz/arca | ||
|
||
.. image:: https://badge.fury.io/py/arca.svg | ||
:target: https://badge.fury.io/py/arca | ||
.. image:: https://img.shields.io/pypi/v/arca.svg | ||
:target: https://pypi.python.org/pypi/arca | ||
|
||
Home repository: `mikicz/arca <https://github.com/mikicz/arca>`_. | ||
.. image:: https://img.shields.io/github/license/mikicz/arca.svg?style=flat | ||
:target: https://github.com/mikicz/arca/blob/master/LICENSE | ||
|
||
Arca is a library for running Python scripts from git repositories in various states of isolation. | ||
Arca can also cache the results of these scripts using `dogpile.cache <https://dogpilecache.readthedocs.io/en/latest/>`_. | ||
|
||
Getting started | ||
*************** | ||
|
||
Requirements | ||
++++++++++++ | ||
|
||
* Python >= 3.6 | ||
* Internet connection | ||
|
||
|
||
Optional requirements for certain backends, they need to be runnable by the user using the library: | ||
* Docker | ||
* Vagrant | ||
|
||
Optional for some caching backends: | ||
* Redis | ||
* Memcached | ||
|
||
Basic Principle | ||
+++++++++++++++ | ||
|
||
To run a Hello World you'll only need the ``arca.Arca`` and ``arca.Task`` classes. | ||
``Task`` is used for defining the task that's supposed to be run in the repositories. | ||
``Arca`` takes care of all the settings and provides the basic API for running tasks. | ||
|
||
Lets presume that we have a repository ``https://example.com/hello_word.git`` with branch ``master`` that contains a | ||
file ``hello_world.py`` with function ``say_hello``. In that case you would use the following example to launch the function | ||
in a isolated environment: | ||
|
||
.. code-block:: python | ||
from arca import Arca, Task | ||
task = Task("hello_world:say_hello") | ||
arca = Arca() | ||
result = arca.run("https://example.com/hello_word.git", "master", task) | ||
Result will contain a ``arca.Result`` instance which currently only has one attribute ``output`` with the output of the function call. | ||
Should running of the task fail, ``arca.exceptions.BuildError`` will be raised. | ||
|
||
By default, the ``CurrentEnvironmentBackend`` is used to run tasks, which uses the python installation which is used to run | ||
arca, launching the code in a subprocess. You can learn about using different backends bellow. | ||
|
||
Configuring arca | ||
++++++++++++++++ | ||
|
||
There are three ways to configure ``Arca``. | ||
|
||
1. You can initialize the class and backends directly and set it's options via constructor arguments. | ||
For example changing the base directory where repositories are cloned and other arca-related things are saved you would call Arca like this: | ||
|
||
.. code-block:: python | ||
arca = Arca(base_dir=".custom_arca_dir") | ||
This option is the most direct but it has one caveat - options set by this method cannot be overidden by the following methods. | ||
|
||
2. You can pass a dict with settings. The keys are always uppercase and prefixed with ``ARCA_``. | ||
For example the same setting as above would be written as: | ||
|
||
.. code-block:: python | ||
arca = Arca(settings={ | ||
"ARCA_BASE_DIR": ".custom_arca_dir" | ||
}) | ||
3. You can configure ``Arca`` with environ variables, with keys being the same as in the second method. | ||
Environ variables override settings from the second method. | ||
|
||
|
||
You can combine these methods as long as you remember that options explicitly specified in constructors | ||
cannot be overridden by the settings and environ methods. | ||
|
||
Backends | ||
++++++++ | ||
|
||
|
||
|
||
Static files | ||
++++++++++++ | ||
|
||
With method ``static_filename`` you can request a file from a repository. | ||
The method accepts a relative path (can be a ``pathlib.Path`` or ``str``) to the file in the repository | ||
and it returns an absolute path to the file is currently stored. The method raises ``arca.exceptions.FileOutOfRangeError`` | ||
when the relative path goes outside the scope of the repository and ``FileNotFoundError`` if the file doesn't exist in the | ||
repository. | ||
|
||
Singe pull | ||
++++++++++ | ||
|
||
You might not want the repositories to be pulled everytime ``run`` is called. | ||
You can specify that you want each repository and branch combination should be pulled only once | ||
per initialization of ``Arca`` with the ``ARCA_SINGLE_PULL`` setting or ``single_pull`` keyword argument. | ||
|
||
You can tell ``Arca`` to pull again by calling the method ``pull_again``. | ||
|
||
Caching | ||
+++++++ | ||
|
||
Arca can cache results of the tasks using ``dogpile.cache``. The default cache backend is ``dogpile.cache.null``, so no caching. | ||
You can setup caching with the backend setting ``ARCA_CACHE_BACKEND`` and all the arguments needed for setup can be set | ||
using ``ARCA_CACHE_BACKEND_ARGUMENTS`` which can either be a dict or or a json string. | ||
|
||
Example setup: | ||
|
||
.. code-block:: python | ||
arca = Arca(settings={ | ||
"ARCA_CACHE_BACKEND": "dogpile.cache.redis", | ||
"ARCA_CACHE_BACKEND_ARGUMENTS": { | ||
"host": "localhost", | ||
"port": 6379, | ||
"db": 0, | ||
} | ||
}) | ||
To see all available cache backends and their settings, | ||
please visit the ``dogpile.cache`` `documentation <https://dogpilecache.readthedocs.io/en/latest/>`_. | ||
|
||
When ``Arca`` is being initialized, a check is made if the cache backend is writable and readable, | ||
which raises an ``arca.exceptions.ArcaMisconfigured`` if it's not. | ||
If you wish to ignore this error, set ``ARCA_IGNORE_CACHE_ERRORS`` to True or initialize ``Arca`` with ``ignore_cache_errors`` keyword argument. | ||
|
||
Running tests | ||
************** | ||
|
||
To run tests you'll need the optional requirements, Docker and Vagrant. Once you have them and they can be used by | ||
the current user you just need to run: | ||
|
||
.. code-block:: bash | ||
python setup.py test | ||
This will launch the tests and a PEP8 check. The tests will take some time since building the custom | ||
docker images is also tested and vagrant, in general, takes a long time to set up. | ||
|
||
Contributing | ||
************ | ||
|
||
I am developing this library as my bachelor thesis and will be not accepting any PRs at the moment. | ||
|
||
Links | ||
***** | ||
|
||
- Repository: `GitHub <https://github.com/mikicz/arca>`_ | ||
- PyPi package: `arca <https://pypi.python.org/pypi/arca>`_ | ||
- CI: `Travis <https://travis-ci.org/mikicz/arca>`_ | ||
- Test coverage: `Codecov <https://codecov.io/gh/mikicz/arca>`_ | ||
|
||
License | ||
******* | ||
|
||
This project is licensed under the MIT License - see the `LICENSE <LICENSE>`_ file for details. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters