Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds archive_name argument to push.fs.Zipper #31

Merged
merged 2 commits into from Nov 23, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
@@ -0,0 +1,16 @@
tasks:
- name: zipper
pull:
plugin: pnp.plugins.pull.simple.Cron
args:
expressions:
- "*/1 * * * * /tmp/backup_folder"
push:
plugin: pnp.plugins.push.fs.Zipper
args:
out_path: "{{env::BACKUP_DIR}}"
selector:
archive_name: "lambda payload: '{}_{}'.format(now().isoformat(), 'backup.zip')"
data: "lambda payload: payload"
deps:
plugin: pnp.plugins.push.simple.Echo
28 changes: 21 additions & 7 deletions docs/source/plugins/push/fs.Zipper.rst
Expand Up @@ -29,13 +29,21 @@ the absolute path to the created zip file.

**Arguments**

+----------+------+------+---------+-----+-------------------------------------------------------------------------------------------------------------------------------------------------+
| name | type | opt. | default | env | description |
+==========+======+======+=========+=====+=================================================================================================================================================+
| source | str | yes | n/a | yes | Specifies the source directory or file to zip. If not passed the source can be specified by the envelope at runtime. |
+----------+------+------+---------+-----+-------------------------------------------------------------------------------------------------------------------------------------------------+
| out_path | str | yes | tmp | no | Specifies the path to the general output path where all target zip files should be generated. If not passed the systems temp directory is used. |
+----------+------+------+---------+-----+-------------------------------------------------------------------------------------------------------------------------------------------------+
+--------------+------+------+---------+-----+-------------------------------------------------------------------------------------------------------------------------------------------------+
| name | type | opt. | default | env | description |
+==============+======+======+=========+=====+=================================================================================================================================================+
| source | str | yes | n/a | yes | Specifies the source directory or file to zip. If not passed the source can be specified by the envelope at runtime. |
+--------------+------+------+---------+-----+-------------------------------------------------------------------------------------------------------------------------------------------------+
| out_path | str | yes | tmp | no | Specifies the path to the general output path where all target zip files should be generated. If not passed the systems temp directory is used. |
+--------------+------+------+---------+-----+-------------------------------------------------------------------------------------------------------------------------------------------------+
| archive_name | str | yes | below | yes | Specifies the path to the general output path where all target zip files should be generated. If not passed the systems temp directory is used. |
+--------------+------+------+---------+-----+-------------------------------------------------------------------------------------------------------------------------------------------------+

The default of ``archive_name`` will be either the original file name (if you zip a single file)
resp. the name of the zipped directory (if you zip a directory).
In both cases the extension ``.zip`` will be added.
If you do not want an extension, you have to provide the ``archive_name``.


**Result**

Expand All @@ -45,3 +53,9 @@ Will return an absolute path to the zip file created.

.. literalinclude:: ../code-samples/plugins/push/fs.Zipper/example.yaml
:language: YAML

The next example is useful for dynamically adjusting the archive name to generate
unique names for storing multiple backups:

.. literalinclude:: ../code-samples/plugins/push/fs.Zipper/example_backup.yaml
:language: YAML
20 changes: 16 additions & 4 deletions pnp/plugins/push/fs.py
Expand Up @@ -85,10 +85,11 @@ class Zipper(PushBase):

"""

def __init__(self, source=None, out_path=None, **kwargs):
def __init__(self, source=None, out_path=None, archive_name=None, **kwargs):
super().__init__(**kwargs)
self.source = self._parse_source(source)
self.out_path = self._parse_out_path(out_path)
self.archive_name = self._parse_archive_name(archive_name)

@staticmethod
def _parse_source(value):
Expand All @@ -107,22 +108,33 @@ def _parse_out_path(value):
validator.is_directory(out_path=value)
return os.path.abspath(str(value))

@staticmethod
def _parse_archive_name(value):
return value and str(value)

@enveloped
@parse_envelope('archive_name')
@drop_envelope
def push(self, payload):
def push(self, archive_name, payload): # pylint: disable=arguments-differ
source = self._parse_source(self.source or payload)

if os.path.isdir(source):
# Directory branch
from ...shared.zipping import zipdir, zipignore
base_dir_name = os.path.basename(os.path.normpath(source))
out_file = os.path.join(self.out_path, base_dir_name + '.zip')
out_file = (
os.path.join(self.out_path, archive_name) if archive_name else
os.path.join(self.out_path, base_dir_name + '.zip')
)
ignore_list = zipignore(source)
zipdir(source, out_file, ignore_list)
return out_file

# File branch
from ...shared.zipping import zipfiles
out_file = os.path.join(self.out_path, os.path.basename(source) + '.zip')
out_file = (
os.path.join(self.out_path, archive_name) if archive_name else
os.path.join(self.out_path, os.path.basename(source) + '.zip')
)
zipfiles(source, out_file)
return out_file
18 changes: 18 additions & 0 deletions tests/plugins/push/test_fs_zipper.py
Expand Up @@ -39,3 +39,21 @@ def test_zipper_push_payload_file():
zip_file_name = dut.push(path)
assert zip_file_name == os.path.join(tmpdir, '1' + '.zip')
assert set(ZipFile(zip_file_name).namelist()) == {'1'}


def test_zipper_push_archive_name():
path = os.path.join(os.path.dirname(__file__), '../../resources/zipping/testdir')
with tempfile.TemporaryDirectory() as tmpdir:
dut = Zipper(name='pytest', out_path=tmpdir, archive_name='foo.zip')
zip_file_name = dut.push(path)
assert zip_file_name == os.path.join(tmpdir, 'foo' + '.zip')
assert set(ZipFile(zip_file_name).namelist()) == {'1', '2/2'}


def test_zipper_push_archive_name_override():
path = os.path.join(os.path.dirname(__file__), '../../resources/zipping/testdir')
with tempfile.TemporaryDirectory() as tmpdir:
dut = Zipper(name='pytest', out_path=tmpdir)
zip_file_name = dut.push({'data': path, 'archive_name': 'foo.zip'})
assert zip_file_name == os.path.join(tmpdir, 'foo' + '.zip')
assert set(ZipFile(zip_file_name).namelist()) == {'1', '2/2'}