* Add support for reading and extracting encrypted archives (memory a… #34

udgover · 2019-04-19T15:56:28Z

Hello, this PR adds support to read and extract encrypted archives. Tests were successful for each libarchive.public reader.

Decrypting such archives is as simple as providing passphrase argument to functions.

dsoprea · 2019-04-22T03:18:15Z

That API wasn't added until libarchive 3.1.900 (libarchive/libarchive@6c222e5) and Ubuntu didn't move past 3.1.2 until 3.2.1-2 was backported into Xenial, and, even then, it's in scope but not the default installed version. So, I updated the distribution, specifically installed the correct version of the library, and added a bunch of version and API verbosity in the CI output for quicker triaging in the future: b4f19a0

I also had to drop Python 3.3 support (since it's not available for Xenial), add support for Python 3.6 and 3.7, and fix an encoding issue with some follow-up commits. Now everything should be green.

I need you to rebase your change on top of latest upstream.

…nd file based)

udgover · 2019-04-22T16:51:28Z

Hello,

Thanks for your feedback. I've just rebased my commit.

coveralls · 2019-04-22T16:52:04Z

Pull Request Test Coverage Report for Build 37

18 of 25 (72.0%) changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 70.455%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
libarchive/adapters/archive_read_add_passphrase.py	2	3	66.67%
libarchive/adapters/archive_read.py	6	9	66.67%
libarchive/calls/archive_read_add_passphrase.py	7	10	70.0%

Totals
Change from base Build 36:	0.01%
Covered Lines:	682
Relevant Lines:	968

💛 - Coveralls

coveralls · 2019-04-22T16:52:05Z

Pull Request Test Coverage Report for Build 37

18 of 25 (72.0%) changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 70.455%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
libarchive/adapters/archive_read_add_passphrase.py	2	3	66.67%
libarchive/adapters/archive_read.py	6	9	66.67%
libarchive/calls/archive_read_add_passphrase.py	7	10	70.0%

Totals
Change from base Build 36:	0.01%
Covered Lines:	682
Relevant Lines:	968

💛 - Coveralls

coveralls · 2019-04-22T16:52:05Z

Pull Request Test Coverage Report for Build 37

18 of 25 (72.0%) changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 70.455%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
libarchive/adapters/archive_read_add_passphrase.py	2	3	66.67%
libarchive/adapters/archive_read.py	6	9	66.67%
libarchive/calls/archive_read_add_passphrase.py	7	10	70.0%

Totals
Change from base Build 36:	0.01%
Covered Lines:	682
Relevant Lines:	968

💛 - Coveralls

coveralls · 2019-04-22T16:52:05Z

Pull Request Test Coverage Report for Build 37

18 of 25 (72.0%) changed or added relevant lines in 4 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.01%) to 70.455%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
libarchive/adapters/archive_read_add_passphrase.py	2	3	66.67%
libarchive/adapters/archive_read.py	6	9	66.67%
libarchive/calls/archive_read_add_passphrase.py	7	10	70.0%

Totals
Change from base Build 36:	0.01%
Covered Lines:	682
Relevant Lines:	968

💛 - Coveralls

coveralls · 2019-04-22T16:52:06Z

Pull Request Test Coverage Report for Build 44

54 of 71 (76.06%) changed or added relevant lines in 10 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.6%) to 71.045%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
libarchive/adapters/archive_read_add_passphrase.py	2	3	66.67%
libarchive/adapters/archive_write_set_passphrase.py	2	3	66.67%
libarchive/adapters/archive_read.py	8	11	72.73%
libarchive/calls/archive_read_add_passphrase.py	7	10	70.0%
libarchive/calls/archive_write_set_passphrase.py	7	10	70.0%
libarchive/adapters/archive_write.py	12	18	66.67%

Totals
Change from base Build 36:	0.6%
Covered Lines:	1428
Relevant Lines:	2010

💛 - Coveralls

dsoprea

Please make the changes requested, and then add the interfaces for adding a passphrase while creating/updating, and then add a unit-test that adds a passphrase during the write and then checks it by reading it with a passphrase. Put this test in test_archive_write.py and name it "test_create_file__with_passphrase". Then, duplicate it into test_archive_read.py and name it "test_enumerate_from_file__with_passphrase". Assert that exactly the right filenames are returned.

dsoprea · 2019-04-22T16:59:01Z

libarchive/calls/archive_read_add_passphrase.py

@@ -0,0 +1,14 @@
+from ctypes import *


This file is not necessary. It's already defined as an interface in calls/archive_read.py (where the read interfaces are all defined).

dsoprea · 2019-04-22T17:00:20Z

libarchive/adapters/archive_read_add_passphrase.py

@@ -0,0 +1,5 @@
+import libarchive.calls.archive_read_add_passphrase


This file is not necessary. It's already integrated in adapters/archive_read.py (where higher-level read operations are integrated).

I've just mimicked your original code for other API call. Let me know how to handle it.

dsoprea · 2019-04-22T17:01:03Z

libarchive/adapters/archive_read.py

    """Return an archive enumerator from a user-defined source, using a user-
    defined entry type.
    """

    archive_res = _archive_read_new()

    try:
+        r = _archive_read_add_passphrase(archive_res, passphrase)


Only call this if not None.

At first, I write a condition to check if None but after doing some tests, providing None to the API call does not raise an error.

Couple of comments:

By saying "only call this if not "None", do you really mean if passphrase:?

'r' is a success/failure code, but it isn't checked for failure. But I'm not sure how a failure would be handled here.

The call below this (which was not modified) never returns a value, so this 'r' should be omitted:
r = _set_read_context(archive_res, format_code, filter_code)

dsoprea · 2019-04-22T17:02:15Z

libarchive/adapters/archive_read.py

@@ -49,6 +49,14 @@ def _archive_read_support_format_all(archive):
        message = c_archive_error_string(archive)
        raise libarchive.exception.ArchiveError(message)

+def _archive_read_add_passphrase(archive, passphrase):
+    try:


Encode passphrase to bytes first: passphrase.encode('utf-8')

That will cause Python 3 to error ...

dsoprea · 2019-04-22T17:02:54Z

libarchive/adapters/archive_read.py

@@ -12,7 +12,7 @@
 import libarchive.adapters.archive_read_support_filter_all
 import libarchive.adapters.archive_read_set_format
 import libarchive.adapters.archive_read_support_format_all
-
+import libarchive.adapters.archive_read_add_passphrase


Not necessary. See below.

udgover · 2019-04-23T08:58:00Z

I will add the interfaces for adding a passphrase while creating/updating and provide some tests.

kbalk · 2019-06-27T03:02:55Z

The C libarchive permits multiple passphrases - could you add support for more than one passphrase, i.e., instead of passing a single passphrase, allow for a list of passphrases?

dsoprea · 2019-07-01T03:55:08Z

@kbalk Interesting idea.
@udgover Bump.

udgover · 2019-12-27T13:48:43Z

Hello,

Long time no see, sorry... I've finally got some time to implement write support with archive_write_set_passphrase. It works like a charm and I just need to write tests and I'll submit the PR.

…nd file based) * Add bytes conversion to passphrase * Check if passphrase is not None before calling add_passphrase * Add encryption write support for Zip files (traditional, aes128 and aes256) * Add test for writing with set_passphrase for Zip files

udgover · 2020-05-28T13:59:11Z

I'm about to clean the mess in this PR, add all necessary tests and provide examples.

Currently, I've also added support to provide several passphrases at once. I've replaced passphrase with passphrases in archive_read._enumerator function which then iterate on the provided list of passphrases and call _archive_read_add_passphrase for each element.

I wonder what is the best implementation between completely dropping the possibility to provide only one passphrase as a string and / or providing a list of passphrases.

Both solutions could also coexist but the prototype of the function would be weird:

def _enumerator(opener, entry_cls, passphrase=None, passphrases=None, format_code=None, filter_code=None):
How to handle the case where both are provided? Take into account both? Raise an error?

Thanks for your feedback.

dsoprea · 2020-05-28T18:56:16Z

You could provide a single parameter that accepts both a scalar or a list, but I'm not a fan of that. You could provide a single parameter that accepts an object with a "get_passphrases" method, but that's over-prohramming the solution. Better to always just require a list. That works for 0, 1, or N passphrases. Is the use-case to attempt to match one of the passphrases to the archive? The first match wins?

…

On Thu, May 28, 2020, 09:59 udgover ***@***.***> wrote: I'm about to clean the mess in this PR, add all necessary tests and provide examples. Currently, I've also added support to provide several passphrases at once. I've replaced passphrase with passphrases in archive_read._enumerator function which then iterate on the provided list of passphrases and call _archive_read_add_passphrase for each element. I wonder what is the best implementation between completely dropping the possibility to provide only one passphrase as a string and / or providing a list of passphrases. Both solutions could also coexist but the prototype of the function would be weird: def _enumerator(opener, entry_cls, passphrase=None, passphrases=None, format_code=None, filter_code=None): How to handle the case where both are provided? Take into account both? Raise an error? Thanks for your feedback. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#34 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAFOW2VSAHK36DS5KG7KRH3RTZU37ANCNFSM4HHE4CQQ> .

udgover · 2020-05-29T09:02:35Z

I'm not a fan with the single parameter for which I have to check type, I could implement an object Passphrases with setter and getter but like you, I think it's over-programming. The list solution looks like to be the best one and I've already done this way for testing.

Concerning the list for encrypted archives, the first match wins.

You could provide a single parameter that accepts both a scalar or a list, but I'm not a fan of that. You could provide a single parameter that accepts an object with a "get_passphrases" method, but that's over-prohramming the solution. Better to always just require a list. That works for 0, 1, or N passphrases. Is the use-case to attempt to match one of the passphrases to the archive? The first match wins?

udgover · 2020-06-02T09:48:39Z

Closing this old messy PR to create another one based on latest commits and taking into account comments of this PR.

* Add support for reading and extracting encrypted archives (memory a…

df8faa7

…nd file based)

dsoprea requested changes Apr 22, 2019

View reviewed changes

ION28 mentioned this pull request Jun 26, 2019

Ability to decrypt zip and 7z files Changaco/python-libarchive-c#87

Closed

udgover added 2 commits December 27, 2019 17:37

Merge branch 'master' of github.com:udgover/PyEasyArchive

da4614d

udgover requested a review from dsoprea January 3, 2020 08:38

Resolve merge issue

fca5469

udgover closed this Jun 2, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

* Add support for reading and extracting encrypted archives (memory a… #34

* Add support for reading and extracting encrypted archives (memory a… #34

udgover commented Apr 19, 2019

dsoprea commented Apr 22, 2019 •

edited

udgover commented Apr 22, 2019

coveralls commented Apr 22, 2019

coveralls commented Apr 22, 2019

coveralls commented Apr 22, 2019

coveralls commented Apr 22, 2019

coveralls commented Apr 22, 2019 •

edited

dsoprea left a comment

dsoprea Apr 22, 2019

dsoprea Apr 22, 2019

udgover Apr 23, 2019

dsoprea Apr 22, 2019

udgover Apr 23, 2019

kbalk Jul 2, 2019

dsoprea Apr 22, 2019

kbalk Jul 2, 2019

dsoprea Apr 22, 2019

udgover commented Apr 23, 2019

kbalk commented Jun 27, 2019

dsoprea commented Jul 1, 2019

udgover commented Dec 27, 2019

udgover commented May 28, 2020

dsoprea commented May 28, 2020 via email

udgover commented May 29, 2020

udgover commented Jun 2, 2020

		@@ -0,0 +1,5 @@
		import libarchive.calls.archive_read_add_passphrase

* Add support for reading and extracting encrypted archives (memory a… #34

* Add support for reading and extracting encrypted archives (memory a… #34

Conversation

udgover commented Apr 19, 2019

dsoprea commented Apr 22, 2019 • edited

udgover commented Apr 22, 2019

coveralls commented Apr 22, 2019

Pull Request Test Coverage Report for Build 37

💛 - Coveralls

coveralls commented Apr 22, 2019

Pull Request Test Coverage Report for Build 37

💛 - Coveralls

coveralls commented Apr 22, 2019

Pull Request Test Coverage Report for Build 37

💛 - Coveralls

coveralls commented Apr 22, 2019

Pull Request Test Coverage Report for Build 37

💛 - Coveralls

coveralls commented Apr 22, 2019 • edited

Pull Request Test Coverage Report for Build 44

💛 - Coveralls

dsoprea left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

udgover commented Apr 23, 2019

kbalk commented Jun 27, 2019

dsoprea commented Jul 1, 2019

udgover commented Dec 27, 2019

udgover commented May 28, 2020

dsoprea commented May 28, 2020 via email

udgover commented May 29, 2020

udgover commented Jun 2, 2020

dsoprea commented Apr 22, 2019 •

edited

coveralls commented Apr 22, 2019 •

edited