Add debian metadata and fixes for 0.1.0 release #26

lukpueh · 2019-12-20T16:16:09Z

UPDATED ON 2021-01-25

Fixes issue #:
closes #11

Description of the changes being introduced by the pull request:

Adds debian metadata
Above all, look at debian/apt-transport-in-toto.install, which installs the transport executable, and copies over a default config and a generic in-toto layout. The latter serves as root of trust, and specifies policies such as: any package must be rebuilt by <n> rebuilders, and the corresponding attestations (in-toto link metadata) must be signed with keys <xyz>. This layout is not yet signed.

TODO: Sign data/root.layout with maintainer key and add corresponding gpg keyid to data/intoto.conf. Signing can be done by, e.g.
```
sudo apt install in-toto
wget https://raw.githubusercontent.com/in-toto/apt-transport-in-toto/debian/data/root.layout
in-toto-sign --file root.layout --gpg <YOUR GPG KEYID>
```
Fixes tests to run during Debian package build
- To do so we had to change calls to the python interpreter in tests to python3 because the former is not available during a python3 only build.
- Running tests during the Debian build also requires the test scripts to have a Debian-style shebang, i.e. #!/usr/bin/python3, which unfortunately breaks the virtualenv-based tox/travis builds.
  
  ~~TODO: Fix tox/travis builds~~ * --> fixed in Debian #27
Fixes a minor linter issue and the test coverage configuration
Removes Python 2.7 builds from tox/travis (we don't build for Python 2) and instead adds 3.7 and 3.8
Fixes a BrokenPipeError bug (see 91af874)

~~TODO: The fix seems to work but might need revision (see commit message)~~ --> reviewed in Debian #27 (comment)

See Debian #27 for further updates

The package built and tested with the changes of this PR can be found on:
https://mentors.debian.net/package/apt-transport-in-toto

Please verify and check that the pull request fulfills the following
requirements:

The code follows the Code Style Guidelines
Tests have been added for the bug fix or new feature
Docs have been added for the bug fix or new feature

We only package securesystemslib and in-toto for Python 3. (End of python 2 is soon). See https://www.debian.org/doc/packaging-manuals/python-policy/programs.html#interpreter-directive for debian interpreter directive best practice.

See debian/control for installation of dependencies and debian/apt-transport-in-toto.install for installation of transport script and config files (apt intoto config and layout). Build instructions ------------------ ``` # Assuming sbuild is installed and chroot is created # https://wiki.debian.org/sbuild # Get sources and debian files git clone https://github.com/in-toto/apt-transport-in-toto cd apt-transport-in-toto git checkout debian -- # Build # NOTE: Securessystemslib and in-toto are not yet in debian, you # need to first build them locally. # See instructions at # https://github.com/secure-systems-lab/securesystemslib/commit/9d36a9fb3c1007362eb9c67adf49769968aadbb0,\ # in-toto/in-toto@9656633 # Don't forget to change the path!! sbuild -A -s -d unstable \ --extra-package /PATH/TO/python3-securesystemslib_0.11.3-1_all.deb \ --extra-package /PATH/TO/python3-in-toto_0.3.0-1_all.deb # Sign debian files (replace keyid if necessary) debsign -k 8BA69B87D43BE294F23E812089A2AD3C07D962E8 ``` TODO ------------------- - Sign layout with proper key - Change keyid in apt intoto config file - Distribute key (as part of installation or out of band instructions)?

Drop debian/compat and replace "debhelper (>= 12~)" with "debhelper-compat (12)" in debian/control.

- Fix timewarp-standards-version lintian warning by updating the changelog entry's date. - Fix closing bug number (use ITP instead of RFS).

If we write to the pipe, used to communicate with the apt parent process, after apt has closed it, Python raises a BrokenPipeError. This commit adds exception handling to the write function and a global boolean BROKENPIPE used as exit condition for the reader threads. The commit leaves a few TODO notes that suggest better (more readable) exception handling.

- Move "omit" field from "report" to "run" section where it belongs - Add additional path patterns to "omit" to not include files in coverage that are used while building on Debian.

Since we only build for python3, "python" is not available during build and thus would make those tests fail.

We only build for Python 3. There is no need to test 2.7.

- Update date in debian/changelog entry - Update debian/rules to run tests while building - Update debian/control to: - add missing dependencies - add a new maintainer - update Standards-Version - Add Rules-Requires-Root: No field

Rename "junk" to "_", which pylint accepts to be redeclared.

According to project PyCQA/bandit#333, the bandit checker cannot be aware of trusted input defined from variable which is in out case APT_METHOD_HTTP. We turn off the check for this subprocess call.

Tests are failing due to previously expired layouts

Use latest python versions 3.9 only. We already do that for salsa CI. See for example reprotest CI.

Debian

lukpueh · 2021-01-25T16:43:12Z

Sign data/root.layout with maintainer key and add corresponding gpg keyid to data/intoto.conf.

.... is the only thing that's currently missing here. This means that in-toto reproducibility verification upon apt installation will fail always right away, because the first check in the verification routine is whether the layout is signed by at least one key.

However, users can still change the layout locally, sign it with their own key and specify the keyid in the config file accordingly, e.g. to experiment with other policies, etc.. I suppose that is what @fepitre did for qubes?

h01ger · 2021-01-25T16:51:52Z

I'm here ;)

fepitre · 2021-01-25T16:52:00Z

* Sign data/root.layout with maintainer key and add corresponding gpg keyid to data/intoto.conf.
.... is the only thing that's currently missing here. This means that in-toto reproducibility verification upon apt installation will fail always right away, because the first check in the verification routine is whether the layout is signed by at least one key.

However, users can still change the layout locally, sign it with their own key and specify the keyid in the config file accordingly, e.g. to experiment with other policies, etc.. I suppose that is what @fepitre did for qubes?

Yes this exactly what we did on a simple example. Currently the only way to have some working setup while installing the transport, would be to have Debian providing root.layout with a list of validated rebuilders and have it signed. We plan to do the same for Qubes once we have stable rebuilders.

h01ger · 2021-01-25T16:58:06Z

On Mon, Jan 25, 2021 at 08:52:17AM -0800, Frédéric Pierret wrote: Yes this exactly what we did on a simple example. Currently the only way to have some working setup while installing the transport, would be to have Debian providing `root.layout` with a list of validated rebuilders and have it signed. We plan to do the same for Qubes once we have stable rebuilders.

i'd be delighted if we could make this happen during the bookworm cycle...

…

-- cheers, Holger ⢀⣴⠾⠻⢶⣦⠀ ⣾⠁⢠⠒⠀⣿⡁ holger@(debian|reproducible-builds|layer-acht).org ⢿⡄⠘⠷⠚⠋⠀ PGP fingerprint: B8BF 5413 7B09 D35C F026 FE9D 091A B856 069A AA1C ⠈⠳⣄ Our civilization is being sacrificed for the opportunity of a very small number of people to continue making enormous amounts of money... It is the sufferings of the many which pay for the luxuries of the few... You say you love your children above all else, and yet you are stealing their future in front of their very eyes... (Greta Thunberg)

lukpueh · 2021-01-25T17:00:36Z

On a second thought, I guess I'm also quite fine if we ship out the package with a layout that has no signature. The rebuilders whose keys it currently authorizes are rather experimental (one of them actually seems to be down at the moment). cc @SantiagoTorres

lukpueh · 2021-01-25T17:06:28Z

Just uploaded to mentors: https://mentors.debian.net/package/apt-transport-in-toto/

Apart from a few lintian errors it looks good. I can take care of these tomorrow. Need to sign off for today.

fepitre · 2021-01-25T17:09:14Z

Just uploaded to mentors: https://mentors.debian.net/package/apt-transport-in-toto/

Apart from a few lintian errors it looks good. I can take care of these tomorrow. Need to sign off for today.

Thank you for all. If you need help tomorrow, let me know.

h01ger · 2021-01-25T17:10:02Z

Lukas, please add me to this repo so I can push my signature on data/root.layout :-)

h01ger · 2021-01-25T17:12:37Z

however, with a new data/root.layout it will become a new version... so maybe I'll still upload what's on mentors today (if its good) and then/at the same time you create that new release and I can reupload that...

marmarek · 2021-01-25T17:34:33Z

Can I ask why root.layout needs a signature? Anyone who could modify it (have write access to that file) can also modify apt-transport-in-toto config to trust another key, so it doesn't seem to provide much added value. Am I missing something?

fepitre · 2021-01-25T17:59:33Z

Can I ask why root.layout needs a signature? Anyone who could modify it (have write access to that file) can also modify apt-transport-in-toto config to trust another key, so it doesn't seem to provide much added value. Am I missing something?

I guess it's useful to get it remotely?

lukpueh · 2021-01-26T11:57:11Z

Can I ask why root.layout needs a signature? Anyone who could modify it (have write access to that file) can also modify apt-transport-in-toto config to trust another key, so it doesn't seem to provide much added value. Am I missing something?

Very good question, @marmarek! The layout is the root of trust for in-toto. The underlying bootstrapping issue, that is why you should trust the root of trust, is out of scope for in-toto.

But the benefit of having signatures on the layout is that it limits the users responsibility to "auditing" the keyid(s) (are they trustworthy?) in the config file. If it weren't signed, the user would have to audit the entire layout file with its supply chain policies and rebuilder authorizations, etc.

Moreover, the signatures allow you to separate the trust in the layout from the trust in the package uploader. As @fepitre says, the layout could be shipped out independently, even in a separate package. #13 has some more infos about this.

This is probably something that should be discussed in the README.md and also made available in a man page or so.

lukpueh · 2021-01-26T14:51:16Z

Given that the tip of this PR is what's currently being processed on Debian FTP masters as 0.1.0, I'll just merge here and push a corresponding 0.1.0 tag on that merge commit. We can add lintian fixes and a signed layout in 0.1.1, which @fepitre is already preparing in #29. Let's continue there.

lukpueh and others added 22 commits June 7, 2019 19:17

Update intoto.py shebang to use python3

35f55d7

We only package securesystemslib and in-toto for Python 3. (End of python 2 is soon). See https://www.debian.org/doc/packaging-manuals/python-policy/programs.html#interpreter-directive for debian interpreter directive best practice.

Add basic default config for installation

a1aed54

Add required ITP bugfix remark to debian changelog

4f5c833

Bump debian standards version to 4.4.0

c844457

Update debian compat specification

7183c59

Drop debian/compat and replace "debhelper (>= 12~)" with "debhelper-compat (12)" in debian/control.

Fix misc debian/changelog

6f83723

- Fix timewarp-standards-version lintian warning by updating the changelog entry's date. - Fix closing bug number (use ITP instead of RFS).

Fix .coveragerc

6c06485

- Move "omit" field from "report" to "run" section where it belongs - Add additional path patterns to "omit" to not include files in coverage that are used while building on Debian.

Fix python3 shebang

1860743

Use "python3" to call interpreter in tests

2b8d4a5

Since we only build for python3, "python" is not available during build and thus would make those tests fail.

Remove Python 2.7 from tests (and add 3.8)

3c846aa

We only build for Python 3. There is no need to test 2.7.

Update debian metadata for 0.1.0 release

b893f10

- Update date in debian/changelog entry - Update debian/rules to run tests while building - Update debian/control to: - add missing dependencies - add a new maintainer - update Standards-Version - Add Rules-Requires-Root: No field

Fix redeclared-assigned-name linter warning

fc6ff13

Rename "junk" to "_", which pylint accepts to be redeclared.

Update .gitignore

f6f621d

README: fix symbolic link command

dc5b8f9

Drop python2 support

9689818

intoto.py: replace deprecated in_toto.util

95384cc

intoto.py: fix missing in_toto.exceptions import

4415a8d

Fix indentation for with respect to pylint rules

2a7131d

intoto.py: make python bandit happy

6c74b05

According to project PyCQA/bandit#333, the bandit checker cannot be aware of trusted input defined from variable which is in out case APT_METHOD_HTTP. We turn off the check for this subprocess call.

tests: update expire test layouts

8fda27e

Tests are failing due to previously expired layouts

lukpueh mentioned this pull request Jan 25, 2021

Debian #27

Merged

fepitre and others added 4 commits January 25, 2021 12:41

debian: update debhelper-compat and Standards-Version

f1dafa0

debian: add python3-securesystemslib as dependency

e6fcb9e

Update tox.ini and .travis-ci.yml

f99a52f

Use latest python versions 3.9 only. We already do that for salsa CI. See for example reprotest CI.

Merge pull request #27 from fepitre/debian

57c31a3

Debian

lukpueh changed the title ~~Add debian metadata and fixes for 0.1.0 release (WIP)~~ Add debian metadata and fixes for 0.1.0 release Jan 26, 2021

lukpueh merged commit 904332d into develop Jan 26, 2021

This was referenced Jan 26, 2021

Prepare downstream debian release 0.1.1-1 #29

Merged

Explore possibility of splitting current /sources into /source and /binary #28

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add debian metadata and fixes for 0.1.0 release #26

Add debian metadata and fixes for 0.1.0 release #26

lukpueh commented Dec 20, 2019 •

edited

Loading

lukpueh commented Jan 25, 2021

h01ger commented Jan 25, 2021

fepitre commented Jan 25, 2021

h01ger commented Jan 25, 2021 via email

lukpueh commented Jan 25, 2021

lukpueh commented Jan 25, 2021

fepitre commented Jan 25, 2021

h01ger commented Jan 25, 2021

h01ger commented Jan 25, 2021

marmarek commented Jan 25, 2021

fepitre commented Jan 25, 2021

lukpueh commented Jan 26, 2021

lukpueh commented Jan 26, 2021

Add debian metadata and fixes for 0.1.0 release #26

Add debian metadata and fixes for 0.1.0 release #26

Conversation

lukpueh commented Dec 20, 2019 • edited Loading

lukpueh commented Jan 25, 2021

h01ger commented Jan 25, 2021

fepitre commented Jan 25, 2021

h01ger commented Jan 25, 2021 via email

lukpueh commented Jan 25, 2021

lukpueh commented Jan 25, 2021

fepitre commented Jan 25, 2021

h01ger commented Jan 25, 2021

h01ger commented Jan 25, 2021

marmarek commented Jan 25, 2021

fepitre commented Jan 25, 2021

lukpueh commented Jan 26, 2021

lukpueh commented Jan 26, 2021

lukpueh commented Dec 20, 2019 •

edited

Loading