pristine-tar: handle orig tar signature (.asc)

[amurzeau]

This require pristine-tar version 1.41.

When commiting a orig tarball, check the existence of a .asc signature.
If present, add it to the pristine-tar commit.

When checkouting a orig tarball, check the presence of a asc signature
in the pristine-tar banch. If present, checkout it along with the tarball.

Closes: #872864

[amurzeau]

Additional notes:
When doing import-orig with existing .asc signature and a additional tarball for component test, the output is:

gbp:debug: Creating pristine tar commit 'test' from '83453483e8f93309056711caac2e7e655609ef7c'
gbp:debug: pristine-tar [] ['--signature-file', '../streamlink_0.8.2.orig-test.tar.gz.asc', 'commit', '../streamlink_0.8.2.orig-test.tar.gz', '83453483e8f93309056711caac2e7e655609ef7c']
gbp:debug: pristine-tar [] ['--signature-file', '../streamlink_0.8.2.orig.tar.gz.asc', 'commit', '../streamlink_0.8.2.orig.tar.gz', '03d9cd2d0bfe308ede4192c32e701fe5f3cd8850']

When doing buildpackage:

gbp:debug: ['git', 'ls-tree', '-z', 'pristine-tar', '--', 'streamlink_0.8.2.orig.tar.gz.asc']
gbp:debug: pristine-tar [] ['--signature-file', '/home/doc/Projets_C-linux/streamlink/test-pristine/build-dir/streamlink_0.8.2.orig.tar.gz.asc', 'checkout', '/home/doc/Projets_C-linux/streamlink/test-pristine/build-dir/streamlink_0.8.2.orig.tar.gz']
gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
gbp:info: Creating /home/doc/Projets_C-linux/streamlink/test-pristine/build-dir/streamlink_0.8.2.orig-test.tar.gz
gbp:debug: ['git', 'show', '--pretty=medium', 'HEAD:debian/source/format']
gbp:debug: ['git', 'ls-tree', '-z', 'pristine-tar', '--', 'streamlink_0.8.2.orig-test.tar.gz.asc']
gbp:debug: pristine-tar [] ['--signature-file', '/home/doc/Projets_C-linux/streamlink/test-pristine/build-dir/streamlink_0.8.2.orig-test.tar.gz.asc', 'checkout', '/home/doc/Projets_C-linux/streamlink/test-pristine/build-dir/streamlink_0.8.2.orig-test.tar.gz']

And the built source package does contains both .asc files:

Files:
 7ab60af051dbe405372be018267f2ff5 2149 python optional streamlink_0.8.2-1.dsc
 09b4e5e7044a4041c7a4967a92a690af 136 python optional streamlink_0.8.2.orig-test.tar.gz
 fa76f8e121d6fad90348b778802e5f98 1097 python optional streamlink_0.8.2.orig-test.tar.gz.asc
 615fa927975465e6aa6b44693859d753 548977 python optional streamlink_0.8.2.orig.tar.gz
 d821e2b4b9d426caa2ce4663f3ca7381 819 python optional streamlink_0.8.2.orig.tar.gz.asc
 eed4e227167a9bdc470bc82cded17ef4 11200 python optional streamlink_0.8.2-1.debian.tar.xz
 f2dbc07397595d51924d3ab382072bf5 6335 python optional streamlink_0.8.2-1_source.buildinfo

I have done this test using an "export-dir".

[amurzeau]

I have a few question a bit related to the test failure.
As of now, /usr/bin/pristine-tar will fail if we tell it to commit / checkout a non existing signature.
So when calling pristine-tar, we need to check if the signature exists or not.

For the commit operation, this is a simple os.path.isfile check.
But for the checkout operation, we need to either:

1. Check if the signature is in the pristine-tar branch.
   • In that case, we need to replicate pristine-tar behavior to keep remote pristine-tar branch supported
   • This is what this PR does, using git ls-tree. But it currently only works with a local pristine-tar branch.
2. Always try to checkout with pristine-tar --signature-file checkout.
   • If this fails, retry without the signature
   • This has the downside of doing an useless try when the package doesn't use upstream signatures.
3. Add a new configuration option to enable signature storage with pristine-tar
   • So we checkout with pristine-tar --signature-file checkout only if that option is enabled.
   • Like 2., if there is no signature, it will fail and we catch that failure to retry a checkout without the signature (this way we still handle checkout of tarball without signature)
4. Ask pristine-tar team to have a way to not fail when doing pristine-tar --signature-file checkout with a non existing signature.
   • If there is no signature inside the pristine-tar branch, it would only extract the tarball without failing.

When this will be sorted out, I adjust/add tests for a more complete PR:

• pristine-tar commit with and without a signature,
• pristine-tar checkout with and without a signature,
• pristine-tar checkout with only a remote branch, unless we don't need this to be supported.

[geor-g]

Any progress on this?

[agx]

I have branch that cleans up the additional tarball processing so that it gets folded into UpstreamSource making it possible to pass around the necessary data. The work is stalled due to -ENOTIME since end of last October though. I'll look into cleaning this up so that others can work from there.

[geor-g]

@agx I would like to see this implemented, and offering help, hereby. Thanks for your work, highly appreciated!

[agx]

@ge-fa I'll try to cleanup what I have and push that out so you can base your work on top. I'm very busy real life stuff atm so it might be a couple of days. Please keep pinging should I not get to it.

[geor-g]

@agx Ping.. :)

[geor-g]

@agx Ping.. :)

[agx]

Hi,

On Tue, Oct 02, 2018 at 10:46:47AM +0000, ge-fa wrote: ***@***.*** Ping.. :)

Sorry for the drag. It's my INBOX still but keeps slipping. I hope to get around to it next week. -- Guido

[agx]

@ge-fa I've cleaned up the mergable stuff and pushed it to master and moved all the rest to https://github.com/agx/git-buildpackage/tree/wip/sig-support . See https://github.com/agx/git-buildpackage/blob/wip/sig-support/TODO . Once the extended UpstreamSource and DebianAdditionalTarball are used everywhere most things will fall into place automatically.

[agx]

we're handling signatures, closing.