aptly publishing using Debian signature (debian-archive-keyring)

[erickeller]

Hi Andrey,

When mirroring an existing archive repository, It would be nice to preserve the Debian signature for the packages matching the debian-archive-keyring public key.

currently as explained in the aptly.info

Command publish
Publishing snapshot as Debian repository which could be served by HTTP/FTP/rsync server. Repository is signed by user's key with GnuPG. Key should be created beforehand (see section GPG Keys). Published repository could be consumed directly by apt.

the user has to generate a GPG key for signing the published packages with a new key.

is the option gpgDisableSign in .aptly.conf configuration or --skip-signing intended to use the original Debian signature of the mirror?

[smira]

Eric, it is not possible to sign repository with debian-archive-keyring key (unless you own it ;).

The only option (which is planned for aptly) is when mirroring remote repository directly, publish it "as is", without metadata files generation. That is something I'm planning for.

As for -skip-signing flag: it generates unsigned repository (without signatures).

[erickeller]

Andrey,

thanks for the additional information...

as the option directly publish it "as is" is concerned, this means not snapshot before publishing?
with metadata you mean files dists/wheezy/main/binary-i386/*, and Release, InRelease, ...

regards

[smira]

Eric, snapshotting would work. But it should be directly mirror -> snapshot -> publish. You can't merge, pull or do anything with that snapshot. (But you might have snapshots from different points in time).

Yes, metadata files are: Release, Packages, Sources, ... (everything except package files .deb and package sources).

[erickeller]

Andrey, this sounds good :)

[azizshamim]

I'm running into the same problem - I think. I'm mirroring precise main, restricted, universe, multiverse etc all into one mirror. It sounds like I'll need to separate by component in order to preserve the Release, Packages, Sources etc and not have to resign.

Correct?

[smira]

@azizshamim currently aptly can't mirror without resigning.

[seasnake-dev]

I'll be very interested in the publish as-is option, we currently have mix of debmirror and reprepro to manage our Ubuntu repositories, we could replace both with aptly.
Geoff

[michaeljoy]

Having the ability publish as-is seems like a key missing feature. While update repo's might justify merging, trying to snapshot merge 20+ repo's is just asking for trouble. It's better to keep them separate so you can manage priorities with pinning as not all servers in your environment need the same package priorities and it doesn't make sense to try and maintain a set of merged repo's per flavor of environment. For testing repo changes, yes, that makes sense, but no need to add a duplicate signing process into the middle of it all.

[mld]

Any news on this? I don't currently have any go experience, but if I can get a pointer where to start I'll at least have a look at how to figure this out.

[faunris]

Hello. We need this functionality. Any news on this?

[bymarshall]

Any news on this? this functionality will be very useful for us.. thanks

[nickjanus]

My company is currently using some in house tooling to manage our package mirrors, which my team recently inherited. I'd like to use aptly, but it probably won't be practical without as-is publishing. Any pointers on how I could get started contributing towards this feature @smira?

[smira]

@nickjanus it's not super-simple to implement, as it requires byte-by-byte compatibility in generated files so that signature would still apply. This is not what aptly was supposed to use for, as it does internal heavy de-duplication, it allows to change mirror contents and so on.

So I would say it might be easier to go with other solution if you need unmodified mirrors of some upstream distribution. If you need more (say, filtering or merging), you would still have to re-sign with your keys.

[ealtintas]

@smira: I would like to implement this feature.

it's not super-simple to implement

I think we could cache the metadata of the mirrors locally and then use these cached copies when we need as-is-publising. Is this a good or bad idea? Can you show me the key problems and the pathways to follow to implement this feature?

[smira]

@ealtintas the worst problem is that we need to generate byte-by-byte copies of package indexes, so that Release file checksums match with original. This implies that these index should be just stored, and aptly won't be able to do anything with them (no changes). As aptly relies a lot on normalization of things (e.g. adding missing package files checksums), it means that it would be really hard to combine with other features like snapshots.

I would say that mirroring byte-by-byte is something which could be done much easier outside of aptly scope.

[adam-kosseck]

We're also looking for "as is" mirroring without re-signing. I've spent a lot of time validating Aptly for our use and the biggest sticking point is the re-signing, everything else looks good.

Is there any chance of this being included in the next release?

[goa111]

will be useful indeed

[elfranne]

I ended up using apt-cacher-ng for caching packages and using aptly for hosting homebrewed packages. Or you could use Nexus as it support caching and hosting.

[ZamanOof]

Eric, it is not possible to sign repository with debian-archive-keyring key (unless you own it ;).

The only option (which is planned for aptly) is when mirroring remote repository directly, publish it "as is", without metadata files generation. That is something I'm planning for.

As for -skip-signing flag: it generates unsigned repository (without signatures).

for now is -ignore-signatures (as i think)

[freehck]

@smira
As you closed #844 with the statement that "for pure mirroring other solutions might be better", you could at least consider closing this issue eventually declaring that aptly is an instrument with another purpose (and it actually is). It's widely known that the officially recommended way for pure mirroring is ftpsync.