Debian apt package & yum package registry

[lunny]

It would be great to support debian apt package registry and yum package registry.

[lunny]

https://github.com/esell/deb-simple has implemented it.

[awkwardbunny]

Hey!

I've had a go at this but it needs a lot of work, primarily on its design.
It was mainly as a proof-of-concept and it would benefit from having a couple of other sets of eyes on this.

https://github.com/awkwardbunny/gitea/tree/feature-apt

I also wanted to discuss some designs, as the way it works right now uses files/directories created in the application data path, which is probably not desirable.

The files are uploaded and deleted through api/packages/<owner>/debian/files/<package>/<version>/<arch>, very similarly to how generic packages are.

The Packages/Releases files are generated in <data>/debian_repo/<owner> and updated after each upload/delete.
The code for generating those are referenced from https://github.com/esell/deb-simple.
For signing, the private key is placed at <data>/debian.gpg and the public key is placed at <data>/debian_public.gpg to be shared.

I wasn't really sure how best to structure this; mainly where to store the generated files as it might be take too much to generate everything each request and didn't want to muck up the database by having it's own table for Debian package repo.

The code may be trash, but I was hoping someone would find it useful if they want to use it for reference or fix it up.

[techknowlogick]

@awkwardbunny this is so amazing! Thank you!

[KN4CK3R]

@awkwardbunny Thank you for your work!

I also wanted to discuss some designs, as the way it works right now uses files/directories created in the application data path, which is probably not desirable.

Yes, the files need to use the existing package storage to benefit from deduplication and all other features. So far I implemented 17 different package registries and they all could be mapped to my schema. This should work for Debian packages too.

For signing, the private key is placed at /debian.gpg and the public key is placed at /debian_public.gpg to be shared.

I did not much research for Debian packages but my thought was to store the signing key(s) as user/system setting or secret. This leads to the next question. Should every user have its own signing key? Are there disadvantages to share a global/system-wide key for multiple user registries? Currently I think one key should be enough.

The code may be trash, but I was hoping someone would find it useful if they want to use it for reference or fix it up.

Not trash at all. I usually need one or two weeks per package type. I spend most of the time researching / reverse engineering how the protocol works and such stuff. Your code is a great starting point. If you don't mind I would adopt your branch and finish it.

[awkwardbunny]

I also wanted to discuss some designs, as the way it works right now uses files/directories created in the application data path, which is probably not desirable.

Yes, the files need to use the existing package storage to benefit from deduplication and all other features. So far I implemented 17 different package registries and they all could be mapped to my schema. This should work for Debian packages too.

The actual packages are stored using the packages service like the other packages types (I used your code for reference).
The files that are created in the application data path are the "Release" and "Packages" files needed for the repository.

The thing that makes it hard/annoying for a Debian repo is that it is split by architecture first, and then name (docs). On top of that, managing versions can also be difficult. In my code, (it's a little hacky) it stores the version and arch as part of the filename (packagename_version_arch.deb) as well as the version in the package registry.

For signing, the private key is placed at /debian.gpg and the public key is placed at /debian_public.gpg to be shared.

I did not much research for Debian packages but my thought was to store the signing key(s) as user/system setting or secret. This leads to the next question. Should every user have its own signing key? Are there disadvantages to share a global/system-wide key for multiple user registries? Currently I think one key should be enough.

I actually did initially add a settings page for signing key. However, I didn't want to muck up the settings, and for simplicity, opted to just read from the key files just for proof-of-concept. I believe there is one signing key for a repo (Debian repo, not git repo). I initially tried to use the user GPG keys but (1) it's per-user keys and (2) only stores public keys, which cannot be used for signing.

This does bring up another point. A single Debian repository is usually split into [distro]/[component/section]/[architecture].
For my code, I just have "gitea" as the distro and a single "main" component hard-coded. If a page in the system settings can be added, maybe these can be configured as well? (The signing key config could also go in there) I've considered having different components or distros per user, but that may be too much.

The code may be trash, but I was hoping someone would find it useful if they want to use it for reference or fix it up.

Not trash at all. I usually need one or two weeks per package type. I spend most of the time researching / reverse engineering how the protocol works and such stuff. Your code is a great starting point. If you don't mind I would adopt your branch and finish it.

Thanks and awesome!

[KN4CK3R]

I added the RPM package registry in #23380.