The repository to operate packages.clickhouse.com
For the repository management the next binaries are used (ubuntu 22.04):
reprepro
version 5.4.1 built from official git reposudo apt-get install build-essential:native libgpgme-dev libdb-dev libbz2-dev liblzma-dev libarchive-dev shunit2:native db-util:native devscripts
dpkg-buildpackage -b --no-sign && sudo dpkg -i ../reprepro_$(dpkg-parsechangelog --show-field Version)_$(dpkg-architecture -q DEB_HOST_ARCH).deb
in the sources directory
createrepo_c
from official ubuntu repositoriesgpg
from official ubuntu repositories
To mount the CloudFlare R2 storage, we use geesefs. Here's the /etc/fstab
mount description:
packages /home/ubuntu/r2 fuse.geesefs _netdev,user_id=1000,group_id=1000,--cheap,--file-mode=0666,--dir-mode=0777,--endpoint=https://account_id.r2.cloudflarestorage.com,--shared-config=/home/ubuntu/.r2_auth,--memory-limit=2050,--gc-interval=100,--max-flushers=2,--max-parallel-parts=3,--max-parallel-copy=2 0 0
The geesefs
binary should be in the $PATH
, --shared-config
has the AWS credentials format
[default]
aws_access_key_id = ***
aws_secret_access_key = ***
The project uses poetry to maintain the dependencies
From the root of the repository:
poetry update
while true; do CHRM_WORKING_DIR=~/ebs/clickhouse-repos-manager poetry run uwsgi --http=[::]:5000 --chdir=clickhouse-repos-manager --module=app --callable=app --http-timeout=10800 --processes=1 --enable-threads; sleep 10; done
The service runs in a single-threaded single-process mode, so the request will be blocked until existing finished.
TODO: a proper systemd.service
All the configuration environment parameters are listed in clickhouse-repos-manager/app_config.py
. Defaults work for ClickHouse/ClickHouse repository.
The ENV parameters starting with DEB_REPO_
are converted from DEB_REPO_PARAMETER_NAME
to ParameterName
and used in reprepro.