digoal
2024-02-27
PostgreSQL , PolarDB , DuckDB , HaloDB , mysql , databend , mssql , oracle , docker , 插件 , Dockerfile , docker build
PostgreSQL 14 版本 on Debian 11.x:
PostgreSQL 15 + PolarDB + DuckDB + HaloDB + MySQL + Databend + mssql + Oracle + 插件 + 工具
# 拉取镜像, 第一次拉取一次即可. 或者需要的时候执行, 将更新到最新镜像版本.
docker pull registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:pg15_with_exts_arm64
# 启动容器
docker run -d -it -P --cap-add=SYS_PTRACE --cap-add SYS_ADMIN --privileged=true --name pg --shm-size=1g registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:pg15_with_exts_arm64
##### 如果你想学习备份恢复、修改参数等需要重启数据库实例的case, 换个启动参数, 使用参数--entrypoint将容器根进程换成bash更好. 如下:
docker run -d -it -P --cap-add=SYS_PTRACE --cap-add SYS_ADMIN --privileged=true --name pg --shm-size=1g --entrypoint /bin/bash registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:pg15_with_exts_arm64
##### 以上启动方式需要进入容器后手工启动数据库实例: su - postgres; pg_ctl start;
# 进入容器
docker exec -ti pg bash
# 连接数据库
psql
插件列表:
- 与amd64 image集成的插件一样. 请参考 《PostgreSQL 14 Docker镜像学习环境 AMD64版》
使用duckdb:
# 进入容器
docker exec -ti pg bash
# 切换用户
su - postgres
# 启动duckdb
./duckdb
1、创建Dockerfile工作目录:
mkdir ~/pg15
2、可选. 可以直接apt-get 安装oracle-libs 包.
下载Oracle OCI到Dockerfile工作目录. (oracle_fdw, dblink_plus插件依赖oracle library.)
《PostgreSQL 商用版本EPAS(阿里云ppas) - 测试环境部署(EPAS 安装、配置、管理、Oracle DBLINK、外表)》
http://www.oracle.com/technetwork/database/features/instant-client/index-097480.html
https://www.oracle.com/database/technologies/instant-client/linux-arm-aarch64-downloads.html
cd ~/pg15
unzip instantclient-basic-linux.arm64-19.19.0.0.0dbru.zip
unzip instantclient-sdk-linux.arm64-19.19.0.0.0dbru.zip
2.1、下载几个较大的源码包, 降低在docker build时下载报错导致build失败的概率.
cd ~/pg15
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L http://faculty.cse.tamu.edu/davis/GraphBLAS/GraphBLAS-3.1.1.tar.gz -o GraphBLAS-3.1.1.tar.gz
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://api.pgxn.org/dist/datasketches/1.7.0/datasketches-1.7.0.zip -o datasketches-1.7.0.zip
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/duckdb/duckdb/releases/download/v0.9.2/libduckdb-linux-aarch64.zip -o libduckdb-linux-aarch64.zip
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/duckdb/duckdb/releases/download/v0.9.2/libduckdb-src.zip -o libduckdb-src.zip
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/duckdb/duckdb/releases/download/v0.9.2/duckdb_cli-linux-aarch64.zip -o duckdb_cli-linux-aarch64.zip
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/Kitware/CMake/releases/download/v3.27.4/cmake-3.27.4.tar.gz -o cmake-3.27.4.tar.gz
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/Kitware/CMake/releases/download/v3.27.9/cmake-3.27.9-linux-aarch64.tar.gz -o cmake-3.27.9-linux-aarch64.tar.gz
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/mongodb/mongo-c-driver/releases/download/1.17.3/mongo-c-driver-1.17.3.tar.gz -o mongo-c-driver-1.17.3.tar.gz
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/json-c/json-c/archive/json-c-0.15-20200726.tar.gz -o json-c-0.15-20200726.tar.gz
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/microsoft/onnxruntime/releases/download/v1.15.1/onnxruntime-linux-aarch64-1.15.1.tgz -o onnxruntime-linux-aarch64-1.15.1.tgz
curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://nodejs.org/download/release/v14.21.3/node-v14.21.3.tar.gz -o node-v14.21.3.tar.gz
pg_task需要修改postgres.c, 很容易下载失败, 预编译好, 防止制作docker image失败.
cd pg15
docker cp pg:/usr/lib/postgresql/15/lib/pg_task.so ./
docker cp pg:/usr/lib/postgresql/15/lib/bitcode/pg_task ./
docker cp pg:/usr/lib/postgresql/15/lib/bitcode/pg_task.index.bc ./
docker cp pg:/usr/share/postgresql/15/extension/pg_task.control ./
docker cp pg:/usr/share/postgresql/15/extension/pg_task--1.0.sql ./
参考文档打包pg_tiktoken:
参考文档打包lantern_extras:
参考文档打包pg_bm25, pg_search, svector:
参考文档打包pg_idkit:
参考文档打包 PgDD:
参考文档打包 pgmq, pg_later, pg_vectorize :
3、准备脚本, 参考末尾Max depth exceeded报错, 目的是减少dockerfile步骤. 但是写成一堆脚本不太好调试, 建议先手工制作后再用Dockerfile来制作.
cd ~/pg15
vi 1.sh
#!/bin/bash
set -vx
cd /tmp
# sed -i "s@http://\(deb\|security\).debian.org@http://mirrors.aliyun.com@g" /etc/apt/sources.list
mv /etc/apt/sources.list.d/debian.sources /etc/apt/sources.list.d/debian.sources.bak
echo "
# 默认注释了源码镜像以提高 apt update 速度，如有需要可自行取消注释
deb http://mirrors.aliyun.com/debian/ bookworm main contrib non-free non-free-firmware
# deb-src http://mirrors.aliyun.com/debian/ bookworm main contrib non-free non-free-firmware
deb http://mirrors.aliyun.com/debian/ bookworm-updates main contrib non-free non-free-firmware
# deb-src http://mirrors.aliyun.com/debian/ bookworm-updates main contrib non-free non-free-firmware
deb http://mirrors.aliyun.com/debian/ bookworm-backports main contrib non-free non-free-firmware
# deb-src http://mirrors.aliyun.com/debian/ bookworm-backports main contrib non-free non-free-firmware
deb http://mirrors.aliyun.com/debian-security bookworm-security main contrib non-free non-free-firmware
# deb-src http://mirrors.aliyun.com/debian-security bookworm-security main contrib non-free non-free-firmware
" > /etc/apt/sources.list
apt-get update
apt-get reinstall -y apt-transport-https ca-certificates
# sed -i "s@http://mirrors.aliyun.com@https://mirrors.aliyun.com@g" /etc/apt/sources.list
echo "
# 默认注释了源码镜像以提高 apt update 速度，如有需要可自行取消注释
deb https://mirrors.aliyun.com/debian/ bookworm main contrib non-free non-free-firmware
# deb-src https://mirrors.aliyun.com/debian/ bookworm main contrib non-free non-free-firmware
deb https://mirrors.aliyun.com/debian/ bookworm-updates main contrib non-free non-free-firmware
# deb-src https://mirrors.aliyun.com/debian/ bookworm-updates main contrib non-free non-free-firmware
deb https://mirrors.aliyun.com/debian/ bookworm-backports main contrib non-free non-free-firmware
# deb-src https://mirrors.aliyun.com/debian/ bookworm-backports main contrib non-free non-free-firmware
deb https://mirrors.aliyun.com/debian-security bookworm-security main contrib non-free non-free-firmware
# deb-src https://mirrors.aliyun.com/debian-security bookworm-security main contrib non-free non-free-firmware
" > /etc/apt/sources.list
apt-get update
apt-get install -y lsb-release wget vim man
# RUN echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list
echo "deb https://mirrors.tuna.tsinghua.edu.cn/postgresql/repos/apt/ $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list
apt-get install -y gnupg2
# RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmour -o /etc/apt/trusted.gpg.d/postgresql.gpg
wget --quiet -O - https://mirrors.tuna.tsinghua.edu.cn/postgresql/repos/apt/ACCC4CF8.asc | gpg --dearmour -o /etc/apt/trusted.gpg.d/postgresql.gpg
apt-get update
apt-get install -y locales
localedef -i en_US -f UTF-8 en_US.UTF-8
apt-get install -y curl libicu-dev icu-devtools libbison-dev libfl-dev git libreadline-dev libedit-dev g++ make cmake man-db dnsutils clang libssl-dev default-jdk strace gdb libsqlite3-dev lsof blktrace
apt-get install -y unixodbc unixodbc-dev bash-completion m4 python3-distutils python glibc-source zlib1g-dev pkg-config default-jre openjdk-17-jdk openjdk-17-jdk-headless
echo "A" | apt-get install -y -q oracle-libs
apt-get install -y postgresql-15 postgresql-client-15 postgresql-server-dev-15 postgresql-doc-15
apt-get install -y postgresql-15-dirtyread postgresql-15-extra-window-functions postgresql-15-first-last-agg postgresql-15-hll postgresql-15-hypopg
apt-get install -y postgresql-15-ip4r postgresql-15-mysql-fdw postgresql-15-jsquery postgresql-15-ogr-fdw postgresql-15-oracle-fdw postgresql-15-pgmemcache
apt-get install -y postgresql-15-pljava postgresql-15-pllua postgresql-15-plpgsql-check postgresql-15-plproxy postgresql-15-prefix postgresql-15-rational
apt-get install -y postgresql-15-rdkit postgresql-15-orafce postgresql-15-pg-qualstats postgresql-15-pg-stat-kcache
apt-get install -y postgresql-15-pg-wait-sampling postgresql-15-pgfincore postgresql-15-pgaudit postgresql-15-pgpool2 postgresql-15-pgrouting postgresql-15-pgrouting-doc
apt-get install -y postgresql-15-pgrouting-scripts postgresql-15-pgsphere postgresql-15-pgvector postgresql-15-pldebugger postgresql-15-pointcloud postgresql-15-plr
apt-get install -y postgresql-15-postgis-3 postgresql-15-postgis-3-scripts postgresql-15-powa powa-collector postgresql-15-q3c postgresql-15-repack
apt-get install -y postgresql-15-rum postgresql-15-show-plans postgresql-15-similarity postgresql-15-tablelog postgresql-15-tdigest postgresql-15-wal2json
apt-get install -y postgresql-15-tds-fdw postgresql-15-plprofiler postgresql-15-cron
apt-get install -y pgagroal pgpool2 pgbouncer pgxnclient pgagent postgresql-plpython3-15 postgresql-15-icu-ext libpq-dev pgreplay pgbackrest pgbackrest-doc elephant-shed-pgbackrest
echo "deb https://packagecloud.io/timescale/timescaledb/debian/ $(lsb_release -c -s) main" | tee /etc/apt/sources.list.d/timescaledb.list
wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | gpg --dearmor -o /etc/apt/trusted.gpg.d/timescaledb.gpg
apt-get update
apt-get install -y timescaledb-2-postgresql-15 timescaledb-toolkit-postgresql-15 timescaledb-2-loader-postgresql-15
wget https://packages.groonga.org/debian/groonga-apt-source-latest-bullseye.deb
apt-get install -y -V ./groonga-apt-source-latest-bullseye.deb
apt-get update
apt-get install -y postgresql-15-pgdg-pgroonga
apt-get install -y postgresql-15-credcheck postgresql-15-decoderbufs postgresql-15-mimeo postgresql-15-pgmp postgresql-15-preprepare postgresql-15-prioritize postgresql-15-squeeze postgresql-15-toastinfo postgresql-15-unit pgbadger pg-auto-failover-cli postgresql-15-auto-failover net-tools apt-utils ora2pg pgloader postgresql-15-partman pg-activity
# 包安装方式暂不支持ARM架构
# https://dev.mysql.com/downloads/repo/apt/
# cd /tmp
# wget -T 36000 -t 0 --waitretry=5 https://repo.mysql.com//mysql-apt-config_0.8.28-1_all.deb
# dpkg -i mysql-apt-config_0.8.28-1_all.deb
# apt-get update
# apt-get install -y mysql-server
# proxysql, High Performance Advanced Proxy for MySQL
# https://proxysql.com/documentation/installing-proxysql/
# cd /tmp
# dpkg -i proxysql_2.5.5-debian11_arm64.deb
vi 2.sh
#!/bin/bash
set -vx
export ROOT_HOME=/root
echo "# add by digoal" >>$ROOT_HOME/.bashrc
echo "alias rm='rm -i'" >>$ROOT_HOME/.bashrc
echo "alias cp='cp -i'" >>$ROOT_HOME/.bashrc
echo "alias ll='ls -larth'" >>$ROOT_HOME/.bashrc
echo "alias mv='mv -i'" >>$ROOT_HOME/.bashrc
echo "export PGHOME=/usr/lib/postgresql/15" >>$ROOT_HOME/.bashrc
echo "export PATH=\$PGHOME/bin:\$PATH" >>$ROOT_HOME/.bashrc
echo "export LD_LIBRARY_PATH=\$PGHOME/lib:/usr/local/lib:/usr/lib/aarch64-linux-gnu:\$LD_LIBRARY_PATH" >>$ROOT_HOME/.bashrc
echo "export PGDATA=/var/lib/postgresql/15/pgdata" >>$ROOT_HOME/.bashrc
echo "export PGUSER=postgres" >>$ROOT_HOME/.bashrc
echo "export PGHOST=\$PGDATA" >>$ROOT_HOME/.bashrc
echo "export PGPORT=1921" >>$ROOT_HOME/.bashrc
echo "export PGDATABASE=postgres" >>$ROOT_HOME/.bashrc
echo "export LC_ALL=en_US.UTF-8" >>$ROOT_HOME/.bashrc
echo "export PATH=/tmp/cmake-3.27.9/bin:\$PATH" >>$ROOT_HOME/.bashrc
echo "export MANPATH=/tmp/cmake-3.27.9/man:\$MANPATH" >>$ROOT_HOME/.bashrc
. $ROOT_HOME/.bashrc
export PG_HOME=/var/lib/postgresql
echo "# add by digoal" >>$PG_HOME/.bash_profile
echo "alias rm='rm -i'" >>$PG_HOME/.bash_profile
echo "alias cp='cp -i'" >>$PG_HOME/.bash_profile
echo "alias ll='ls -larth'" >>$PG_HOME/.bash_profile
echo "alias mv='mv -i'" >>$PG_HOME/.bash_profile
echo "export PGHOME=/usr/lib/postgresql/15" >>$PG_HOME/.bash_profile
echo "export PATH=\$PGHOME/bin:\$PATH" >>$PG_HOME/.bash_profile
echo "export LD_LIBRARY_PATH=\$PGHOME/lib:/usr/local/lib:/usr/lib/aarch64-linux-gnu:\$LD_LIBRARY_PATH" >>$PG_HOME/.bash_profile
echo "export PGDATA=/var/lib/postgresql/15/pgdata" >>$PG_HOME/.bash_profile
echo "export PGUSER=postgres" >>$PG_HOME/.bash_profile
echo "export PGHOST=\$PGDATA" >>$PG_HOME/.bash_profile
echo "export PGPORT=1921" >>$PG_HOME/.bash_profile
echo "export PGDATABASE=postgres" >>$PG_HOME/.bash_profile
echo "export LC_ALL=en_US.UTF-8" >>$PG_HOME/.bash_profile
echo "export PATH=/tmp/cmake-3.27.9/bin:\$PATH" >>$PG_HOME/.bash_profile
echo "export MANPATH=/tmp/cmake-3.27.9/man:\$MANPATH" >>$PG_HOME/.bash_profile
echo ". ~/.bash_profile" > $PG_HOME/.profile
chown postgres:postgres $PG_HOME/.bash_profile
chown postgres:postgres $PG_HOME/.profile
vi 3.sh
#!/bin/bash
set -vx
export ROOT_HOME=/root
. $ROOT_HOME/.bashrc
export TEMP_DIR=/tmp
cd $TEMP_DIR
git clone --depth 1 https://github.com/jaiminpan/pg_jieba
cd $TEMP_DIR/pg_jieba
git submodule update --init --recursive --depth 1
mkdir build
cd $TEMP_DIR/pg_jieba/build
cmake -DPostgreSQL_TYPE_INCLUDE_DIR=/usr/include/postgresql/15/server ..
make && make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/ChenHuajun/pg_roaringbitmap
cd $TEMP_DIR/pg_roaringbitmap
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/theirix/parray_gin
cd $TEMP_DIR/parray_gin
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/jirutka/smlar
cd $TEMP_DIR/smlar
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/alitrack/duckdb_fdw
# 15版本 bug fix 之前
# git clone https://github.com/alitrack/duckdb_fdw
cd $TEMP_DIR/duckdb_fdw
# git checkout 87c66cf04243c7bc43e0e75d0b8ce5dd76d81cd5
# wget -T 36000 -t 0 --waitretry=5 https://github.com/duckdb/duckdb/releases/download/v0.9.2/libduckdb-linux-aarch64.zip
# curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/duckdb/duckdb/releases/download/v0.9.2/libduckdb-linux-aarch64.zip -o libduckdb-linux-aarch64.zip
# cp $TEMP_DIR/libduckdb-linux-aarch64.zip $TEMP_DIR/duckdb_fdw/
# unzip -n -d . libduckdb-linux-aarch64.zip
# cp libduckdb.so $(pg_config --libdir)
# USE_PGXS=1 make
# USE_PGXS=1 make install
mkdir /tmp/duckdb_fdw/libduckdb
mv /tmp/libduckdb-src.zip /tmp/duckdb_fdw/libduckdb/
cd /tmp/duckdb_fdw/libduckdb
unzip libduckdb-src.zip
unalias cp
cp -f duckdb.h ../
cp -f duckdb.hpp ../
# 编译 libduckdb.so 可能比较费内存, 建设调大docker 内存限制到8GB以上
# 编译参数需要 -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0 与 duckdb_fdw src Makefile 一致
clang++ -c -fPIC -std=c++11 -D_GLIBCXX_USE_CXX11_ABI=0 duckdb.cpp -o duckdb.o
clang++ -shared -o libduckdb.so *.o
cp -f libduckdb.so $(pg_config --libdir)
cp -f libduckdb.so /tmp/duckdb_fdw/
cd /tmp/duckdb_fdw
USE_PGXS=1 make uninstall
USE_PGXS=1 make clean
USE_PGXS=1 make distclean
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/EnterpriseDB/hdfs_fdw
cd $TEMP_DIR/hdfs_fdw/libhive
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64 JDK_INCLUDE=$JAVA_HOME/include INSTALL_DIR=/usr/lib/postgresql/15/lib PATH=/usr/lib/postgresql/15/bin:$PATH make
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64 JDK_INCLUDE=$JAVA_HOME/include INSTALL_DIR=/usr/lib/postgresql/15/lib PATH=/usr/lib/postgresql/15/bin:$PATH make install
cd $TEMP_DIR/hdfs_fdw/libhive/jdbc
javac MsgBuf.java
javac HiveJdbcClient.java
jar cf HiveJdbcClient-1.0.jar *.class
cp HiveJdbcClient-1.0.jar /usr/lib/postgresql/15/lib
cd $TEMP_DIR/hdfs_fdw
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64 JDK_INCLUDE=$JAVA_HOME/include INSTALL_DIR=/usr/lib/postgresql/15/lib PATH=/usr/lib/postgresql/15/bin:$PATH USE_PGXS=1 make
JAVA_HOME=/usr/lib/jvm/java-17-openjdk-arm64 JDK_INCLUDE=$JAVA_HOME/include INSTALL_DIR=/usr/lib/postgresql/15/lib PATH=/usr/lib/postgresql/15/bin:$PATH USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://gitlab.com/dalibo/postgresql_anonymizer.git
cd $TEMP_DIR/postgresql_anonymizer
make extension
make install
cd $TEMP_DIR
git clone --depth 1 --branch stable https://github.com/jedisct1/libsodium
cd $TEMP_DIR/libsodium
./configure
make
make check
make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/michelp/pgsodium
cd $TEMP_DIR/pgsodium
C_INCLUDE_PATH=/usr/include/postgresql/15/server PG_LDFLAGS=-L/usr/lib/postgresql/15/lib USE_PGXS=1 make
C_INCLUDE_PATH=/usr/include/postgresql/15/server PG_LDFLAGS=-L/usr/lib/postgresql/15/lib USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://github.com/libgd/libgd/archive/refs/tags/gd-2.3.3.tar.gz
tar -zxvf gd-2.3.3.tar.gz
cd $TEMP_DIR/libgd-gd-2.3.3
mkdir build
cd $TEMP_DIR/libgd-gd-2.3.3/build
cmake ..
make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/digoal/imgsmlr
# git clone --depth 1 https://github.com/postgrespro/imgsmlr
cd $TEMP_DIR/imgsmlr
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/postgrespro/vops
cd $TEMP_DIR/vops
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b pg15 https://github.com/ossc-db/pg_hint_plan
cd $TEMP_DIR/pg_hint_plan
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
apt-get install -y libselinux1-dev libpam0g-dev libkrb5-dev liblz4-dev
git clone --depth 1 https://github.com/ossc-db/pg_bulkload
cd $TEMP_DIR/pg_bulkload
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b 1.6.1 https://github.com/ossc-db/pg_store_plans
cd $TEMP_DIR/pg_store_plans
USE_PGXS=1 make
USE_PGXS=1 make install
# cd $TEMP_DIR
# git clone --depth 1 -b REL-5_5_1 https://github.com/EnterpriseDB/mongo_fdw
# cd $TEMP_DIR/mongo_fdw
# ./autogen.sh --with-master
# apt-get install -y libmongoc-dev
# # C_INCLUDE_PATH="/include/libmongoc-1.0/mongoc:/include/libbson-1.0" USE_PGXS=1 make
# # C_INCLUDE_PATH="/include/libmongoc-1.0/mongoc:/include/libbson-1.0" USE_PGXS=1 make install
# USE_PGXS=1 make
# USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b REL-5_5_1 https://github.com/EnterpriseDB/mongo_fdw
cd $TEMP_DIR/mongo_fdw
cp $TEMP_DIR/mongo-c-driver-1.17.3.tar.gz ./
tar -xzvf mongo-c-driver-1.17.3.tar.gz
rm -rf mongo-c-driver
mv mongo-c-driver-1.17.3 mongo-c-driver
cd $TEMP_DIR/mongo_fdw/mongo-c-driver
cmake -DENABLE_AUTOMATIC_INIT_AND_CLEANUP=OFF -DENABLE_SSL=AUTO .
make install
cd $TEMP_DIR/mongo_fdw
cp $TEMP_DIR/json-c-0.15-20200726.tar.gz ./
tar -xzvf json-c-0.15-20200726.tar.gz
rm -rf json-c
mv json-c-json-c-0.15-20200726 json-c
cd $TEMP_DIR/mongo_fdw/json-c
cmake .
make -j 2
make install
apt-get install -y libmongoc-dev
cd $TEMP_DIR/mongo_fdw
echo "#ifdef __CONFIG__" >> config.h
echo "#define META_DRIVER" >> config.h
echo "#endif" >> config.h
export PKG_CONFIG_PATH=mongo-c-driver/src/libmongoc/src:mongo-c-driver/src/libbson/src
mv Makefile Makefile.origin
cp Makefile.meta Makefile
# C_INCLUDE_PATH="/include/libmongoc-1.0/mongoc:/include/libbson-1.0" USE_PGXS=1 make
# C_INCLUDE_PATH="/include/libmongoc-1.0/mongoc:/include/libbson-1.0" USE_PGXS=1 make install
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/neondatabase/neon
cd $TEMP_DIR/neon/pgxn/hnsw
USE_PGXS=1 make
USE_PGXS=1 make install
# cd $TEMP_DIR
# curl https://install.citusdata.com/community/deb.sh > add-citus-repo.sh
# bash add-citus-repo.sh
# apt-get install -y postgresql-15-citus-12
# Unfortunately, the Citus repository does not contain packages for non-x86_64 architectures.
apt-get install -y libzstd-dev libxslt1-dev libxml2-dev libcurl-ocaml-dev
cd $TEMP_DIR
git clone --depth 1 https://github.com/citusdata/citus
cd $TEMP_DIR/citus
./autogen.sh
./configure
make -j8
make install
cd $TEMP_DIR
apt-get install -y libboost-all-dev
# wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/datasketches/1.7.0/datasketches-1.7.0.zip
unzip datasketches-1.7.0.zip
cd $TEMP_DIR/datasketches-1.7.0
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
# get GraphBLAS, compile with debug symbols
# curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L http://faculty.cse.tamu.edu/davis/GraphBLAS/GraphBLAS-3.1.1.tar.gz -o GraphBLAS-3.1.1.tar.gz
tar -zxvf GraphBLAS-3.1.1.tar.gz
cd GraphBLAS-3.1.1
make library CMAKE_OPTIONS='-DCMAKE_BUILD_TYPE=Debug'
make install
cd $TEMP_DIR
git clone --depth 1 --branch 22July2019 https://github.com/GraphBLAS/LAGraph.git && cd LAGraph && make -j4 library && make install
cd $TEMP_DIR
curl --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -s -L https://github.com/theory/pgtap/archive/v0.99.0.tar.gz | tar zxvf - && cd pgtap-0.99.0 && make -j4 && make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/michelp/pggraphblas
cd $TEMP_DIR/pggraphblas
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b REL1_2_STABLE https://github.com/pgbigm/pg_bigm
cd $TEMP_DIR/pg_bigm
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/percona/pg_stat_monitor
cd $TEMP_DIR/pg_stat_monitor
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/neondatabase/pg_embedding
cd $TEMP_DIR/pg_embedding
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/cybertec-postgresql/pgfaceting
cd $TEMP_DIR/pgfaceting
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/pgexperts/pg_plan_filter
cd $TEMP_DIR/pg_plan_filter
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/postgrespro/pg_variables
cd $TEMP_DIR/pg_variables
USE_PGXS=1 make install
cd $TEMP_DIR
apt-get install -y libcurl-ocaml-dev
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/pg_curl/2.2.2/pg_curl-2.2.2.zip
unzip pg_curl-2.2.2.zip
cd $TEMP_DIR/pg_curl-2.2.2
USE_PGXS=1 make install
cd $TEMP_DIR
apt-get install -y systemtap-sdt-dev
mkdir -p /usr/lib/postgresql/15/lib
mkdir -p /usr/lib/postgresql/15/lib/bitcode
mkdir -p /usr/share/postgresql/15/extension
mv pg_task.so /usr/lib/postgresql/15/lib/
mv pg_task.index.bc /usr/lib/postgresql/15/lib/bitcode/
mv pg_task /usr/lib/postgresql/15/lib/bitcode/
mv pg_task.control /usr/share/postgresql/15/extension/
mv pg_task--1.0.sql /usr/share/postgresql/15/extension/
cd $TEMP_DIR
git clone --depth 1 -b extension https://github.com/RekGRpth/pg_task
# OR
# git clone --depth 1 -b extension https://github.com/RekGRpth/pg_task
# cd $TEMP_DIR/pg_task
# USE_PGXS=1 make install
# OR
# wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/pg_task/2.0.41/pg_task-2.0.41.zip
# unzip pg_task-2.0.41.zip
# cd $TEMP_DIR/pg_task-2.0.41
# USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/temporal_tables/1.2.2/temporal_tables-1.2.2.zip
unzip temporal_tables-1.2.2.zip
cd $TEMP_DIR/temporal_tables-1.2.2
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b v3.1 https://github.com/darold/pgtt
cd $TEMP_DIR/pgtt
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/pg_query_rewrite/0.0.5/pg_query_rewrite-0.0.5.zip
unzip pg_query_rewrite-0.0.5.zip
cd $TEMP_DIR/pg_query_rewrite-0.0.5
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/pg_track_settings/2.1.2/pg_track_settings-2.1.2.zip
unzip pg_track_settings-2.1.2.zip
cd $TEMP_DIR/pg_track_settings-2.1.2
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/aggs_for_vecs/1.3.0/aggs_for_vecs-1.3.0.zip
unzip aggs_for_vecs-1.3.0.zip
cd $TEMP_DIR/aggs_for_vecs-1.3.0
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/quantile/1.1.7/quantile-1.1.7.zip
unzip quantile-1.1.7.zip
cd $TEMP_DIR/quantile-1.1.7
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/pg_utility_trigger_functions/1.9.1/pg_utility_trigger_functions-1.9.1.zip
unzip pg_utility_trigger_functions-1.9.1.zip
cd $TEMP_DIR/pg_utility_trigger_functions-1.9.1
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/pg_safer_settings/0.8.10/pg_safer_settings-0.8.10.zip
unzip pg_safer_settings-0.8.10.zip
cd $TEMP_DIR/pg_safer_settings-0.8.10
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/ddlx/0.27.0/ddlx-0.27.0.zip
unzip ddlx-0.27.0.zip
cd $TEMP_DIR/ddlx-0.27.0
USE_PGXS=1 make install
cd $TEMP_DIR
wget -T 36000 -t 0 --waitretry=5 https://api.pgxn.org/dist/pgtelemetry/1.6.0/pgtelemetry-1.6.0.zip
unzip pgtelemetry-1.6.0.zip
cd $TEMP_DIR/pgtelemetry-1.6.0
USE_PGXS=1 make install
# cd $TEMP_DIR
# git clone --depth 1 -b REL15_0 https://github.com/ossc-db/pg_dbms_stats
# cd $TEMP_DIR/pg_dbms_stats
# USE_PGXS=1 make install
cd $TEMP_DIR
apt-get install -y libzlcore-dev
git clone --depth 1 -b REL_15_STABLE https://github.com/ossc-db/pg_rman
cd $TEMP_DIR/pg_rman
make install
# http://sigaev.ru/ 和 http://www.sigaev.ru/git/gitweb.cgi 需要开网络授权, 否则无法访问导致build image报错.
cd $TEMP_DIR
git clone --depth 1 git://sigaev.ru/online_analyze
cd $TEMP_DIR/online_analyze/
USE_PGXS=1 make
USE_PGXS=1 make install
# http://sigaev.ru/ 和 http://www.sigaev.ru/git/gitweb.cgi 需要开网络授权, 否则无法访问导致build image报错.
cd $TEMP_DIR
git clone --depth 1 git://sigaev.ru/plantuner
cd $TEMP_DIR/plantuner/
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b pg15 https://github.com/digoal/gevel
cd $TEMP_DIR/gevel
. ./install.sh
cd $TEMP_DIR
git clone --depth 1 -b 4.4 https://github.com/zubkov-andrei/pg_profile
cd $TEMP_DIR/pg_profile
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
# git clone --depth 1 https://github.com/s-hironobu/pg_plan_inspector
# cd $TEMP_DIR/pg_plan_inspector
# bug 修复之前 : https://github.com/s-hironobu/pg_plan_inspector/issues/1
git clone https://github.com/s-hironobu/pg_plan_inspector
cd $TEMP_DIR/pg_plan_inspector
git checkout fa845045ed5a776779f2d5308608ac18ed045aad
USE_PGXS=1 make
USE_PGXS=1 make install
# apt-get install -y libhealpix-cxx-dev
# cd $TEMP_DIR
# git clone --depth 1 -b 1.2.3 https://github.com/postgrespro/pgsphere
# cd $TEMP_DIR/pgsphere
# USE_PGXS=1 make
# USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/sraoss/pg_ivm
cd $TEMP_DIR/pg_ivm
USE_PGXS=1 make
USE_PGXS=1 make install
# cd $TEMP_DIR
# git clone --depth 1 https://github.com/pgvector/pgvector
# cd $TEMP_DIR/pgvector
# USE_PGXS=1 make
# USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth=1 https://github.com/vyruss/pg_statviz.git
cd $TEMP_DIR/pg_statviz
USE_PGXS=1 make install
apt-get install -y python3-pip
# pip install pg_statviz
cd $TEMP_DIR
## curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/Kitware/CMake/releases/download/v3.27.4/cmake-3.27.4.tar.gz -o cmake-3.27.4.tar.gz
# tar -zxvf cmake-3.27.4.tar.gz
# cd $TEMP_DIR/cmake-3.27.4
# env CC=clang CXX=clang++ ./bootstrap -- -DCMAKE_BUILD_TYPE:STRING=Release
# make -j8
# make install
tar -zxvf cmake-3.27.9-linux-aarch64.tar.gz
mv cmake-3.27.9-linux-aarch64 cmake-3.27.9
cd $TEMP_DIR
git clone --depth 1 https://github.com/zachasme/h3-pg
cd $TEMP_DIR/h3-pg
USE_PGXS=1 make all
USE_PGXS=1 make install
cd $TEMP_DIR
export RUSTUP_DIST_SERVER=https://mirrors.ustc.edu.cn/rust-static
export RUSTUP_UPDATE_ROOT=https://mirrors.ustc.edu.cn/rust-static/rustup
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs -o ./rust.sh
chmod 500 rust.sh
./rust.sh -y
source "$HOME/.cargo/env"
cd $TEMP_DIR
git clone --depth 1 https://github.com/postgresml/pgcat
cd $TEMP_DIR/pgcat
cargo build --release
cd $TEMP_DIR
git clone --depth 1 https://github.com/chimpler/postgres-aws-s3
cd $TEMP_DIR/postgres-aws-s3
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b PG15 https://github.com/apache/age
cd $TEMP_DIR/age
USE_PGXS=1 make
USE_PGXS=1 make install
apt-get install -y npm
# cd $TEMP_DIR
# tar -zxvf node-v14.21.3.tar.gz
# cd node-v14.21.3
# ./configure
# CC=clang CXX=clang++ make -j 4
# CC=clang CXX=clang++ make install
npm config set registry https://registry.npmmirror.com
npm config set fetch-retries 100
npm config set fetch-retry-mintimeout 120000
npm config set fetch-retry-maxtimeout 60000000
npm config set cache-min 3600
npm i pm2
cd $TEMP_DIR
git clone --depth 1 https://github.com/apache/age-viewer
cd $TEMP_DIR/age-viewer
npm run setup
cd $TEMP_DIR
git clone --depth 1 https://github.com/michelp/pgjwt
cd $TEMP_DIR/pgjwt
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/supabase/pg_net
cd $TEMP_DIR/pg_net
USE_PGXS=1 make install
# bug : ERROR: extension "pg_net" has no installation script nor update path for version "0.7.3"
# cd $TEMP_DIR/pg_net/sql
# cp -f -n ./pg_net.sql ./pg_net--0.7.3.sql /usr/share/postgresql/15/extension/
cd $TEMP_DIR
wget -q -O - http://www.xunsearch.com/scws/down/scws-1.2.3.tar.bz2 | tar jxf -
cd $TEMP_DIR/scws-1.2.3
./configure
make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/amutu/zhparser.git
cd $TEMP_DIR/zhparser
PG_CONFIG=/usr/lib/postgresql/15/bin/pg_config make && make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/pramsey/pgsql-http
cd $TEMP_DIR/pgsql-http
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/pgsentinel/pgsentinel
cd $TEMP_DIR/pgsentinel/src
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/postgrespro/zson
cd $TEMP_DIR/zson
USE_PGXS=1 make install
cd $TEMP_DIR
cp pg_tiktoken--0.0.1.sql /usr/share/postgresql/15/extension/
cp pg_tiktoken.control /usr/share/postgresql/15/extension/
cp pg_tiktoken.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
git clone --depth 1 --recursive https://github.com/lanterndata/lantern.git
cd $TEMP_DIR/lantern
mkdir build
cd $TEMP_DIR/lantern/build
cmake ..
make install
cd $TEMP_DIR
cp lantern_extras--0.0.3.sql /usr/share/postgresql/15/extension/
cp lantern_extras.control /usr/share/postgresql/15/extension/
cp lantern_extras.so /usr/lib/postgresql/15/lib/
cp lantern-cli /var/lib/postgresql/
chown postgres:postgres /var/lib/postgresql/lantern-cli
cd $TEMP_DIR
git clone --depth 1 -b v1.1.0 https://github.com/hydradatabase/hydra
cd $TEMP_DIR/hydra/columnar
./configure
USE_PGXS=1 make install
cd $TEMP_DIR
mkdir -p /usr/local/onnxruntime
tar vzxf onnxruntime-linux-aarch64-1.15.1.tgz -C /usr/local/onnxruntime --strip-components=1
echo "/usr/local/onnxruntime/lib" > /etc/ld.so.conf.d/onnxruntime.conf
ldconfig
cd $TEMP_DIR
git clone --depth 1 https://github.com/kibae/onnxruntime-server
cd $TEMP_DIR/onnxruntime-server
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build --parallel 4
cmake --install build --prefix /usr/local/onnxruntime-server
cd $TEMP_DIR
git clone --depth 1 -b v1.15.1 https://github.com/microsoft/onnxruntime
cp /tmp/onnxruntime/include/onnxruntime/core/session/* /usr/local/onnxruntime/
cd $TEMP_DIR
git clone --depth 1 --recursive https://github.com/kibae/pg_onnx.git
cd $TEMP_DIR/pg_onnx
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build --target pg_onnx --parallel 4
cmake --install build/pg_onnx
cd $TEMP_DIR
# curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash
# apt-get install -y git-lfs
for ((i=1;i>=0;i=1))
do
curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | bash
if [ $? -eq 0 ]; then
break
fi
done
for ((i=1;i>=0;i=1))
do
apt-get install -y git-lfs
if [ $? -eq 0 ]; then
break
fi
done
cd $TEMP_DIR
GIT_LFS_SKIP_SMUDGE=1 git clone --depth 1 https://github.com/onnx/models
cd $TEMP_DIR/models
git lfs install
cd $TEMP_DIR
git clone --depth 1 https://github.com/fboulnois/pg_uuidv7
cd $TEMP_DIR/pg_uuidv7
USE_PGXS=1 make install
# apt-get install -y libcurl4-openssl-dev uuid-dev libpulse-dev
# cd $TEMP_DIR
# # git clone --depth 1 -b apache-arrow-12.0.1 https://github.com/apache/arrow.git
# git clone --depth 1 -b apache-arrow-14.0.2 https://github.com/apache/arrow.git
# cd $TEMP_DIR/arrow/cpp
# mkdir build-release
# cd $TEMP_DIR/arrow/cpp/build-release
#
# # build选项: https://arrow.apache.org/docs/developers/cpp/building.html
# for ((i=1;i>=0;i=1))
# do
# # cmake -DARROW_DEPENDENCY_SOURCE=BUNDLED -DARROW_PARQUET=ON -DARROW_ORC=ON -DARROW_S3=ON -DARROW_WITH_LZ4=ON -DARROW_WITH_SNAPPY=ON -DARROW_WITH_ZLIB=ON -DARROW_WITH_ZSTD=ON -DPARQUET_REQUIRE_ENCRYPTION=ON ..
# cmake -DARROW_DEPENDENCY_SOURCE=BUNDLED -DARROW_PARQUET=ON ..
# if [ $? -eq 0 ]; then
# break
# fi
# done
#
# for ((i=1;i>=0;i=1))
# do
# make -j4
# if [ $? -eq 0 ]; then
# break
# fi
# done
#
# make install
# ldconfig
# https://arrow.apache.org/install/
cd $TEMP_DIR
apt-get install -y libcurl4-openssl-dev uuid-dev libpulse-dev
for ((i=1;i>=0;i=1))
do
apt-get update
if [ $? -eq 0 ]; then
break
fi
done
apt-get install -y -V ca-certificates lsb-release
for ((i=1;i>=0;i=1))
do
wget -T 36000 -t 0 --waitretry=5 https://apache.jfrog.io/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
if [ $? -eq 0 ]; then
break
fi
done
apt-get install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb
for ((i=1;i>=0;i=1))
do
apt-get update
if [ $? -eq 0 ]; then
break
fi
done
apt-get install -y -V libarrow-dev # For C++
apt-get install -y -V libarrow-glib-dev # For GLib (C)
apt-get install -y -V libarrow-dataset-dev # For Apache Arrow Dataset C++
apt-get install -y -V libarrow-dataset-glib-dev # For Apache Arrow Dataset GLib (C)
apt-get install -y -V libarrow-acero-dev # For Apache Arrow Acero
apt-get install -y -V libarrow-flight-dev # For Apache Arrow Flight C++
apt-get install -y -V libarrow-flight-glib-dev # For Apache Arrow Flight GLib (C)
apt-get install -y -V libarrow-flight-sql-dev # For Apache Arrow Flight SQL C++
apt-get install -y -V libarrow-flight-sql-glib-dev # For Apache Arrow Flight SQL GLib (C)
apt-get install -y -V libgandiva-dev # For Gandiva C++
apt-get install -y -V libgandiva-glib-dev # For Gandiva GLib (C)
apt-get install -y -V libparquet-dev # For Apache Parquet C++
apt-get install -y -V libparquet-glib-dev # For Apache Parquet GLib (C)
cd $TEMP_DIR
git clone --depth 1 https://github.com/adjust/parquet_fdw
cd $TEMP_DIR/parquet_fdw
# 这个bug 解决之前 (这个clang flag不支持arm平台)
# clang: error: unknown argument: '-moutline-atomics'
/usr/bin/clang-11 -xc++ -Wno-ignored-attributes -Wno-register -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -emit-llvm -c -Wall -Wpointer-arith -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -std=c++17 -O3 -fPIC -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -o src/common.bc src/common.cpp
/usr/bin/clang-11 -xc++ -Wno-ignored-attributes -Wno-register -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -emit-llvm -c -Wall -Wpointer-arith -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -std=c++17 -O3 -fPIC -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -o src/reader.bc src/reader.cpp
/usr/bin/clang-11 -xc++ -Wno-ignored-attributes -Wno-register -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -emit-llvm -c -Wall -Wpointer-arith -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -std=c++17 -O3 -fPIC -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -o src/exec_state.bc src/exec_state.cpp
/usr/bin/clang-11 -xc++ -Wno-ignored-attributes -Wno-register -fno-strict-aliasing -fwrapv -O2 -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -emit-llvm -c -Wall -Wpointer-arith -Wendif-labels -Wmissing-format-attribute -Wimplicit-fallthrough=3 -Wcast-function-type -Wformat-security -fno-strict-aliasing -fwrapv -g -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -std=c++17 -O3 -fPIC -I. -I./ -I/usr/include/postgresql/15/server -I/usr/include/postgresql/internal -Wdate-time -D_FORTIFY_SOURCE=2 -D_GNU_SOURCE -I/usr/include/libxml2 -o src/parquet_impl.bc src/parquet_impl.cpp
make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/pgspider/sqlite_fdw
cd $TEMP_DIR/sqlite_fdw
USE_PGXS=1 make
USE_PGXS=1 make install
cd $TEMP_DIR
cp pg_bm25--0.5.7.sql /usr/share/postgresql/15/extension/
cp pg_bm25.control /usr/share/postgresql/15/extension/
cp pg_bm25.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
cp pg_analytics--0.5.7.sql /usr/share/postgresql/15/extension/
cp pg_analytics.control /usr/share/postgresql/15/extension/
cp pg_analytics.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
cp pg_search--0.0.0.sql /usr/share/postgresql/15/extension/
cp pg_search.control /usr/share/postgresql/15/extension/
cp pg_search.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
cp pg_sparse--0.0.0.sql /usr/share/postgresql/15/extension/
cp pg_sparse.control /usr/share/postgresql/15/extension/
cp pg_sparse.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
git clone --depth 1 -b 15.2 https://github.com/ossc-db/pg_statsinfo
cd $TEMP_DIR/pg_statsinfo
PG_LDFLAGS=-L/usr/lib/postgresql/15/lib make
PG_LDFLAGS=-L/usr/lib/postgresql/15/lib make install
cd /usr/lib/postgresql/15
git clone --depth 1 -b 15.1 https://github.com/ossc-db/pg_stats_reporter
cd $TEMP_DIR/
git clone --depth 1 https://gitlab.com/pg_proctab/pg_proctab
cd $TEMP_DIR/pg_proctab
USE_PGXS=1 make install
apt-get install -y libelf-dev
cd $TEMP_DIR/
git clone --depth 1 https://gitlab.com/pg_top/pg_top
cd $TEMP_DIR/pg_top
cmake CMakeLists.txt
USE_PGXS=1 make install
cd $TEMP_DIR/
wget https://golang.org/dl/go1.17.linux-arm64.tar.gz
tar -zxvf go1.17.linux-arm64.tar.gz -C /usr/local/
export PATH=/usr/local/go/bin:${PATH}
echo "export PATH=/usr/local/go/bin:${PATH}" >>/root/.bashrc
echo "export PATH=/usr/local/go/bin:${PATH}" >>/var/lib/postgresql/.bash_profile
cd $TEMP_DIR/
git clone --depth 1 https://github.com/lesovsky/pgcenter
cd $TEMP_DIR/pgcenter
make dep
make build
make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/petere/pguint
cd $TEMP_DIR/pguint
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b v0.5.7 https://github.com/paradedb/paradedb
cd $TEMP_DIR/paradedb/pg_sparse
USE_PGXS=1 make install
chmod 644 ./sql/svector--0.5.7.sql
cp ./sql/svector--0.5.7.sql /usr/share/postgresql/15/extension/
cd $TEMP_DIR
git clone --depth 1 https://github.com/bdrouvot/pg_subtrans_infos
cd $TEMP_DIR/pg_subtrans_infos
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/bdrouvot/pg_subxact_counters
cd $TEMP_DIR/pg_subxact_counters/c
USE_PGXS=1 make install
cd $TEMP_DIR
cp pg_idkit--0.2.1.sql /usr/share/postgresql/15/extension/
cp pg_idkit.control /usr/share/postgresql/15/extension/
cp pg_idkit.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
apt-get install -y build-essential libproj-dev libjson-c-dev libgsl-dev libgeos-dev
git clone --depth 1 https://github.com/MobilityDB/MobilityDB
cd $TEMP_DIR/MobilityDB
mkdir build
cd $TEMP_DIR/MobilityDB/build
cmake ..
make -j 4
make install
cd $TEMP_DIR
cp pgdd--0.5.2.sql /usr/share/postgresql/15/extension/
cp pgdd.control /usr/share/postgresql/15/extension/
cp pgdd.so /usr/lib/postgresql/15/lib/
# cd $TEMP_DIR
# git clone --depth 1 https://github.com/PGer/pipelinedb_pg15.git
# cd $TEMP_DIR/pipelinedb_pg15
# apt-get install -y libczmq4 libczmq-dev
# USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/jaiminpan/pg_nanoid.git
cd $TEMP_DIR/pg_nanoid
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/andrielfn/pg-ulid.git
cd $TEMP_DIR/pg-ulid
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/phillbaker/pg_migrate.git
cd $TEMP_DIR/pg_migrate/
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/df7cb/pg_filedump.git
cd $TEMP_DIR/pg_filedump/
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/petere/pgpcre
cd $TEMP_DIR/pgpcre
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://github.com/omniti-labs/pg_jobmon
cd $TEMP_DIR/pg_jobmon
USE_PGXS=1 make install
# cd $TEMP_DIR
# git clone --depth 1 https://github.com/pgpartman/pg_partman
# cd $TEMP_DIR/pg_partman
# USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 https://gitee.com/seanguo_007/plpgsql_pg4ml.git
cd $TEMP_DIR/plpgsql_pg4ml
USE_PGXS=1 make install
cd $TEMP_DIR
cp pgmq--1.1.1.sql /usr/share/postgresql/15/extension/
cp pgmq.control /usr/share/postgresql/15/extension/
cp pgmq.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
cp pg_later--0.0.14.sql /usr/share/postgresql/15/extension/
cp pg_later.control /usr/share/postgresql/15/extension/
cp pg_later.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
cp vectorize--0.9.0.sql /usr/share/postgresql/15/extension/
cp vectorize.control /usr/share/postgresql/15/extension/
cp vectorize.so /usr/lib/postgresql/15/lib/
cd $TEMP_DIR
git clone --depth 1 https://github.com/pgEdge/snowflake
cd $TEMP_DIR/snowflake
USE_PGXS=1 make install
cd $TEMP_DIR
git clone --depth 1 -b walminer_3.0_stable https://gitee.com/movead/XLogMiner
cd $TEMP_DIR/XLogMiner/walminer/
USE_PGXS=1 MAJORVERSION=15 make install
# https://github.com/pgspider/influxdb_fdw
# InfluxDB Foreign Data Wrapper for PostgreSQL.
cd /tmp
git clone --depth 1 https://github.com/pgspider/influxdb_fdw
cd /tmp/influxdb_fdw
go get github.com/influxdata/influxdb1-client/v2
git clone --depth 1 https://github.com/pgspider/influxdb-cxx
cd /tmp/influxdb_fdw/influxdb-cxx
cmake . -DINFLUXCXX_WITH_BOOST=OFF -DINFLUXCXX_TESTING=OFF
make install
cd /tmp/influxdb_fdw
make CC=gcc CXX=g++ USE_PGXS=1 with_llvm=no GO_CLIENT=1 CXX_CLIENT=1
make install CC=gcc CXX=g++ USE_PGXS=1 with_llvm=no GO_CLIENT=1 CXX_CLIENT=1
# https://github.com/HexaCluster/pg_dbms_metadata
# pg_dbms_metadata v1.0.0 - Oracle's DBMS_METADATA Compatibility for PostgreSQL DDL Extraction
cd /tmp
git clone --depth 1 https://github.com/HexaCluster/pg_dbms_metadata
cd /tmp/pg_dbms_metadata
make install USE_PGXS=1
# https://github.com/viggy28/pg_savior
# pg_savior is a PostgreSQL extension designed to prevent accidental data loss due to non-parameterized DELETE queries without a WHERE clause.
cd /tmp
git clone --depth 1 https://github.com/viggy28/pg_savior
cd /tmp/pg_savior
make install USE_PGXS=1
# https://github.com/dimitri/pgcopydb
# https://pgcopydb.readthedocs.io/en/latest/
apt-get install -y libgc-dev
cd /tmp
git clone --depth 1 https://github.com/dimitri/pgcopydb
cd /tmp/pgcopydb
make -s clean
make -s -j4 install
cd /tmp
git clone --depth 1 https://github.com/supabase/supa_audit
cd /tmp/supa_audit
USE_PGXS=1 make install
## plv8/Makefile
## ifeq ($(UNAME_S),Linux)
## SHLIB_LINK += -lrt -std=c++17
#cd /tmp
## apt-get install -y libtinfo5 build-essential pkg-config
#git clone --depth 1 -b v3.2.2 https://github.com/plv8/plv8
#cd /tmp/plv8
#export SHLIB_LINK = " -lrt -std=c++17 "
#EXECUTION_TIMEOUT=1 USE_ICU=1 USE_PGXS=1 PG_CONFIG=$(which pg_config) make
#EXECUTION_TIMEOUT=1 USE_ICU=1 USE_PGXS=1 PG_CONFIG=$(which pg_config) make install
# 安装HaloDB,
groupadd -g 3000 halo
useradd -u 3000 -g halo -m -d /home/halo -s /bin/bash halo
cd /tmp
tar -zxvf halo_14.debian11.aarch64.tar.gz -C /home/halo/
echo "# add by digoal" >> /home/halo/.bashrc
echo "alias rm='rm -i'" >> /home/halo/.bashrc
echo "alias cp='cp -i'" >> /home/halo/.bashrc
echo "alias ll='ls -larth'" >> /home/halo/.bashrc
echo "alias mv='mv -i'" >> /home/halo/.bashrc
echo "export HALO_HOME=/home/halo/product/dbms/14" >> /home/halo/.bashrc
echo "export LD_LIBRARY_PATH=\$HALO_HOME/lib" >> /home/halo/.bashrc
echo "export PGDATA=/home/halo/halodata" >> /home/halo/.bashrc
echo "export PGHOST=\$PGDATA" >> /home/halo/.bashrc
echo "export PGUSER=postgres" >> /home/halo/.bashrc
echo "export PATH=\$HALO_HOME/bin:\$PATH" >> /home/halo/.bashrc
echo "export PGPORT=1521" >> /home/halo/.bashrc
# 初始化halodata
su - halo -c "mkdir /home/halo/halodata"
su - halo -c "chmod 700 /home/halo/halodata"
su - halo -c "/home/halo/product/dbms/14/bin/initdb -D /home/halo/halodata -E UTF8 --lc-collate=C --lc-ctype=en_US.utf-8 -U postgres"
cd /home/halo/halodata
echo "host all all 0.0.0.0/0 scram-sha-256" >> ./pg_hba.conf
echo "standard_parserengine_auxiliary = on" >> ./postgresql.auto.conf
echo "database_compat_mode = 'oracle'" >> ./postgresql.auto.conf
echo "oracle.use_datetime_as_date = true" >> ./postgresql.auto.conf
echo "transform_null_equals = off" >> ./postgresql.auto.conf
echo "listen_addresses = '0.0.0.0'" >> ./postgresql.auto.conf
echo "port = 1521" >> ./postgresql.auto.conf
echo "max_connections = 2000" >> ./postgresql.auto.conf
echo "unix_socket_directories = '.'" >> ./postgresql.auto.conf
echo "shared_buffers = 128MB" >> ./postgresql.auto.conf
echo "dynamic_shared_memory_type = posix" >> ./postgresql.auto.conf
echo "vacuum_cost_delay = 0" >> ./postgresql.auto.conf
echo "bgwriter_delay = 20ms" >> ./postgresql.auto.conf
echo "bgwriter_lru_maxpages = 500" >> ./postgresql.auto.conf
echo "bgwriter_lru_multiplier = 5.0" >> ./postgresql.auto.conf
echo "max_parallel_workers_per_gather = 0" >> ./postgresql.auto.conf
echo "synchronous_commit = off" >> ./postgresql.auto.conf
echo "wal_compression = on" >> ./postgresql.auto.conf
echo "wal_writer_delay = 10ms" >> ./postgresql.auto.conf
echo "max_wal_size = 1GB " >> ./postgresql.auto.conf
echo "min_wal_size = 80MB " >> ./postgresql.auto.conf
echo "random_page_cost = 1.1" >> ./postgresql.auto.conf
echo "log_destination = 'csvlog'" >> ./postgresql.auto.conf
echo "logging_collector = on" >> ./postgresql.auto.conf
echo "log_truncate_on_rotation = on" >> ./postgresql.auto.conf
echo "log_timezone = 'Etc/UTC' " >> ./postgresql.auto.conf
echo "autovacuum = on" >> ./postgresql.auto.conf
echo "autovacuum_vacuum_cost_delay = 0ms" >> ./postgresql.auto.conf
echo "vacuum_freeze_table_age = 750000000 " >> ./postgresql.auto.conf
echo "vacuum_multixact_freeze_table_age = 750000000 " >> ./postgresql.auto.conf
echo "datestyle = 'iso, mdy' " >> ./postgresql.auto.conf
echo "timezone = 'Etc/UTC' " >> ./postgresql.auto.conf
echo "lc_messages = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "lc_monetary = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "lc_numeric = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "lc_time = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "default_text_search_config = 'pg_catalog.english'" >> ./postgresql.auto.conf
# 启动halo, 创建Oracle兼容性
# su - halo
# pg_ctl start
# psql -c "create extension aux_oracle cascade;"
# databend [在宇宙最强数据库镜像中部署databend](../202402/20240201_03.md)
groupadd -g 3001 databend
useradd -u 3001 -g databend -m -d /home/databend -s /bin/bash databend
mkdir -p /var/log/databend
mkdir -p /var/lib/databend/raft
chown -R databend:databend /var/log/databend
chown -R databend:databend /var/lib/databend
# su - databend -c "curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 --proto '=https' --tlsv1.2 -sSf -L https://sh.rustup.rs | sh -s -- -y"
# su - databend -c "source /home/databend/.cargo/env"
# echo "[source.crates-io]" >> /home/databend/.cargo/config
# echo "replace-with = 'ustc'" >> /home/databend/.cargo/config
# echo " " >> /home/databend/.cargo/config
# echo "[source.ustc]" >> /home/databend/.cargo/config
# echo "registry = \"sparse+https://mirrors.ustc.edu.cn/crates.io-index/\"" >> /home/databend/.cargo/config
echo "# add by digoal" >> /home/databend/.bashrc
echo "alias rm='rm -i'" >> /home/databend/.bashrc
echo "alias cp='cp -i'" >> /home/databend/.bashrc
echo "alias ll='ls -larth'" >> /home/databend/.bashrc
echo "alias mv='mv -i'" >> /home/databend/.bashrc
# echo ". /home/databend/.cargo/env" >> /home/databend/.bashrc
cd /tmp
# tar -zxvf databend-v1.2.307-aarch64-unknown-linux-gnu.tar.gz -C /home/databend/
mkdir -p /home/databend/databend_software
tar -zxvf databend-aarch64.tar.gz -C /home/databend/databend_software
tar -zxvf bendsql-aarch64-unknown-linux-gnu.tar.gz -C /usr/local/bin/
cd /home/databend/databend_software
echo " " >> ./configs/databend-query.toml
echo "[[query.users]]" >> ./configs/databend-query.toml
echo "name = \"root\"" >> ./configs/databend-query.toml
echo "auth_type = \"no_password\"" >> ./configs/databend-query.toml
chown -R databend:databend /home/databend/databend_software
cd /tmp
# su - databend -c "cargo install bendsql"
# 启动databend
# su - databend
# cd /home/databend/databend_software
# ./scripts/start.sh
# 连接databend
# su - databend
# bendsql
# 停止databend
# su - databend
# cd /home/databend/databend_software
# ./scripts/stop.sh
cd $TEMP_DIR
cp plrust--1.1.sql /usr/share/postgresql/15/extension/
cp plrust.control /usr/share/postgresql/15/extension/
cp plrust.so /usr/lib/postgresql/15/lib/
cd /tmp
git clone --depth 1 https://github.com/okbob/pgmeminfo
cd /tmp/pgmeminfo
USE_PGXS=1 make install
cd /usr/lib/postgresql/15
git clone --depth 1 https://github.com/swida/sqlbench
cd /usr/lib/postgresql/15/sqlbench
autoreconf -if
./configure --with-postgresql="$PGHOME"
C_INCLUDE_PATH=/usr/include/postgresql make
C_INCLUDE_PATH=/usr/include/postgresql make install
apt-get install -y libpcap-dev libnl-genl-3-dev
cd /tmp
git clone --depth 1 -b 8.4.0-stable https://github.com/ntop/PF_RING
cd /tmp/PF_RING/userland/lib
./configure && make
make install
cd /tmp
git clone --depth 1 https://github.com/heterodb/pg-strom
cd /tmp/pg-strom/arrow-tools
PG_CONFIG=/usr/lib/postgresql/15/bin/pg_config C_INCLUDE_PATH=/tmp/PF_RING/kernel make
PG_CONFIG=/usr/lib/postgresql/15/bin/pg_config C_INCLUDE_PATH=/tmp/PF_RING/kernel make install
cd /usr/lib/postgresql/15
git clone --depth 1 https://github.com/apache/madlib
cd /usr/lib/postgresql/15/madlib
mkdir build
cd /usr/lib/postgresql/15/madlib/build
cmake ..
make -j 4
# $BUILD_ROOT/src/bin/madpack -s madlib -p postgres -c [user[/password]@][host][:port][/database] install
# cd /tmp
# git clone --depth 1 -b apache-arrow-12.0.0 https://github.com/apache/arrow.git
# cd /tmp/arrow/cpp
# mkdir build-release
# cd /tmp/arrow/cpp/build-release
# cmake -DARROW_DEPENDENCY_SOURCE=BUNDLED ..
# make -j4
# make install
#
# cd /tmp
# apt-get install -y libcurl4-openssl-dev uuid-dev libpulse-dev
# git clone --depth 1 -b 1.11.91 https://github.com/aws/aws-sdk-cpp
# cd /tmp/aws-sdk-cpp
# git submodule update --init --recursive --depth 1
# mkdir build
# cd /tmp/aws-sdk-cpp/build
# cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="s3;core"
# make -j4
# make install
#
# cd /tmp
# git clone --depth 1 -b v1.1.0 https://github.com/pgspider/parquet_s3_fdw
# cd /tmp/parquet_s3_fdw
# PG_CPPFLAGS="-Wno-register -D_GLIBCXX_USE_CXX11_ABI=0" USE_PGXS=1 make
# PG_CPPFLAGS="-std=c++17 -Wno-register -D_GLIBCXX_USE_CXX11_ABI=0" USE_PGXS=1 make
# PG_CPPFLAGS="-std=c++17 -Wno-register -D_GLIBCXX_USE_CXX11_ABI=0" USE_PGXS=1 make install
# echo "/usr/local/lib" >>/etc/ld.so.conf
# ldconfig
vi 4.sh
#!/bin/bash
set -vx
echo "* soft nofile 1024000" >> /etc/security/limits.conf
echo "* hard nofile 1024000" >> /etc/security/limits.conf
echo "* soft nproc unlimited" >> /etc/security/limits.conf
echo "* hard nproc unlimited" >> /etc/security/limits.conf
echo "* soft core unlimited" >> /etc/security/limits.conf
echo "* hard core unlimited" >> /etc/security/limits.conf
echo "* soft memlock unlimited" >> /etc/security/limits.conf
echo "* hard memlock unlimited" >> /etc/security/limits.conf
vi 5.sh
#!/bin/bash
set -vx
export PG_HOME=/var/lib/postgresql
. $PG_HOME/.bash_profile
initdb -D $PGDATA -U postgres -E UTF8 --lc-collate=C --lc-ctype=en_US.UTF8
cd $PGDATA
echo "host all all 0.0.0.0/0 scram-sha-256" >> ./pg_hba.conf
echo "listen_addresses = '0.0.0.0'" >> ./postgresql.auto.conf
echo "port = 1921" >> ./postgresql.auto.conf
echo "max_connections = 2000" >> ./postgresql.auto.conf
echo "unix_socket_directories = '., /var/run/postgresql'" >> ./postgresql.auto.conf
echo "shared_buffers = 128MB" >> ./postgresql.auto.conf
echo "dynamic_shared_memory_type = posix" >> ./postgresql.auto.conf
echo "vacuum_cost_delay = 0" >> ./postgresql.auto.conf
echo "bgwriter_delay = 20ms" >> ./postgresql.auto.conf
echo "bgwriter_lru_maxpages = 500" >> ./postgresql.auto.conf
echo "bgwriter_lru_multiplier = 5.0" >> ./postgresql.auto.conf
echo "max_parallel_workers_per_gather = 0" >> ./postgresql.auto.conf
echo "synchronous_commit = off" >> ./postgresql.auto.conf
echo "wal_compression = on" >> ./postgresql.auto.conf
echo "wal_writer_delay = 10ms" >> ./postgresql.auto.conf
echo "max_wal_size = 1GB " >> ./postgresql.auto.conf
echo "min_wal_size = 80MB " >> ./postgresql.auto.conf
echo "random_page_cost = 1.1" >> ./postgresql.auto.conf
echo "log_destination = 'csvlog'" >> ./postgresql.auto.conf
echo "logging_collector = on" >> ./postgresql.auto.conf
echo "log_truncate_on_rotation = on" >> ./postgresql.auto.conf
echo "log_timezone = 'Etc/UTC' " >> ./postgresql.auto.conf
echo "autovacuum = on" >> ./postgresql.auto.conf
echo "autovacuum_vacuum_cost_delay = 0ms" >> ./postgresql.auto.conf
echo "vacuum_freeze_table_age = 750000000 " >> ./postgresql.auto.conf
echo "vacuum_multixact_freeze_table_age = 750000000 " >> ./postgresql.auto.conf
echo "datestyle = 'iso, mdy' " >> ./postgresql.auto.conf
echo "timezone = 'Etc/UTC' " >> ./postgresql.auto.conf
echo "lc_messages = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "lc_monetary = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "lc_numeric = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "lc_time = 'en_US.UTF-8'" >> ./postgresql.auto.conf
echo "default_text_search_config = 'pg_catalog.english'" >> ./postgresql.auto.conf
4、准备Dockerfile
cd ~/pg15
vi Dockerfile
FROM --platform=$TARGETPLATFORM debian:12.5
MAINTAINER digoal zhou "dege.zzz@alibaba-inc.com"
ARG TARGETPLATFORM
ARG BUILDPLATFORM
RUN echo "I am running on $BUILDPLATFORM, building for $TARGETPLATFORM"
ENV DEBIAN_FRONTEND=noninteractive TEMP_DIR=/tmp ROOT_HOME=/root PG_HOME=/var/lib/postgresql PG_DATA=/var/lib/postgresql/15/pgdata PGHOME=/usr/lib/postgresql/15
STOPSIGNAL SIGINT
COPY --chmod=555 1.sh 2.sh 3.sh 4.sh 5.sh $TEMP_DIR/
COPY GraphBLAS-3.1.1.tar.gz libduckdb-linux-aarch64.zip duckdb_cli-linux-aarch64.zip cmake-3.27.4.tar.gz mongo-c-driver-1.17.3.tar.gz json-c-0.15-20200726.tar.gz pg_tiktoken--0.0.1.sql pg_tiktoken.control pg_tiktoken.so lantern_extras--0.0.3.sql lantern_extras.control lantern_extras.so lantern-cli onnxruntime-linux-aarch64-1.15.1.tgz node-v14.21.3.tar.gz pg_bm25.control pg_bm25.so pg_bm25--0.5.7.sql pg_search.control pg_search.so pg_search--0.0.0.sql pg_sparse.so pg_sparse.control pg_sparse--0.0.0.sql pg_task.so pg_task.index.bc pg_task.control pg_task--1.0.sql pg_idkit--0.2.1.sql pg_idkit.control pg_idkit.so pgdd--0.5.2.sql pgdd.control pgdd.so pgroll pgmq.control pgmq--1.1.1.sql pgmq.so pg_later.control pg_later.so pg_later--0.0.14.sql vectorize.control vectorize.so vectorize--0.9.0.sql cmake-3.27.9-linux-aarch64.tar.gz libduckdb-src.zip datasketches-1.7.0.zip halo_14.debian11.aarch64.tar.gz proxysql_2.5.5-debian11_arm64.deb pg_analytics.so pg_analytics.control pg_analytics--0.5.7.sql databend-aarch64.tar.gz plrust--1.1.sql plrust.control plrust.so bendsql-aarch64-unknown-linux-gnu.tar.gz $TEMP_DIR/
RUN mkdir /tmp/pg_task
COPY pg_task /tmp/pg_task
RUN $TEMP_DIR/1.sh
RUN $TEMP_DIR/2.sh
RUN $TEMP_DIR/3.sh
# WORKDIR $TEMP_DIR
# RUN apt-get install -y libtinfo5 build-essential ninja-build python3
# RUN git clone --depth 1 -b v3.1.7 https://github.com/plv8/plv8
# WORKDIR $TEMP_DIR/plv8
# RUN USE_ICU=1 USE_PGXS=1 make
# RUN USE_ICU=1 USE_PGXS=1 make install
#
# WORKDIR $TEMP_DIR
# RUN git clone --depth 1 -b apache-arrow-12.0.1 https://github.com/apache/arrow.git
# WORKDIR $TEMP_DIR/arrow/cpp
# RUN mkdir build-release
# WORKDIR $TEMP_DIR/arrow/cpp/build-release
# RUN cmake -DARROW_DEPENDENCY_SOURCE=BUNDLED ..
# RUN make -j4
# RUN make install
#
# WORKDIR $TEMP_DIR
# RUN apt-get install -y libcurl4-openssl-dev uuid-dev libpulse-dev
# RUN git clone --depth 1 -b 1.9.263 https://github.com/aws/aws-sdk-cpp
# WORKDIR $TEMP_DIR/aws-sdk-cpp
# RUN git submodule update --init --recursive --depth 1
# RUN mkdir build
# WORKDIR $TEMP_DIR/aws-sdk-cpp/build
# RUN cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_ONLY="s3;core"
# RUN make -j4
# RUN make install
#
# WORKDIR $TEMP_DIR
# RUN git clone --depth 1 -b v1.0.0 https://github.com/pgspider/parquet_s3_fdw
# WORKDIR $TEMP_DIR/parquet_s3_fdw
# RUN PG_CPPFLAGS="-Wno-register -D_GLIBCXX_USE_CXX11_ABI=0" USE_PGXS=1 make
# RUN PG_CPPFLAGS="-std=c++17 -Wno-register -D_GLIBCXX_USE_CXX11_ABI=0" USE_PGXS=1 make
# RUN PG_CPPFLAGS="-std=c++17 -Wno-register -D_GLIBCXX_USE_CXX11_ABI=0" USE_PGXS=1 make install
# RUN echo "/usr/local/lib" >>/etc/ld.so.conf
# RUN ldconfig
ENV ORACLE_BASE /usr/lib/postgresql/15/lib
WORKDIR $ORACLE_BASE
COPY instantclient_19_19/libclntsh.so.19.1 .
RUN ln -s libclntsh.so.19.1 libclntsh.so
RUN mkdir -p oracle/network/admin
COPY instantclient_19_19 ./oracle
RUN echo "export LD_LIBRARY_PATH=\$PGHOME/lib/oracle:\$LD_LIBRARY_PATH" >>$PG_HOME/.bash_profile
RUN echo "export ORACLE_HOME=\$PGHOME/lib/oracle" >>$PG_HOME/.bash_profile
RUN echo "export TNS_ADMIN=\$ORACLE_HOME/network/admin/" >>$PG_HOME/.bash_profile
RUN echo "export LD_LIBRARY_PATH=\$PGHOME/lib/oracle:\$LD_LIBRARY_PATH" >>$ROOT_HOME/.bashrc
RUN echo "export ORACLE_HOME=\$PGHOME/lib/oracle" >>$ROOT_HOME/.bashrc
RUN echo "export TNS_ADMIN=\$ORACLE_HOME/network/admin/" >>$ROOT_HOME/.bashrc
RUN . $ROOT_HOME/.bashrc
RUN echo "/usr/lib/postgresql/15/lib/oracle" >> /etc/ld.so.conf
RUN echo "/usr/local/lib" >> /etc/ld.so.conf
RUN echo "/usr/lib/postgresql/15/lib" >> /etc/ld.so.conf
RUN ldconfig
WORKDIR $TEMP_DIR
# RUN apt-get install -y libsqlite3-dev
RUN git clone --depth 1 https://github.com/ossc-db/dblink_plus
WORKDIR $TEMP_DIR/dblink_plus
RUN cp -r /usr/lib/postgresql/15/lib/oracle/sdk/include/* ./
RUN PG_CFLAGS=-I/usr/lib/postgresql/15/lib/oracle/sdk/include PG_LDFLAGS=-L/usr/lib/postgresql/15/lib/oracle USE_PGXS=1 make
RUN PG_CFLAGS=-I/usr/lib/postgresql/15/lib/oracle/sdk/include PG_LDFLAGS=-L/usr/lib/postgresql/15/lib/oracle USE_PGXS=1 make install
# RUN rm -rf $TEMP_DIR/*
RUN $TEMP_DIR/4.sh
USER postgres
RUN $TEMP_DIR/5.sh
WORKDIR $PG_HOME
# RUN wget -T 36000 -t 0 --waitretry=5 https://github.com/duckdb/duckdb/releases/download/v0.9.2/duckdb_cli-linux-aarch64.zip
# RUN curl -Z --connect-timeout 120 -m 36000 --retry 12000 --retry-delay 5 --retry-max-time 1200 -L https://github.com/duckdb/duckdb/releases/download/v0.9.2/duckdb_cli-linux-aarch64.zip -o duckdb_cli-linux-aarch64.zip
RUN cp $TEMP_DIR/duckdb_cli-linux-aarch64.zip $PG_HOME/
RUN unzip -d . duckdb_cli-linux-aarch64.zip
RUN cp $TEMP_DIR/pgroll $PG_HOME/
USER root
WORKDIR $ROOT_HOME
EXPOSE 1921
EXPOSE 3000
EXPOSE 8080
EXPOSE 80
EXPOSE 3306
ENTRYPOINT ["su", "-", "postgres", "-c", "/usr/lib/postgresql/15/bin/postgres -D \"/var/lib/postgresql/15/pgdata\""]
5、配置ignore文件
cd ~/pg15
mkdir logs
vi .dockerignore
logs/
6、制作镜像
6.1、确认已开启docker 实验属性: "experimental": true . 在docker desktop setting: docker engine中配置. 例如:
{
"builder": {
"gc": {
"defaultKeepStorage": "20GB",
"enabled": true
}
},
"dns": [
"8.8.8.8"
],
"experimental": true,
"registry-mirrors": [
"https://xxxxxx.mirror.aliyuncs.com"
]
}
下载debian arm64架构基础镜像
docker pull --platform=linux/arm64 debian:12.5
确认下载的debian 基础镜像 架构符合预期: arm64
当前打包镜像的是macbook m2芯片机器:
U-4G77XXWF-1921:pg15 digoal$ arch
arm64
下载的debian 基础镜像 架构符合预期: arm64 , 因为我们目标是打包 arm64 的镜像
U-4G77XXWF-1921:pg15 digoal$ docker image inspect debian:12.5|grep Archi
"Architecture": "arm64",
6.2、制作postgresql 15镜像:
cd ~/pg15
docker build --platform=linux/arm64 -t="digoal/pg15:with_exts_arm64" . 2>&1 | tee ./logs/build.log
# docker build --platform=linux/arm64 -t="digoal/pg15:with_exts_arm64" --no-cache . 2>&1 | tee ./logs/build.log
仔细检查是否有错误并解决, 例如:
grep Error ./logs/build.log
grep -i fail ./logs/build.log
grep -i fatal ./logs/build.log
grep ERROR ./logs/build.log
grep "ERR\!" ./logs/build.log
grep "E: " logs/build.log
grep error ./logs/build.log | grep -v "\-Werror"
修复问题后, 可以使用build好的镜像启动容器测试.
docker run -d -it -P --cap-add=SYS_PTRACE --cap-add SYS_ADMIN --privileged=true --name pg --shm-size=1g digoal/pg15:with_exts_arm64
docker exec -ti pg bash
psql
7、推送镜像到阿里云个人镜像服务
参考:
7.1、 将镜像推送到Registry
docker login --username=dig***@126.com registry.cn-hangzhou.aliyuncs.com
docker tag [ImageId] registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:[镜像版本号]
docker push registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:[镜像版本号]
根据实际镜像信息替换示例中的[ImageId]和[镜像版本号]参数, 例如:
docker tag d9ed34ee3c95 registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:pg15_with_exts_arm64
docker push registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:pg15_with_exts_arm64
7.2、 从阿里云Docker public Registry拉取image不需要login.
docker pull registry.cn-hangzhou.aliyuncs.com/digoal/opensource_database:pg15_with_exts_arm64
和这里一样:
amd64芯片参考:
