build lineageos

huataihuang · Nov 1, 2023 · dc65023 · dc65023
1 parent a3d35d9
commit dc65023
Show file tree

Hide file tree

Showing 19 changed files with 217 additions and 4 deletions.
diff --git a/source/android/build/build_lineageos_20_pixel_4.rst b/source/android/build/build_lineageos_20_pixel_4.rst
@@ -7,7 +7,11 @@
 准备工作
 ==========
 
-- ref:`android_build_env_ubuntu`
+- :ref:`android_build_env_ubuntu`
+
+.. note::
+
+   编译LineageOS需要惊人的磁盘空间，按照官方文档参考，linage-18.1就需要300GB空间，更高版本要求的空间更多...
 
 安装仓库
 ==========
@@ -47,7 +51,26 @@
 
 .. note::
 
-   访问GitHub仓库可能受到GFW干扰，所以需要采用 :ref:`git_proxy`
+   访问GitHub仓库可能受到GFW干扰，所以需要采用:
+
+   - :ref:`git_proxy`
+   - :ref:`curl_proxy`
+
+   不过，我实践还是遇到一个困难 :ref:`git_proxy` 使用HTTP/HTTPS代理时需要使用 :ref:`git-openssl` 来解决 :ref:`squid_socks_peer` 出现的TLS连接报错:
+
+   .. literalinclude:: ../../devops/git/git-openssl/git_tls_connection_err
+      :caption: git HTTPS代理访问 ``googlesource`` 报错 TLS连接中断
+      :emphasize-lines: 4
+
+   采用 :ref:`git-openssl` 解决
+
+   此外，使用 OpenSSL 的 :ref:`curl` 也需要重新编译最新版本 :ref:`compile_curl_ssl` ，以便解决OpenSSL 3.0.x 的 ``unexpected EOF`` 报错
+
+.. warning::
+
+   **我最后放弃了通过代理翻墙方式来同步仓库** 原因是 ``实在难以找到合适的能够在墙内畅通访问的VPS`` (真是非常非常恶劣的网络环境)
+
+   实际采用的方式是直接在墙外租用一个小型VPS进行源代码同步，然后通过 :ref:`rsync` 同步到墙内高性能服务器上进行编译(或许可以部署一个持续集成环境?)
 
 .. note::
 
@@ -56,7 +79,7 @@
    - ``-j 4`` 表示并发4个同步线程(连接)
    - ``-c`` 表示 ``repo`` 值同步当前分支而不是GitHub上该仓库的所有分支
 
-   LineageOS 建议使用默认配置，不过我发现由于翻墙网络非常缓慢，适当增加同步并发可以加快同步。例如 ``repo sync -j 20``
+   LineageOS 建议使用默认配置，不过我发现由于翻墙网络非常缓慢，适当增加同步并发可以加快同步。例如 ``repo sync -j 40`` (按照服务器的cpu core数量调整)
 
 - 准备设备特定代码:
 

diff --git a/source/android/build/build_lineageos_20_pixel_4/repo_sync b/source/android/build/build_lineageos_20_pixel_4/repo_sync
@@ -3,7 +3,7 @@ mkdir -p ~/android/lineage/
 cd ~/android/lineage/
 
 # 初始化仓库(这里需要根据设备对应的代码分支):
-repo init -u https://github.com/LineageOS/android.git -b lineage-20.0 --git-lfs
+repo init -u https://github.com/LineageOS/android.git -b lineage-20.0
 
 # 代码同步
 repo sync
diff --git a/source/devops/git/git-openssl.rst b/source/devops/git/git-openssl.rst
@@ -0,0 +1,55 @@
+.. _git-openssl:
+
+====================
+使用OpenSSL的git
+====================
+
+当我采用 :ref:`git_proxy` 在 :ref:`squid_socks_peer` 后面通过代理访问Android源代码仓库 :ref:`build_lineageos_20_pixel_4` 遇到一个持续报错:
+
+.. literalinclude:: git-openssl/git_tls_connection_err
+   :caption: git HTTPS代理访问 ``googlesource`` 报错 TLS连接中断
+   :emphasize-lines: 4
+
+这个问题看起来是Ubuntu ``gnutls_handshake`` 解决方案的一个问题，需要通过重新编译 ``git`` 来解决(也就是采用 ``libcurl-openssl-dev`` 替代 ``gnutls`` )
+
+- 安装编译环境:
+
+.. literalinclude:: git-openssl/gt_build_dependencies
+   :caption: 安装git编译依赖环境
+
+- 修改 ``/etc/apt/sources.list`` 将源代码仓库激活(默认没有激活或配置):
+
+.. literalinclude:: git-openssl/sources.list
+   :caption: 配置apt源代码源
+   :emphasize-lines: 2,4,6
+
+- 更新仓库索引然后安装 ``git`` 源代码:
+
+.. literalinclude:: git-openssl/apt_source_git
+   :caption: 更新仓库索引然后安装 ``git`` 源代码
+
+- 安装 ``libcurl`` :
+
+.. literalinclude:: git-openssl/apt_libcurl
+   :caption: 安装 libcurl
+
+- 进入git源代码目录，修改2个文件，然后重新编译git:
+
+.. literalinclude:: git-openssl/recompile_git_with_openssl
+   :caption: 修订配置后重新编译git with openssl
+
+- 然后进入上级目录安装编译后的deb包:
+
+.. literalinclude:: git-openssl/pkg_install_git_with_openssl
+   :caption: 安装编译后的deb包
+
+安装输出信息:
+
+.. literalinclude:: git-openssl/pkg_install_git_with_openssl_output
+   :caption: 安装编译后的git包输出信息
+
+参考
+======
+
+- `GnuTLS recv error (-110): The TLS connection was non-properly terminated <https://stackoverflow.com/questions/52529639/gnutls-recv-error-110-the-tls-connection-was-non-properly-terminated>`_
+- `git-openssl-shellscript (GitHub) <https://github.com/niko-dunixi/git-openssl-shellscript>`_
diff --git a/source/devops/git/git-openssl/apt_libcurl b/source/devops/git/git-openssl/apt_libcurl
@@ -0,0 +1 @@
+sudo apt-get install libcurl4-openssl-dev
diff --git a/source/devops/git/git-openssl/apt_source_git b/source/devops/git/git-openssl/apt_source_git
@@ -0,0 +1,9 @@
+sudo apt update
+mkdir ~/git-rectify
+cd ~/git-rectify
+
+# 获取git源代码
+sudo apt-get source git
+
+# 安装git编译依赖
+sudo apt-get build-dep git
diff --git a/source/devops/git/git-openssl/git_tls_connection_err b/source/devops/git/git-openssl/git_tls_connection_err
@@ -0,0 +1,6 @@
+platform/external/ppp:
+fatal: unable to access 'https://android.googlesource.com/platform/external/ppp/': gnutls_handshake() failed: The TLS connection was non-properly terminated.
+platform/external/ppp: sleeping 4.0 seconds before retrying
+fatal: unable to access 'https://android.googlesource.com/platform/external/ppp/': GnuTLS recv error (-110): The TLS connection was non-properly terminated.
+error: Cannot fetch platform/external/rust/crates/ash from https://android.googlesource.com/platform/external/rust/crates/ash
+error: Cannot fetch platform/external/ppp from https://android.googlesource.com/platform/external/ppp
diff --git a/source/devops/git/git-openssl/gt_build_dependencies b/source/devops/git/git-openssl/gt_build_dependencies
@@ -0,0 +1 @@
+sudo apt-get install build-essential fakeroot dpkg-dev
diff --git a/source/devops/git/git-openssl/pkg_install_git_with_openssl b/source/devops/git/git-openssl/pkg_install_git_with_openssl
@@ -0,0 +1,2 @@
+cd .. # is in the ~/git-rectify directory
+sudo dpkg -i git_2.34.1-1ubuntu1.10_amd64.deb
diff --git a/source/devops/git/git-openssl/pkg_install_git_with_openssl_output b/source/devops/git/git-openssl/pkg_install_git_with_openssl_output
@@ -0,0 +1,4 @@
+(Reading database ... 39077 files and directories currently installed.)
+Preparing to unpack git_2.34.1-1ubuntu1.10_amd64.deb ...
+Unpacking git (1:2.34.1-1ubuntu1.10) over (1:2.34.1-1ubuntu1.10) ...
+Setting up git (1:2.34.1-1ubuntu1.10) ...
diff --git a/source/devops/git/git-openssl/recompile_git_with_openssl b/source/devops/git/git-openssl/recompile_git_with_openssl
@@ -0,0 +1,4 @@
+cd git-2.34.1
+vim ./debian/control # Modify libcurl4-gnutls-dev to libcurl4-openssl-dev
+vim ./debian/rules # delete the entire line of TEST=test
+sudo dpkg-buildpackage -rfakeroot -b
diff --git a/source/devops/git/git-openssl/sources.list b/source/devops/git/git-openssl/sources.list
@@ -0,0 +1,6 @@
+deb http://archive.ubuntu.com/ubuntu jammy main universe
+deb-src  http://archive.ubuntu.com/ubuntu jammy main universe
+deb http://archive.ubuntu.com/ubuntu jammy-security main universe
+deb-src http://archive.ubuntu.com/ubuntu jammy-security main universe
+deb http://archive.ubuntu.com/ubuntu jammy-updates main universe
+deb-src http://archive.ubuntu.com/ubuntu jammy-updates main universe
diff --git a/source/devops/git/git_proxy.rst b/source/devops/git/git_proxy.rst
@@ -12,6 +12,12 @@ git配置代理
    :language: bash
    :caption: 全局配置git使用HTTP/HTTPS代理
 
+.. note::
+
+   当配置git使用HTTP/HTTPS代理时，会使得git采用HTTPS方式代替 :ref:`ssh` 方式来访问git仓库。我遇到一个问题是 :ref:`build_lineageos_20_pixel_4` 结合使用 :ref:`squid_socks_peer` 同步 ``googlesource`` 仓库报错:
+
+   解决的方法是独立编译 :ref:`git-openssl`
+
 - 配置 ``git`` 使用socks5代理:
 
 .. literalinclude:: git_proxy/git_config_http.proxy_socks5

diff --git a/source/devops/git/index.rst b/source/devops/git/index.rst
@@ -12,6 +12,7 @@ git
    git_merge_fix_conflicts.rst
    git_config.rst
    git_proxy.rst
+   git-openssl.rst
    reduce_git_repo_size.rst
    git-flow.rst
 

diff --git a/source/linux/ubuntu_linux/admin/apt.rst b/source/linux/ubuntu_linux/admin/apt.rst
@@ -155,6 +155,42 @@ apt hold保持包不更新
 
    apt-mark unhold kubelet kubeadm kubectl
 
+apt源代码安装和编译
+=====================
+
+我在 :ref:`build_lineageos_20_pixel_4` 遇到一个需要编译 ``git with openssl`` 来解决通过 :ref:`squid_socks_peer` 使用 :ref:`git_proxy` 的问题。这个案例采用了ubuntu发行版提供的软件包源代码编译安装，步骤是可借鉴的:
+
+- 安装编译环境:
+
+.. literalinclude:: ../../../devops/git/git-openssl/gt_build_dependencies
+   :caption: 安装git编译依赖环境
+
+- 修改 ``/etc/apt/sources.list`` 将源代码仓库激活(默认没有激活或配置):
+
+.. literalinclude:: ../../../devops/git/git-openssl/sources.list
+   :caption: 配置apt源代码源
+   :emphasize-lines: 2,4,6
+
+- 更新仓库索引然后安装 ``git`` 源代码:
+
+.. literalinclude:: ../../../devops/git/git-openssl/apt_source_git
+   :caption: 更新仓库索引然后安装 ``git`` 源代码
+
+- 安装 ``libcurl`` :
+
+.. literalinclude:: ../../../devops/git/git-openssl/apt_libcurl
+   :caption: 安装 libcurl
+
+- 进入git源代码目录，修改2个文件，然后重新编译git:
+
+.. literalinclude:: ../../../devops/git/git-openssl/recompile_git_with_openssl
+   :caption: 修订配置后重新编译git with openssl
+
+- 然后进入上级目录安装编译后的deb包:
+
+.. literalinclude:: ../../../devops/git/git-openssl/pkg_install_git_with_openssl
+   :caption: 安装编译后的deb包
+
 参考
 ========
 

diff --git a/source/web/curl/compile_curl_ssl.rst b/source/web/curl/compile_curl_ssl.rst
@@ -0,0 +1,34 @@
+.. _compile_curl_ssl:
+
+=======================
+编译OpenSSL支持的curl
+=======================
+
+在 :ref:`build_lineageos_20_pixel_4` 采用了自己编译的 :ref:`git-openssl` ，此时系统升级安装了 ``OpenSSL`` 的 3.0.x 版本，在 Ubuntu 22.04 LTS 上默认安装的 ``curl`` 是 ``7.81.0`` ，此时和 ``OpenSSL`` 的 3.0.x 版本一起工作会出现异常 ``unexpected eof while reading`` 报错:
+
+.. literalinclude:: compile_curl_ssl/curl_openssl_unexpected_eof_err
+   :caption: 在 Ubuntu 22.04 LTS 默认 ``curl`` 是 ``7.81.0`` 和 ``OpenSSL`` 3.0.x 版本异常报错
+   :emphasize-lines: 2,4
+
+.. note::
+
+   不过我还是不能确定是否真正解决了这个问题，看起来还是有同样的报错。翻墙网络不太稳定似乎也是有可能的
+
+解决方法是重新编译支持OpenSSL 3.0.x的最新版本curl:
+
+.. literalinclude:: compile_curl_ssl/build_curl
+   :caption: 编译OpenSSL支持的最新版本curl
+
+不过，此时运行curl下载https内容时候会有报错:
+
+.. literalinclude:: compile_curl_ssl/curl_ssl_err
+   :caption: 编译后curl访问https有 ``no version information available`` 报错
+
+这个问题参考 `libcurl.so.4 no version information available <https://serverfault.com/questions/696631/libcurl-so-4-no-version-information-available>`_ ::
+
+   sudo ln -fs /usr/lib/libcurl.so.4 /usr/local/lib/
+
+参考
+=======
+
+- `OpenSSL Error messages: error:0A000126:SSL routines::unexpected eof while reading <https://stackoverflow.com/questions/72627218/openssl-error-messages-error0a000126ssl-routinesunexpected-eof-while-readin>`_
diff --git a/source/web/curl/compile_curl_ssl/build_curl b/source/web/curl/compile_curl_ssl/build_curl
@@ -0,0 +1,15 @@
+apt remove curl
+apt purge curl
+apt-get update
+apt-get install -y libssl-dev autoconf libtool make
+cd /usr/local/src
+wget https://github.com/curl/curl/releases/download/curl-8_4_0/curl-8.4.0.tar.gz
+tar xfz curl-8.4.0.tar.gz
+cd curl-8.4.0
+./buildconf
+./configure --with-ssl 
+make
+sudo make install
+sudo cp /usr/local/bin/curl /usr/bin/curl
+sudo ldconfig
+curl -V
diff --git a/source/web/curl/compile_curl_ssl/curl_openssl_unexpected_eof_err b/source/web/curl/compile_curl_ssl/curl_openssl_unexpected_eof_err
@@ -0,0 +1,5 @@
+platform/external/rust/crates/grpcio-sys:
+fatal: unable to access 'https://android.googlesource.com/platform/external/rust/crates/grpcio-sys/': OpenSSL SSL_read: error:0A000126:SSL routines::unexpected eof while reading, errno 0
+platform/external/rust/crates/grpcio-sys: sleeping 4.0 seconds before retrying
+fatal: unable to access 'https://android.googlesource.com/platform/external/rust/crates/grpcio-sys/': error:0A000126:SSL routines::unexpected eof while reading
+error: Cannot fetch platform/external/rust/crates/grpcio-sys from https://android.googlesource.com/platform/external/rust/crates/grpcio-sys
diff --git a/source/web/curl/compile_curl_ssl/curl_ssl_err b/source/web/curl/compile_curl_ssl/curl_ssl_err
@@ -0,0 +1,4 @@
+platform/external/rust/crates/macaddr:
+/usr/lib/git-core/git-remote-https: /usr/local/lib/libcurl.so.4: no version information available (required by /usr/lib/git-core/git-remote-https)
+fatal: unable to access 'https://android.googlesource.com/platform/external/rust/crates/macaddr/': OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to android.googlesource.com:443
+platform/external/rust/crates/macaddr: sleeping 4.0 seconds before retrying
diff --git a/source/web/curl/index.rst b/source/web/curl/index.rst
@@ -11,6 +11,7 @@ curl
    curl_show_request_headers.rst
    curl_post_json.rst
    curl_https.rst
+   compile_curl_ssl.rst
 
 .. only::  subproject and html