[lldb] Building the documentation should not depend on libLLDB

[hwhsu1231]

Question

Is it necessary for the lldb-python-doc-package target to depend on the lldb-python target?

What happened...

Hello, LLVM Maintenance Team.

Recently, I attempted to build the documentation for the LLVM project. During the process, I noticed that building the LLDB documentation consumes a significant amount of time.

To minimize the time spent on the build, I opted to use clangdev installed via Conda to skip the step of building the Clang project from sources. Below are the commands I used for llvmorg-19.1.6 tag and the complete log output:

Click to expand the commands

git clone --branch=llvmorg-19.1.6 --depth=1 https://github.com/llvm/llvm-project.git llvm-19.1.6
cd llvm-19.1.6
conda create --prefix ./.venv --yes
conda activate ./.venv
conda install gcc gxx binutils ninja clangdev=19.1.6 python=3.12 doxygen graphviz perl swig --channel conda-forge --yes
export LANG=en_US.UTF-8
export PYTHONNOUSERSITE=1
pip install -r llvm/docs/requirements.txt
export CMAKE_PROGRAM_PATH=$(pwd)/.venv/bin
export CMAKE_INCLUDE_PATH=$(pwd)/.venv/include
export CMAKE_LIBRARY_PATH=$(pwd)/.venv/lib
export CMAKE_INSTALL_PREFIX=$(pwd)/.venv
cmake -B ./build -S ./llvm -G Ninja -D CMAKE_BUILD_TYPE=Release -D LLVM_ENABLE_PROJECTS="clang;lldb" -D LLVM_ENABLE_SPHINX=ON -D LLDB_TEST_COMPILER=clang
cmake --build ./build --target docs-lldb-html --parallel 4
^C

log-build-19.1.6-lldb-docs.txt

From the log output, we can see that nearly 4,700 steps are required to complete the final build of the docs-lldb-html target:

(/home/hwhsu1231/Repo/testing/llvm-19.1.6/.venv) hwhsu1231@vb-kubuntu:~/Repo/testing/llvm-19.1.6$ cmake --build ./build --target docs-lldb-html --parallel 4
[15/4754] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/Base64.cpp.o^C
(/home/hwhsu1231/Repo/testing/llvm-19.1.6/.venv) hwhsu1231@vb-kubuntu:~/Repo/testing/llvm-19.1.6$

The reason seems to lie in the following code (the lldb-python target must be built before building the lldb-python-doc-package target):

llvm-project/lldb/docs/CMakeLists.txt

Line 37 in 7ec139a

add_dependencies(lldb-python-doc-package swig_wrapper_python lldb-python)

Experiment

To verify my hypothesis, I conducted the following 2 experiments:

Experiment 1

First, when building the LLDB documentation for the llvmorg-18.1.8 tag, it did not take as much time. This is because, at that point, lldb-python-doc-package did not depend on lldb-python:

llvm-project/lldb/docs/CMakeLists.txt

Line 30 in 3b5b5c1

add_dependencies(lldb-python-doc-package swig_wrapper_python)

Click to expand the commands

git clone --branch=llvmorg-18.1.8 --depth=1 https://github.com/llvm/llvm-project.git llvm-18.1.8
cd llvm-18.1.8
conda create --prefix ./.venv --yes
conda activate ./.venv
conda install gcc gxx binutils ninja clangdev=18.1.8 python=3.12 doxygen graphviz perl swig --channel conda-forge --yes
export LANG=en_US.UTF-8
export PYTHONNOUSERSITE=1
pip install -r llvm/docs/requirements.txt
export CMAKE_PROGRAM_PATH=$(pwd)/.venv/bin
export CMAKE_INCLUDE_PATH=$(pwd)/.venv/include
export CMAKE_LIBRARY_PATH=$(pwd)/.venv/lib
export CMAKE_INSTALL_PREFIX=$(pwd)/.venv
cmake -B ./build -S ./llvm -G Ninja -D CMAKE_BUILD_TYPE=Release -D LLVM_ENABLE_PROJECTS="clang;lldb" -D LLVM_ENABLE_SPHINX=ON -D LLDB_TEST_COMPILER=clang
cmake --build ./build --target docs-lldb-html --parallel 4
firefox build/tools/lldb/docs/html/index.html

log-exp1-build-18.1.8-lldb-docs.txt

Experiment 2

Next, before building the LLDB documentation for the llvmorg-19.1.6 tag, I removed the lldb-python dependency target using the sed command:

sed -i 's/add_dependencies(lldb-python-doc-package swig_wrapper_python lldb-python)/add_dependencies(lldb-python-doc-package swig_wrapper_python)/' ./lldb/docs/CMakeLists.txt

Click to expand the commands

git clone --branch=llvmorg-19.1.6 --depth=1 https://github.com/llvm/llvm-project.git llvm-19.1.6
cd llvm-19.1.6
conda create --prefix ./.venv --yes
conda activate ./.venv
conda install gcc gxx binutils ninja clangdev=19.1.6 python=3.12 doxygen graphviz perl swig --channel conda-forge --yes
export LANG=en_US.UTF-8
export PYTHONNOUSERSITE=1
pip install -r llvm/docs/requirements.txt
export CMAKE_PROGRAM_PATH=$(pwd)/.venv/bin
export CMAKE_INCLUDE_PATH=$(pwd)/.venv/include
export CMAKE_LIBRARY_PATH=$(pwd)/.venv/lib
export CMAKE_INSTALL_PREFIX=$(pwd)/.venv
sed -i 's/add_dependencies(lldb-python-doc-package swig_wrapper_python lldb-python)/add_dependencies(lldb-python-doc-package swig_wrapper_python)/' ./lldb/docs/CMakeLists.txt
cmake -B ./build -S ./llvm -G Ninja -D CMAKE_BUILD_TYPE=Release -D LLVM_ENABLE_PROJECTS="clang;lldb" -D LLVM_ENABLE_SPHINX=ON -D LLDB_TEST_COMPILER=clang
cmake --build ./build --target docs-lldb-html --parallel 4
firefox build/tools/lldb/docs/html/index.html

log-exp2-build-19.1.6-lldb-docs.txt

As a result, we can see that no critical errors occurred, and the build process required only 5 steps to complete.

Conclusion

If building the lldb-python target does not affect the LLDB documentation build, I would suggest removing the lldb-python dependency from the lldb-python-doc-package target in LLDB. This change should significantly reduce the time required to build the LLDB documentation.

[llvmbot]

@llvm/issue-subscribers-lldb

Author: Haowei Hsu (hwhsu1231)

## Question

Is it necessary for the lldb-python-doc-package target to depend on the lldb-python target?

What happened...

Hello, LLVM Maintenance Team.

Recently, I attempted to build the documentation for the LLVM project. During the process, I noticed that building the LLDB documentation consumes a significant amount of time.

To minimize the time spent on the build, I opted to use clangdev installed via Conda to skip the step of building the Clang project from sources. Below are the commands I used for llvmorg-19.1.6 tag and the complete log output:

<details><summary>Click to expand the commands</summary>

git clone --branch=llvmorg-19.1.6 --depth=1 https://github.com/llvm/llvm-project.git llvm-19.1.6
cd llvm-19.1.6
conda create --prefix ./.venv --yes
conda activate ./.venv
conda install gcc gxx binutils ninja clangdev=19.1.6 python=3.12 doxygen graphviz perl swig --channel conda-forge --yes
export LANG=en_US.UTF-8
export PYTHONNOUSERSITE=1
pip install -r llvm/docs/requirements.txt
export CMAKE_PROGRAM_PATH=$(pwd)/.venv/bin
export CMAKE_INCLUDE_PATH=$(pwd)/.venv/include
export CMAKE_LIBRARY_PATH=$(pwd)/.venv/lib
export CMAKE_INSTALL_PREFIX=$(pwd)/.venv
cmake -B ./build -S ./llvm -G Ninja -D CMAKE_BUILD_TYPE=Release -D LLVM_ENABLE_PROJECTS="clang;lldb" -D LLVM_ENABLE_SPHINX=ON -D LLDB_TEST_COMPILER=clang
cmake --build ./build --target docs-lldb-html --parallel 4
^C

</details>

log-build-19.1.6-lldb-docs.txt

From the log output, we can see that nearly 4,700 steps are required to complete the final build of the docs-lldb-html target:

(/home/hwhsu1231/Repo/testing/llvm-19.1.6/.venv) hwhsu1231@<!-- -->vb-kubuntu:~/Repo/testing/llvm-19.1.6$ cmake --build ./build --target docs-lldb-html --parallel 4
[15/4754] Building CXX object lib/Support/CMakeFiles/LLVMSupport.dir/Base64.cpp.o^C
(/home/hwhsu1231/Repo/testing/llvm-19.1.6/.venv) hwhsu1231@<!-- -->vb-kubuntu:~/Repo/testing/llvm-19.1.6$

The reason seems to lie in the following code (the lldb-python target must be built before building the lldb-python-doc-package target):

llvm-project/lldb/docs/CMakeLists.txt

Line 37 in 7ec139a

add_dependencies(lldb-python-doc-package swig_wrapper_python lldb-python)

Experiment

To verify my hypothesis, I conducted the following 2 experiments:

Experiment 1

First, when building the LLDB documentation for the llvmorg-18.1.8 tag, it did not take as much time. This is because, at that point, lldb-python-doc-package did not depend on lldb-python:

llvm-project/lldb/docs/CMakeLists.txt

Line 30 in 3b5b5c1

add_dependencies(lldb-python-doc-package swig_wrapper_python)

<details><summary>Click to expand the commands</summary>

git clone --branch=llvmorg-18.1.8 --depth=1 https://github.com/llvm/llvm-project.git llvm-18.1.8
cd llvm-18.1.8
conda create --prefix ./.venv --yes
conda activate ./.venv
conda install gcc gxx binutils ninja clangdev=18.1.8 python=3.12 doxygen graphviz perl swig --channel conda-forge --yes
export LANG=en_US.UTF-8
export PYTHONNOUSERSITE=1
pip install -r llvm/docs/requirements.txt
export CMAKE_PROGRAM_PATH=$(pwd)/.venv/bin
export CMAKE_INCLUDE_PATH=$(pwd)/.venv/include
export CMAKE_LIBRARY_PATH=$(pwd)/.venv/lib
export CMAKE_INSTALL_PREFIX=$(pwd)/.venv
cmake -B ./build -S ./llvm -G Ninja -D CMAKE_BUILD_TYPE=Release -D LLVM_ENABLE_PROJECTS="clang;lldb" -D LLVM_ENABLE_SPHINX=ON -D LLDB_TEST_COMPILER=clang
cmake --build ./build --target docs-lldb-html --parallel 4
firefox build/tools/lldb/docs/html/index.html

</details>

log-exp1-build-18.1.8-lldb-docs.txt

Experiment 2

Next, before building the LLDB documentation for the llvmorg-19.1.6 tag, I removed the lldb-python dependency target using the sed command:

sed -i 's/add_dependencies(lldb-python-doc-package swig_wrapper_python lldb-python)/add_dependencies(lldb-python-doc-package swig_wrapper_python)/' ./lldb/docs/CMakeLists.txt

<details><summary>Click to expand the commands</summary>

git clone --branch=llvmorg-19.1.6 --depth=1 https://github.com/llvm/llvm-project.git llvm-19.1.6
cd llvm-19.1.6
conda create --prefix ./.venv --yes
conda activate ./.venv
conda install gcc gxx binutils ninja clangdev=19.1.6 python=3.12 doxygen graphviz perl swig --channel conda-forge --yes
export LANG=en_US.UTF-8
export PYTHONNOUSERSITE=1
pip install -r llvm/docs/requirements.txt
export CMAKE_PROGRAM_PATH=$(pwd)/.venv/bin
export CMAKE_INCLUDE_PATH=$(pwd)/.venv/include
export CMAKE_LIBRARY_PATH=$(pwd)/.venv/lib
export CMAKE_INSTALL_PREFIX=$(pwd)/.venv
sed -i 's/add_dependencies(lldb-python-doc-package swig_wrapper_python lldb-python)/add_dependencies(lldb-python-doc-package swig_wrapper_python)/' ./lldb/docs/CMakeLists.txt
cmake -B ./build -S ./llvm -G Ninja -D CMAKE_BUILD_TYPE=Release -D LLVM_ENABLE_PROJECTS="clang;lldb" -D LLVM_ENABLE_SPHINX=ON -D LLDB_TEST_COMPILER=clang
cmake --build ./build --target docs-lldb-html --parallel 4
firefox build/tools/lldb/docs/html/index.html

</details>

log-exp2-build-19.1.6-lldb-docs.txt

As a result, we can see that no critical errors occurred, and the build process required only 5 steps to complete.

Conclusion

If building the lldb-python target does not affect the LLDB documentation build, I would suggest removing the lldb-python dependency from the lldb-python-doc-package target in LLDB. This change should significantly reduce the time required to build the LLDB documentation.

[DavidSpickett]

First, when building the LLDB documentation for the llvmorg-18.1.8 tag, it did not take as much time.

59f4267

Went in in 19.x.

As a result, we can see that no critical errors occurred, and the build process required only 5 steps to complete.
If building the lldb-python target does not affect the LLDB documentation build

I don't know if the documentation build is a superset of the website content, @medismailben?

Was there something about the scripting extensions docs that meant they needed this dependency, vs. the API documentation that, IIRC, was on the website/part of the docs before this change.

[JDevlieghere]

The only thing we should depend on is on the SWIG wrapper (lldb.py) which only requires parsing the API headers. We shouldn't need to build all of libLLDB and at the time we added this, we were very careful about that. I suspect someone changed some dependency that unintentionally pulled in all of libLLDB rather than just the bindings.

[JDevlieghere]

I've renamed the issue to be more generic.