Add Rust Formatters to Debugger (Vector, Map etc.) #2219

MrSarius · 2023-05-16T10:30:52Z

This PR adds LLDB formatters so that variables are human-readable when debugging Rust code in VS Code. This includes Tuple, Slice, String, Vector, Map, Enum etc.

Python in LLDB

For performing more complex calculations on debugger variables (as required in this PR), LLDB uses Python. This requires building LLDB with Python support, but there are some hurdles to overcome.

LLDB links to the Python C ABI, which is not compatible between minor versions of Python. Although a stable Python ABI exists, LLDB does not stick to this subset. Means we have to ensure that the exact version of Python that we are using in our pipeline is available on the user's machine. Even if the version is correct, LLDB cannot link against Python from different distributions (e.g. homebrew and python.org).

Therefore I decided to distribute a stand-alone Python version with LLDB. This solution enables high portability. This way, Ubuntu 20.04 and 22.04 can for example still be supported with the same build since glibc is statically linked in the Python build, also making it easier to support more operating systems in the future.

This approach also seems to be the GOTO solution in comparable cases (e.g. XCode also uses LLDB and distributes it with a own Python build with each version).

Swig3 MacOs

Inside the MacOs pipeline I had to downgrade from swig4 to swig3 because swig4 causes the LLDB build to fail. Perhaps this can be removed at a later date.

Tests

This was tested on MacOs13-aarch64 and Ubuntu22.04-x86.

I've also written unit and integration tests for my implementation, however for anything related to the VS Code extension there haven't been any tests at all yet, I'll open another PR for that.

Known Issues

Enum types are not displayed correctly. For example, for optionals LLDB resolves this type: core::option::Option<> instead of core::option::Option<&str>... (see screenshot). This must be an issue in the patched LLDB build, the formatters themselves work fine with native LLDB.

I'm currently investigating this, but it should not be related to the changes from this PR.

Test changes

Click me

Build iwasm with source debugging feature.

cd ${WAMR_ROOT}/product-mini/platforms/${YOUR_PLATFORM}
mkdir build && cd build
cmake .. -DWAMR_BUILD_DEBUG_INTERP=1
make

Note: On MacOS M1 environment, pass the additional -DWAMR_DISABLE_HW_BOUND_CHECK=1 cmake configuration.

Compile your rust file with debug symbols and no optimisation to avoid inlining.

rustc --target wasm32-wasi ./test.rs -g -C opt-level=0

Download the LLDB build (with python) for your platform from my pipeline and move it to the extension's resource folder.

cd ${WAMR_ROOT}/test-tools/wamr-ide/VSCode-Extension/resource/debug
wget ${LLDB_FOR_YOUR_PLATFORM}.tar.gz -O lldb.tar.gz
tar -xvf lldb.tar.gz
cp -R wamr-lldb/* ./${LLDB_FOR_YOUR_PLATFORM}

Execute iwasm with debug engine enabled.

product-mini/platforms/darwin/build/iwasm -g=127.0.0.1:1234 test.wasm

Open vs code in ${WAMR_ROOT}/test-tools/wamr-ide/VSCode-Extension, go to the debug tab and launch Launch Extension. In this new window, the extension is now enabled.

Set your breakpoints in vs code and launch the debugger with this launch.json

{
    "configurations": [
        {
            "type": "wamr-debug",
            "request": "attach",
            "name": "Attach Debugger",
            "stopOnEntry": true,
            "attachCommands": [
                "process connect -p wasm connect://127.0.0.1:1234"
            ]
        }
    ]
}

MrSarius · 2023-05-16T17:13:11Z

.github/workflows/build_wamr_lldb.yml

@@ -63,7 +63,8 @@ jobs:
      - name: install utils macos
        if: steps.lldb_build_cache.outputs.cache-hit != 'true' && contains(inputs.runner, 'macos')
        run: |
-          brew install swig cmake ninja libedit
+          brew remove swig


Is there a better way to remove a pre-installed dependency from a GitHub image?

As far as I know, there isn't a better way. I think removing the existing version and reinstalling the version needed is a solid way.

MrSarius · 2023-05-16T17:21:48Z

.github/workflows/build_wamr_lldb.yml

@@ -111,7 +112,7 @@ jobs:
            -DLLVM_INCLUDE_TESTS:BOOL=OFF \
            -DLLVM_ENABLE_BINDINGS:BOOL=OFF \
            -DLLVM_ENABLE_LIBXML2:BOOL=ON \
-            -DLLDB_ENABLE_PYTHON:BOOL=OFF \
+            -DLLDB_ENABLE_PYTHON:BOOL=ON \


Do we need to include a full version of Python along with the LLDB download or what is the common way to deal with this dependency when shipping LLDB binaries?

@wenyongh @lum1n0us
Any ideas on how to create a distributable version of LLDB with python scripting support?

lum1n0us · 2023-05-19T07:08:27Z

https://lldb.llvm.org/resources/build.html

IIUC, just need to enable LLDB_ENABLE_PYTHON

MrSarius · 2023-05-19T16:56:42Z

If I do so and execute the resulting binary on a different machine. I get this:

./lldb
dyld[17796]: Library not loaded: /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/Python
...

I think I have to bundle Python with LLDB, so that version and the path are the same.

lum1n0us · 2023-05-23T10:14:02Z

How about using the default configuration?
$ cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS="clang;lldb" ../llvm

test-tools/wamr-ide/VSCode-Extension/formatters/rust.py

wenyongh · 2023-06-11T03:23:32Z

.github/workflows/build_wamr_lldb.yml

-            -DLLVM_BUILD_BENCHMARKS:BOOL=OFF \
-            -DLLVM_BUILD_DOCS:BOOL=OFF \
-            -DLLVM_BUILD_EXAMPLES:BOOL=OFF  \
+            -DLLVM_BUILD_BENCHMARKS:BOOL=OFF -DLLVM_BUILD_DOCS:BOOL=OFF \


Had better put -DLLVM_BUILD_DOCS:BOOL=OFF on another line

wenyongh · 2023-06-11T03:25:22Z

.github/workflows/build_wamr_lldb.yml

            -DCMAKE_BUILD_TYPE:STRING="Release" \
-            -DCMAKE_EXPORT_COMPILE_COMMANDS=ON \
-            -DLLVM_ENABLE_PROJECTS="clang;lldb" \
+            -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DLLVM_ENABLE_PROJECTS="clang;lldb" \


Had better put -DLLVM_ENABLE_PROJECTS="clang;lldb" on another line

TianlongLiang · 2023-06-13T03:20:42Z

Hi, @MrSarius I think it's really cool to better format the container data structure in the debugger panel of VS Code. I wanted to give the new lldb a try, however, as a VS Code rookie I got stuck when writing launch.json. The only way I know how to use a customized lldb is to set miDebuggerPath but it requires an lldb-mi rather than lldb. Could you please help me out? It would be great if you can share your launch.json and what kind of extension and configuration type is needed(I am currently using cppdbg)

- remove redundant flags - improve formatting

MrSarius · 2023-06-13T15:39:26Z

Hi, @MrSarius I think it's really cool to better format the container data structure in the debugger panel of VS Code. I wanted to give the new lldb a try, however, as a VS Code rookie I got stuck when writing launch.json. The only way I know how to use a customized lldb is to set miDebuggerPath but it requires an lldb-mi rather than lldb. Could you please help me out? It would be great if you can share your launch.json and what kind of extension and configuration type is needed(I am currently using cppdbg)

Hey @TianlongLiang ,
I have added a "Test changes" section at the bottom of the PR. Hope that helps.

TianlongLiang · 2023-06-14T10:45:17Z

LGTM

This PR adds tests for #2219 by changing the `compilation_on_android_ubuntu.yml` workflow. The first run will take about two hours, since LLDB is built from scratch. Later, the build is cached and the whole job should not take more than three minutes. Core of the PR is an integration test that boots up vscode and lets it debug a test WASM file.

…2219) This PR adds LLDB formatters so that variables are human-readable when debugging Rust code in VS Code. This includes Tuple, Slice, String, Vector, Map, Enum etc. It also distributes a standalone Python version with LLDB. This solution enables high portability, so Ubuntu 20.04 and 22.04 can for example still be supported with the same build since glibc is statically linked in the Python build, also making it easier to support more operating systems in the future. Known Issues: Enum types are not displayed correctly. For more details, refer to: bytecodealliance#2219

This PR adds tests for bytecodealliance#2219 by changing the `compilation_on_android_ubuntu.yml` workflow. The first run will take about two hours, since LLDB is built from scratch. Later, the build is cached and the whole job should not take more than three minutes. Core of the PR is an integration test that boots up vscode and lets it debug a test WASM file.

MrSarius force-pushed the benjuri/add-debug-rust-formatters branch from 9ca4cb3 to 1b489de Compare May 16, 2023 17:09

MrSarius commented May 16, 2023

View reviewed changes

MrSarius force-pushed the benjuri/add-debug-rust-formatters branch 2 times, most recently from 0ee3579 to 37207c7 Compare May 30, 2023 09:41

MrSarius force-pushed the benjuri/add-debug-rust-formatters branch 4 times, most recently from 9fdf8e3 to 728fa44 Compare June 6, 2023 10:42

MrSarius marked this pull request as ready for review June 6, 2023 17:00

MrSarius force-pushed the benjuri/add-debug-rust-formatters branch from 728fa44 to c5de03c Compare June 6, 2023 17:03

wenyongh reviewed Jun 11, 2023

View reviewed changes

MrSarius force-pushed the benjuri/add-debug-rust-formatters branch 3 times, most recently from a8c2757 to 13f9de3 Compare June 12, 2023 15:21

MrSarius added 3 commits June 13, 2023 11:19

add rust formatters to debugger

540fb34

improve workflow file

67906f4

- remove redundant flags - improve formatting

add license identifier

e319f72

MrSarius force-pushed the benjuri/add-debug-rust-formatters branch from 13f9de3 to e319f72 Compare June 13, 2023 10:20

update package version

24fe1ae

MrSarius mentioned this pull request Jun 13, 2023

Add debugger extension tests #2292

Merged

eloparco approved these changes Jun 14, 2023

View reviewed changes

fix typos

0813209

wenyongh merged commit fca81fc into bytecodealliance:main Jun 14, 2023
1 check passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Rust Formatters to Debugger (Vector, Map etc.) #2219

Add Rust Formatters to Debugger (Vector, Map etc.) #2219

MrSarius commented May 16, 2023 •

edited

MrSarius May 16, 2023

TianlongLiang May 18, 2023 •

edited

MrSarius May 16, 2023 •

edited

MrSarius May 18, 2023

lum1n0us commented May 19, 2023

MrSarius commented May 19, 2023

lum1n0us commented May 23, 2023

wenyongh Jun 11, 2023

MrSarius Jun 12, 2023

wenyongh Jun 11, 2023

MrSarius Jun 12, 2023

TianlongLiang commented Jun 13, 2023

MrSarius commented Jun 13, 2023 •

edited

TianlongLiang commented Jun 14, 2023

Add Rust Formatters to Debugger (Vector, Map etc.) #2219

Add Rust Formatters to Debugger (Vector, Map etc.) #2219

Conversation

MrSarius commented May 16, 2023 • edited

Python in LLDB

Swig3 MacOs

Tests

Known Issues

Test changes

MrSarius May 16, 2023

Choose a reason for hiding this comment

TianlongLiang May 18, 2023 • edited

Choose a reason for hiding this comment

MrSarius May 16, 2023 • edited

Choose a reason for hiding this comment

MrSarius May 18, 2023

Choose a reason for hiding this comment

lum1n0us commented May 19, 2023

MrSarius commented May 19, 2023

lum1n0us commented May 23, 2023

wenyongh Jun 11, 2023

Choose a reason for hiding this comment

MrSarius Jun 12, 2023

Choose a reason for hiding this comment

wenyongh Jun 11, 2023

Choose a reason for hiding this comment

MrSarius Jun 12, 2023

Choose a reason for hiding this comment

TianlongLiang commented Jun 13, 2023

MrSarius commented Jun 13, 2023 • edited

TianlongLiang commented Jun 14, 2023

MrSarius commented May 16, 2023 •

edited

TianlongLiang May 18, 2023 •

edited

MrSarius May 16, 2023 •

edited

MrSarius commented Jun 13, 2023 •

edited