Ship a custom LLDB with Rust support

[alexcrichton]

I hear that @tromey has been doing some awesome work with Rust-specific support in LLDB, and we should enable easily shipping that work to users! This is intended to be a checklist/tracking issue of sorts of the work needed to be done to enable this.

• One of the first things we'll need to do is to add it to the Rust source tree (off by default). This will probably involve adding a submodule and adding rustbuild logic to build LLDB. This will currently, as of this writing, require syncing the LLDB work to get based off the same LLVM version we have in tree, presently the release_60 branch.
• Next up we'll want to confirm that LLDB works with sccache and is being cached properly when using sccache. The change here would basically be ensuring that --enable-ccache works for LLDB locally.
• After that we'll want to build LLDB on the dist bots, but not upload it yet. This'll help us evaluate impacts on cycle time, if any. The change here should mostly just be tweaking flags in travis/appveyor/dockerfiles
• Next we'd add rustbuild support for creating a new LLDB component that we'd publish on the dist bots. This would probably look similar to the support for RLS (ish).
• And finally we'd teach rustup.rs about the LLDB component, allowing rustup component add lldb (or whatever the name is), but this is much farther down the line!

[michaelwoerister]

cc me

[tromey]

#52716

[tromey]

The first item is done. I'm testing the second item now. I had thought that the third item would be covered by the .travis.yml change in the patch for the first item -- is that not so? I think the rustbuild bits are done too (?). I haven't looked into rustup.rs yet.

[alexcrichton]

@tromey I think we'll need to wait for a new nightly to see whether or not rustup works, at that point we should have a look at the manifest and it ay already work!

[tromey]

Thanks. Also, a local --enable-ccache build worked fine.

[michaelwoerister]

@tromey, I saw that dynamically linking to LLVM is disabled when shipping LLDB:

rust/src/bootstrap/native.rs

Line 175 in 142bb27

!want_lldb {

Is that a hard requirement? Because we might want to enable dynamic linking when compiling LLVM with ThinLTO (see #53245).

[tromey]

Is that a hard requirement?

I don't think so.

[tromey]

This reports that it didn't work: https://internals.rust-lang.org/t/rust-debugging-quest/6753/13?u=tromey

I haven't looked into it yet.

[tromey]

lldb isn't mentioned in channel-rust-nightly.toml, which I guess it should be. I don't know how this is made.

[cynecx]

@tromey I am not familiar with the code but if I follow the code correctly, there might be an issue here:

rust/src/tools/build-manifest/src/main.rs

Lines 461 to 467 in afd0a2f

	let version = match *self.cached_version(pkgname) {
	Some(ref version) => version.clone(),
	None => {
	println!("Skipping package {}", pkgname);
	return;
	}
	};

cached_version(...) is called with "lldb-preview":

rust/src/tools/build-manifest/src/main.rs

Lines 534 to 535 in afd0a2f

	} else if component == "lldb" || component == "lldb-preview" {
	&self.lldb_version

lldb_version is initialized here:

rust/src/tools/build-manifest/src/main.rs

Line 276 in afd0a2f

self.lldb_version = self.version("lldb", "x86_64-unknown-linux-gnu");

So I think the problem here is that call to version with x86_64-unknown-linux-gnu as target, it basically tries to extract the version of that particular target archive. And according to #52716, lldb is only built for macOS on nightly. Therefore the call version returns None and lldb_version is then None.

[tromey]

Thanks for your note. That's a good find!

[tromey]

I have the obvious patch here: https://github.com/tromey/rust/tree/lldb-manifest-fix - but I don't know how to test it locally.

[japaric]

I see that #52716 is building LLDB for macOS. What's the (long term) plan for Windows and Linux? Will we also ship LLDB on those two OSes? At some point I heard that we might ship GDB on Linux (because GDB is better, or maybe easier to ship, than LLDB on Linux?) and then that we'll probably not do that (because system GDB is good enough?).

For the embedded use case having a debugger with support for architectures other than x86 (notably ARM and RISCV) shipped with the compiler would be godsend as the user would have one less tool to install (e.g. arm-none-eabi-gdb) or at least the installation would be straightforward (rustup component add lldb-preview). Remote debugging of microcontrollers, which have non-x86 processors, is a must have for professional embedded development; making the setup process simpler would be a big win for embedded Rust.

In that regard, an lldb component would fit the bill as it has multiarch support by default, and a gdb component would too but only if it's compiled with --enable-targets=all or similar (Most (?) distros ship GDB with support only for the host architecture; that's why you usually need to install a second, prefixed GDB for embedded development). IME (from several months ago) GDB integrates much better with embedded debugging tools (e.g. OpenOCD) but I expect that LLDB will eventually catch up (this PR indicates progress in that area).

cc @dvc94ch

[tromey]

The main reason lldb is only built for macOS is that on other platforms, the Python dependency is hard to satisfy. On Windows I guess we'd have to ship Python. On Linux, ideally you'd use the system Python, but I think there is no compatibility across distros, due to the Python 2/3 schism, and perhaps also due to differences in filesystem layout (? I can't recall).

On Linux, there may be an additional difficulty setting the path for the system's separate debuginfo. However I haven't really looked to see what distros other than Fedora do here. It's possible there are also other dependency-related issues.

The upstream situations with gdb and lldb are different. gdb accepted all the Rust code, and I'm the maintainer for that, so it is very easy for us to add rust-related improvements there. The main issue with gdb is that it is on a twice-per-year release schedule, so there is some lag relative to rustc. This is one reason that it would be good to ship a gdb -- we could synchronize rust improvements better.

lldb, on the other hand, did not want the rust plugin until it has been proven. I put this info in the rust lldb wiki but here's the message: http://lists.llvm.org/pipermail/lldb-dev/2018-January/013213.html (IIRC there are some other notes that month including some discussion of rust specifically). So, we have to ship a fork and I have no concrete plans to upstream anything. In a sense this is good because it avoids the synchronization problem.

[tromey]

I sent the PR: #53452

[dvc94ch]

The main reason lldb is only built for macOS is that on other platforms, the Python dependency is hard to satisfy. On Windows I guess we'd have to ship Python. On Linux, ideally you'd use the system Python, but I think there is no compatibility across distros, due to the Python 2/3 schism, and perhaps also due to differences in filesystem layout (? I can't recall).

I think every distro supports both python2 and python3. What python simlinks to may be different. If you have a specific issue can you provide more info so that I can help debug it?

On Linux, there may be an additional difficulty setting the path for the system's separate debuginfo. However I haven't really looked to see what distros other than Fedora do here. It's possible there are also other dependency-related issues.

I think users can set this in their .gdbinit in their home directory. I'm sure we can figure this one out too...

Are there any concrete steps that can be taken to get lldb distributed with rust on linux?

[tromey]

If you have a specific issue can you provide more info so that I can help debug it?

If we ship an lldb that is linked against some system Python, then that version of Python has to exist on the user's machine. But AFAIK we can't influence whether this is true or not. Perhaps it's even possible to have an install without Python. Also, library paths differ between distros (IIRC Debian and Fedora handle multi-arch differently), though perhaps this could be worked around in rustup.

If the dependency problems can be solved somehow then we should also ship gdb on Linux. Less sure about shipping either debugger on Windows.

[dvc94ch]

How about building gdb statically? It seems to be supported https://stackoverflow.com/questions/9364685/how-i-can-static-build-gdb-from-source

[tromey]

How about building gdb statically?

Will that work if Python code loads a C extension? I wouldn't think so but I am not 100% certain.

[dvc94ch]

Will that work if Python code loads a C extension? I wouldn't think so but I am not 100% certain.

Is that necessary for the rust gdb part? I think that 99% of users will use a ide for debugging. I currently use a python gdb gui which is a huge improvement over just using the cli, but I'll be adding lldb/gdb support to https://github.com/theia-ide/theia-rust-extension after theia-ide/theia#2447 lands.

[tromey]

Is that necessary for the rust gdb part?

It isn't necessary in the sense that gdb will still work without it. However, I'd rather not limit what users can do. For example I use this facility all the time.

I currently use a python gdb gui

OOC what is it?

One idea I had is that maybe we could have rustup check for required dependencies before installing. Then, if the dependencies were missing, it could point users to a wiki page explaining how to install the dependencies for their distro. It isn't just Python, for best results gdb also needs zlib, expat, curses, and lzma (and maybe mpfr as well).

The second part of this idea would be to pick sufficiently old versions of these libraries to build against so that the dependencies can be satisfied on all the target distros, whatever those are. Also, we'd modify gdb to find the distro's separate debug directory and installed gdb python code.

At runtime the dependencies would be satisfied by having rustup (continue to) set LD_LIBRARY_PATH in a distro-specific way.

lldb would work similarly, though I think there are fewer dependencies, since it doesn't support some of the Linux-specific things that gdb does.

[dvc94ch]

I currently use a python gdb gui

OOC what is it?

https://github.com/cyrus-and/gdb-dashboard

[dvc94ch]

One idea I had is that maybe we could have rustup check for required dependencies before installing. Then, if the dependencies were missing, it could point users to a wiki page explaining how to install the dependencies for their distro. It isn't just Python, for best results gdb also needs zlib, expat, curses, and lzma (and maybe mpfr as well).

One problem with that is that some distros (fedora) don't provide symlinks to old libs. So if I compile a lib with a dependency on libreadline with travis which uses libreadline.so.6 the dep isn't found on fedora which only has libreadline.so.7. A possible solution would be to change the lib names to libreadline.so in the binary before distributing. I currently am working around this on my machine by adding the appropriate symlinks. Distributing software on Linux in a distro independent way is a PITA.

[tromey]

We'd have to survey the dependencies on the distros we care about to see if it's possible. Note that readline in particular needn't be an issue because gdb vendors it (distros tend to disable this but we don't have to).

[tromey]

Let's move the gdb discussion to #34457.

[tromey]

rustup PR is here: rust-lang/rustup.rs#1492

[tromey]

Latest problem is #54126.

[vadimcn]

Regarding Python dependency on Linux: I doubt that any distro will drop Python 2.7 before 2020, even if it might not be the default. So for the next year we could ship LLDB linked to Python 2.7.

Some time after that, Python 2 will start going extinct, so we'll need to switch to Python 3. Which is going to be a headache, because a new version comes every once in a while. At that point, one of the following needs to happen:

• LLDB adopts Rust plugin in the main tree,
• We provide several versions of rust-lldb, compatible with latest 2 or 3 Python versions,
• We bundle Python with rust-lldb,
• SWIG finally supports stable Python ABI, which will make it possible to use any Python version past 3.2 (some changes will also be needed on LLDB side as well, but those should be minimal).

[tromey]

I doubt that any distro will drop Python 2.7 before 2020, even if it might not be the default

One question is whether we can rely on it being installed. But maybe we could teach rustup to check, if it can't already (I don't know).

LLDB adopts Rust plugin in the main tree,

I don't think this will change anything here, because even if lldb accepts the Rust plugin, we will probably still want to ship lldb, so that we aren't blocked on making debuginfo improvements. There's an analogous case for shipping gdb, see #34457.

I'm generally agreed on the other points.

[crlf0710]

On http://lldb.llvm.org/build.html it mentions that python dependency(and some other dependencies) can be opt-out by adding -DLLDB_DISABLE_PYTHON=1 but i'm not sure how much it hurts functionality. Maybe someone can give it a try?

The full list:

-DLLDB_DISABLE_LIBEDIT=1
-DLLDB_DISABLE_CURSES=1
-DLLDB_DISABLE_PYTHON=1
-DLLVM_ENABLE_TERMINFO=0

[tromey]

but i'm not sure how much it hurts functionality

I think we want Python, because it is how the pretty-printers in rust-lldb work; and also Python support in lldb is required by the debuginfo tests.