Update on the rust port

[faho]

As asked by @PoignardAzur:

some people over at r/rust are curious how the rewrite is going?

In short: We're "mostly" done. Quantifying that isn't all that easy because there are separate measurements - do you use lines or bytes, do you use rust vs C++ or do you use the amount of C++ that was removed?

Depending on how you count, we're at anywhere between 60% (c++ sloc removed) and 74% (rust sloc vs c++ sloc) done.

And a lot of that is scaffolding we've duplicated that can be removed once the last caller is gone, so you get a bit of a domino effect. E.g. we have already translated wgetopt, but kept the C++ version around until the last caller is translated, which is simpler than adding translation gunk to all the C++ callers.

What is left to do?

The last big component that remains to be translated is the reader with the input system. This is 3.5k for the reader plus 1.3k for the screen handling plus 1k line for input plus 700 lines for the pager. This is pretty tightly coupled.

After that the rest is some stragglers and the de-C++ifying of the build system - removing cmake, getting cargo to do everything we want (cmake still substantially drives the build, and we need e.g. cargo to build and install the documentation and completion files. this isn't just compiling a bunch of rust, we have a lot of auxilliary stuff!)

This isn't really the sort of thing drive-by contributions are good for, I'm afraid.

---

Alright, that's the update, now for answering some questions / correcting some misconceptions I've seen in what I assume is this reddit thread:

1. We will be removing all of the C++.
2. A windows port is not planned, and has never been planned. C++ is also perfectly well-supported on Windows, so rust doesn't really help here. The language has never been the problem with a windows port, the problem is that windows is not unix!
3. We won't be changing the name or the mascot. We probably won't add "written in rust" to the tag line (but we might mention it on https://fishshell.com/)
4. We have no real idea what the performance of the finished port is going to be. Early results are encouraging, but it is entirely possible this will be ~20% slower in some cases. Even once the initial port is done, the project won't be. There's a loooot to do.

[PoignardAzur]

We won't be changing the name or the mascot.

But but but turbofish!

[itslychee]

gotta go fast 💨

[Xiretza]

Honestly that wouldn't be a terrible codename for the initial Rust release, if fish did those ;)

[Sir-Photch]

but it is entirely possible this will be ~20% slower in some cases

"turbo"fish

[faho]

Another thing:

We plan on not doing any partial-rust release.

That means we would be doing e.g. fish 4.0 as fully rust and zero C++, and I think, contrary to what we usually do that warrants a beta. (Ordinarily we've stopped doing betas and release candidates because they simply don't get any testing).

We also still want to do a 3.7.0 in-between release that is still purely C++, so we have a better jumping off point for platforms that can't handle the rust version. It would be 3.6.1 with some neat changes backported.

[faho]

3.7.0 is built in the Integration_3.7.0 branch in case you want to see what's happening there - including the (preliminary) CHANGELOG.

Highlights:

• A timeout for key combinations, useful for binding "kj" or "jj" to switching mode in vi-mode
• Usability improvements with the history-pager (ctrl-r) - e.g. it'll start with the existing search if you decide to switch from up-arrow to ctrl-r
• Some rendering glitches are fixed
• Faster globs

[Berrysoft]

You have done a really hard work to port the code from C++ to Rust. As a user of fish, I admire your perseverance.

I'm also a Windows user. I use fish with MSYS2 environment, which is a fork of cygwin. Fish ran really well on it because it provides POSIX utilities. However, Rust doesn't provide a cygwin or MSYS2 target, which means fish cannot be compiled target cygwin or MSYS2 after rewritten by Rust. The language was not a problem with a Windows port, but it will be.

I'm just a user, so this comment is just a reminder, that the rewriting work might block some fish users on Windows to update their favorite shell. I'm not blaming the work or the language. Anyway, Windows is not Unix:)

Thanks again for all of your hard work.

[Berrysoft]

Because if we only want to compile fish to cygwin, an easier way is to cross compile with an *-msvc rustc or *-gnu rustc. They don't require an LLVM on cygwin.

[RReverser]

Rust itself still ships with LLVM in either scenario. Those targets only change which standard libraries your binary is linked against - the native Windows ones or the GNU ones provided by MSYS2.

[BatmanAoD]

It also changes binary codegen, but yes, LLVM is currently required for all stable toolchains.

[Berrysoft]

Yes, and LLVM always support targeting cygwin. Rustc cannot be compiled targeting cygwin because the LLVM version cygwin distributed is a little old. An *-msvc rustc is enough to create a cygwin exe. If the std and other rust crates fixed, I believe we can compile fish targeting cygwin.

[krobelus]

These are the remaining files:

src/fish_indent_common.cpp
src/fish_indent.cpp
src/fish_key_reader.cpp
src/builtins/bind.cpp
src/builtins/commandline.cpp
src/fish_tests.cpp
src/pager.cpp
src/reader.cpp

[adaszko]

Sorry if this is a tiny bit off-topic. I'm the author complgen -- a tool that (hopefully) makes writing shell completions easier, faster and less error-prone. It shines especially if you're writing a completion script for a command with subcommands, like git(1) (git fe<TAB>, git pu<TAB>) where it frees you from maintaining a state machine by hand in shell script code. The tool is based on shell word-based regular grammars which I find to be adequate for at least 99% of real-life shell completions. The remaining 1% is for tools like find(1) whose grammar is non-regular due to its parenthesis use. Even in those cases complgen is usable and useful, just not 100% accurate. My ultimate goal for the best UX is if all shells included something like complgen out of the box, so I want to use this occasion of rewriting fish in Rust to probe your thoughts on this. I realise a rewrite is not meant to add new features. I'd be happy to clear up any aspects of how complgen works.

[faho]

Sorry if this is a tiny bit off-topic

It is off-topic, yes. You're basically asking for an unrelated enhancement.

---

git fe, git pu

That step isn't really the one we have a problem with.

It's the next - what do you complete for git add <TAB>? And the answer there is complex, and basically requires code, and that code will be different between fish and zsh.

---

Anyway, none of this has to do with what programming language fish uses. I'm going to say that I don't think we are interested in adding a second way to write completions at this time.

[adaszko]

git fe, git pu
That step isn't really the one we have a problem with.
It's the next - what do you complete for git add ? And the answer there is complex, and basically requires code, and that code will be different between fish and zsh.

I may have given an unrepresentative example above. git add <TAB> is exactly the type of cases complgen solves. As I mentioned, there's a state machine being maintained automatically.