Suggestion: Cache x --help so that it doesn't rebuild to show help message

[thenorili]

running x --help takes around 60s because it redownloads and rebuilds x every time. If building x created or updated a cached file containing the help message, one could avoid this process and read the most up-to-date help message whenever they like!

[thenorili]

Suggestion add'ly: the help message about the config toml is very helpful, but it'd be more helpful if it showed up when running x --help when folks are generally preparing to execute x for the first time!

help: consider running ./x.py setup or copying config.example.toml by running cp config.example.toml config.toml

[compiler-errors]

running x --help takes around 60s because it redownloads and rebuilds x every time.

This definitely doesn't happen to me -- this sounds like a bug (or maybe I'm missing something here?). x should only be downloading the beta toolchain and rebuilding bootstrap as needed.

[onur-ozkan]

running x --help takes around 60s because it redownloads and rebuilds x every time.

It doesn't happen to me. How can we reproduce this? What is your build configuration(config.toml)?

[thenorili]

I've tested it with no config.toml and with the config_example.toml on windows. Perhaps the times are different for other people, but the core of the matter is that running x --help when it was just built a minute before shouldn't trigger a download and rebuild, it should just print the help, the side effects are unusual and unwanted.

[thenorili]

I notice that now it only takes 0.69 seconds for me. The last thing I did with it last night was finish a successful x build. It occurs to me that perhaps it takes longer if you haven't successfully finished an x build already and running x and x --help a few times and having a failed x build left it in a state where it took a long time every time I ran x --help. Jyn remarked that build artifacts are cached but not reused, as explanation.

I nuked my build directory and get the slow start only the first time I run x --help. When I run it subsequent times it takes 0.05s -- still longer than a normal help command, but nothing too long.

I can't reproduce a second long help command today. I did switch my antivirus off in my build directory since that was happening, maybe that slowed down the 0.05s build enough? I doubt it, but I'm unsure.

[jyn514]

nori ran x clean between invocations, which clears bootstrap's build cache as well. we might want to reconsider whether that should clear bootstrap artifacts, but nori's idea will also help for people building for the first time.

[onur-ozkan]

nori ran x clean between invocations, which clears bootstrap's build cache as well. we might want to reconsider whether that should clear bootstrap artifacts, but nori's idea will also help for people building for the first time.

Ignoring bootstrap artifacts on x clean seems quite fine. Since it only takes a couple of lines to change, we can easily revert it if it becomes a problem. Would you like to make that change, or should I do it? :) @jyn514

[jyn514]

i am not planning to make a pr for that change

[onur-ozkan]

With #117723, we no longer delete bootstrap artifacts unless the --all flag is used, which, in turn, removes the entire build directory. I will consider this task completed, as we are not going to move x --help to shell scripts or python (that's the only way of not building anything for seeing the output of x --help).

[jyn514]

we are not going to move x --help to shell scripts or python (that's the only way of not building anything for seeing the output of x --help)

this is not necessarily true - you could run the bootstrap binary to generate the help, save that to a file, and then print the file in the python script. CI could enforce that the file is kept up to date.

[onur-ozkan]

Seems very interesting solution 🤔. We can also achieve the 'instant help' thing by dowloading pre-compiled bootstrap (not for all the architectures, but x86 and arm should be enough for developers I guess?), which is already a focus for the team.

Regarding 'printing from python', we are attempting to minimize the bootstrap logic there(in python scripts) as much as possible, ultimately aiming not to require having python for bootstrapping. So 'print the file' has to be done in shell scripts.

At the end it seems very easy to implement and maintain anyway, nice suggestion :)