Rust compiler creating huge files on tmp directory

[spastorino]

Was surprised to get the following no space left error ...

Compiling rustc_driver v0.0.0 (/home/santiago/src/oss/rust1/src/librustc_driver)
error: failed to write version script: No space left on device (os error 28)

This happens because rustc_driver is placing a lot of huge places on /tmp and tmp on my machine is of type tmpfs which is stored usually in RAM or swap and it's by default half of the RAM, so 4GB in my old machine.

The following are the files created ...

[santiago@archlinux rustcaSq42I]$ pwd
/tmp/rustcaSq42I
[santiago@archlinux rustcaSq42I]$ ls -lh
total 4.0G
-rw-r--r-- 1 santiago santiago  6.9M Aug 15 14:44 libcc-580cb3ec679a1ef0.rlib
-rw-r--r-- 1 santiago santiago  243K Aug 15 14:44 libcrc32fast-df4710f26ff3874f.rlib
-rw-r--r-- 1 santiago santiago   746 Aug 15 14:44 libeither-be66c1a9963b273c.rlib
-rw-r--r-- 1 santiago santiago  2.1M Aug 15 14:44 libenv_logger-39812d84bbfe2b11.rlib
-rw-r--r-- 1 santiago santiago  595K Aug 15 14:44 libflate2-31c9dbe4de46ec52.rlib
-rw-r--r-- 1 santiago santiago  414K Aug 15 14:44 libhumantime-d1a9d86e783663fa.rlib
-rw-r--r-- 1 santiago santiago   742 Aug 15 14:44 libitoa-cafbf9ae61da85d2.rlib
-rw-r--r-- 1 santiago santiago  202K Aug 15 14:44 liblog_settings-2620b484081ec8f2.rlib
-rw-r--r-- 1 santiago santiago   64K Aug 15 14:44 libminiz_sys-829f35f627be716a.rlib
-rw-r--r-- 1 santiago santiago  584K Aug 15 14:44 libpunycode-bc98388f5faa9996.rlib
-rw-r--r-- 1 santiago santiago   764 Aug 15 14:44 libquick_error-b52ae6e62fe18a00.rlib
-rw-r--r-- 1 santiago santiago   770 Aug 15 14:44 libremove_dir_all-a10b98c02f91914a.rlib
-rw-r--r-- 1 santiago santiago  666M Aug 15 14:44 librustc-300974c3ebad3d8b.rlib
-rw-r--r-- 1 santiago santiago  139M Aug 15 14:44 librustc_ast_borrowck-064564744d0b66e4.rlib
-rw-r--r-- 1 santiago santiago  158M Aug 15 14:44 librustc_codegen_ssa-0b961b0d1ebdaa34.rlib
-rw-r--r-- 1 santiago santiago  141M Aug 15 14:44 librustc_codegen_utils-3c4fec2d7a4aaa58.rlib
-rw-r--r-- 1 santiago santiago  215M Aug 15 14:44 librustc_incremental-413cebfc5fe72a28.rlib
-rw-r--r-- 1 santiago santiago  163M Aug 15 14:44 librustc_interface-4a187b71ebecda3f.rlib
-rw-r--r-- 1 santiago santiago  128M Aug 15 14:44 librustc_lint-f5d7618b6d033ce8.rlib
-rw-r--r-- 1 santiago santiago  315M Aug 15 14:44 librustc_metadata-a179352ab6d85b74.rlib
-rw-r--r-- 1 santiago santiago  822M Aug 15 14:44 librustc_mir-232a965ccb3e9077.rlib
-rw-r--r-- 1 santiago santiago  143M Aug 15 14:44 librustc_passes-c08b598d7f5694c2.rlib
-rw-r--r-- 1 santiago santiago   36M Aug 15 14:44 librustc_plugin-8f9749116c8b6ad6.rlib
-rw-r--r-- 1 santiago santiago  138M Aug 15 14:44 librustc_privacy-1eb27d780fbbcb81.rlib
-rw-r--r-- 1 santiago santiago   76M Aug 15 14:44 librustc_resolve-23cfc118665ed372.rlib
-rw-r--r-- 1 santiago santiago  138M Aug 15 14:44 librustc_save_analysis-970ae44e6e8aee42.rlib
-rw-r--r-- 1 santiago santiago  267M Aug 15 14:44 librustc_traits-1f50191ae0f8c9ba.rlib
-rw-r--r-- 1 santiago santiago  469M Aug 15 14:44 librustc_typeck-0246067c1ba94397.rlib
-rw-r--r-- 1 santiago santiago  298K Aug 15 14:44 libryu-17527a943b47993b.rlib
-rw-r--r-- 1 santiago santiago  3.6M Aug 15 14:44 libserde_json-f180caae537f064f.rlib
-rw-r--r-- 1 santiago santiago   71M Aug 15 14:44 libsyntax_ext-c3562645f3bbdcaa.rlib
-rw-r--r-- 1 santiago santiago 1006K Aug 15 14:44 libtempfile-7dcc5500a694e84f.rlib
-rw-r--r-- 1 santiago santiago  1.1M Aug 15 14:44 list

I think that the compilation process shouldn't place this kind of files there.

For more information, read the Zulip discussion https://rust-lang.zulipchat.com/#narrow/stream/182449-t-compiler.2Fhelp/topic/no.20space.20left.20on.20device.20(tmpfs).20when.20compiling.20rustc

[Mark-Simulacrum]

Hm, so it seems like we do need a temporary directory for these files. If we can't place them in /tmp or it's equivalent, then they'll probably end up going into the current directory of the compiler, which seems somewhat bad as well.

Do we know what other compilers do here? Presumably, gcc/clang etc all need to write out version scripts and such too?

[the8472]

O_TMPFILE descriptors could be an option, they never pollute the directory tree unless they get linked into it.

That said I would also like them to go into a separate path designated for temporary files. My project folders are under btrfs and get snapshotted via cron jobs. It is preferable if tempfiles don't make it into the snapshots.

[Mark-Simulacrum]

I believe we can't use O_TMPFILE and such as we usually need to pass paths to linkers and such that don't accept file descriptors as inputs.

[the8472]

You can pass them as /proc/<pid>/fd/<n>, with pid either being the parent process or the child if the fd hasn't been closed on exec.

[Mark-Simulacrum]

Unfortunately, that's not supported on all platforms I believe, though I could well be wrong. It's also likely to cause problem when trying to debug what went wrong since all of the files are by definition gone :)

[tmandry]

Hit this as well. I think any build outputs (including temporary ones) should go on the same filesystem as the final outputs, since we can expect that to be large enough. So a good strategy is to make a subdirectory under the out dir (or path of one of the outputs).