New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

static array of zeroes can take minutes to lint check #55795

Open
kazcw opened this Issue Nov 8, 2018 · 0 comments

Comments

Projects
None yet
3 participants
@kazcw

kazcw commented Nov 8, 2018

This program takes an inordinate amount of time and memory to compile (it's worse if the array is actually used, but this is a minimal test case):

const SIZE: usize = 1 << 30;
static SLICE: [u8; SIZE] = [0u8; SIZE];
fn main() {}

I was hoping this would be a viable way to get a big chunk of .bss so I don't have to depend on an mmap crate to get a lot of static zeroes, but the effect on compile time makes that impractical.

If I run rustc -Z time-passes, the big offender is:
time: 47.666; rss: 2486MB lint checking

Although the time (and memory) is reportedly spent checking lints, setting --cap-lints allow doesn't make any difference. I'm guessing the "lint checking" pass includes some things that need to be checked even if lints are suppressed? If not, it seems like a separate issue is that a lot of work could be saved with cap-lints set (e.g. when compiling dependencies).

Here are the top results from perf report, for rustc 1.32.0-nightly (25a42b2ce 2018-11-07):

  31.62%  rustc     librustc_mir-714845413a99e6ff.so              [.] <rustc_mir::interpret::memory::Memory<'a, 'mir, 'tcx, M>>::copy_repeatedly
  22.94%  rustc     librustc_mir-714845413a99e6ff.so              [.] <core::iter::Map<I, F> as core::iter::iterator::Iterator>::fold
   7.32%  rustc     librustc-0eb8c117db37850c.so                  [.] rustc::mir::interpret::UndefMask::grow
   7.31%  rustc     librustc-0eb8c117db37850c.so                  [.] rustc::mir::interpret::UndefMask::set_range
   6.94%  rustc     libc-2.27.so                                  [.] __memmove_sse2_unaligned_erms
   5.91%  rustc     librustc_mir-714845413a99e6ff.so              [.] <rustc_mir::interpret::memory::Memory<'a, 'mir, 'tcx, M>>::check_bytes

So it looks like miri is actually creating the array and folding over it. I know it's not going to find any problems, because I have ECC memory 😆.

There are already some bugs relating to slow compilation of large arrays, with the most relevant I could find being #37155, #49330. I think this is separate from those cases because:

  • the input in this case is a [0; _] array, whereas the others initialize arrays from sequences of elements
  • those cases seem to refer to superlinear runtime; this issue appears roughly linear in time and space and only becomes noticeable for much larger arrays
  • the bottleneck in this case occurs during lint checking, which I didn't see in any other array performance bugs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment