Out of memory when compiling

[shahinrostami]

Hello,

Looking forward to trying clippy out. cargo install cippy outputs the following:

Updating registry `https://github.com/rust-lang/crates.io-index`
   Compiling rustc-serialize v0.3.19
   Compiling tool v0.1.28
   Compiling quine-mc_cluskey v0.2.2
   Compiling unicode-normalisation v0.1.2
   Compiling nom v1.2.3
   Compiling server v0.2.3
   Compiling regex-syntax v0.3.1
   Compiling clip v0.0.66
   [ 3145.287705] Out of memory: Kill process 58555 (rustc) score 586 or sacrifice child
   [ 3145.288245] Killed proces 58555 (rustc) total-vm:466964kB, anon-rss:205204kB, file-rss:0kB, shmem-rss:4436kB
   error: failed to compile `clippy v0.0.66`, intermediate artifacts can be found at `/tmp/cargo-install.T4Grti07DRt0`

   Caused by:
    Could not compile `clip`.

On Arch Linux with 512 MB

[oli-obk]

Yea, that memory is not enough, see a trace on my machine:

time: 0.086; rss: 68MB  parsing
time: 0.023; rss: 69MB  configuration 1
time: 0.000; rss: 69MB  recursion limit
time: 0.000; rss: 69MB  crate injection
time: 0.198; rss: 120MB macro loading
time: 0.000; rss: 120MB plugin loading
time: 0.000; rss: 120MB plugin registration
time: 0.210; rss: 128MB expansion
time: 0.034; rss: 128MB configuration 2
time: 0.000; rss: 128MB gated configuration checking
time: 0.015; rss: 128MB maybe building test harness
time: 0.015; rss: 128MB prelude injection
time: 0.003; rss: 128MB checking that all macro invocations are gone
time: 0.000; rss: 128MB checking for inline asm in case the target doesn't support it
time: 0.005; rss: 128MB complete gated feature checking
time: 0.015; rss: 128MB assigning node ids
time: 0.007; rss: 128MB collecting defs
time: 0.060; rss: 151MB external crate/lib resolution
time: 0.029; rss: 154MB lowering ast -> hir
time: 0.003; rss: 155MB indexing hir
time: 0.003; rss: 155MB attribute checking
time: 0.012; rss: 155MB early lint checks
time: 0.003; rss: 155MB language item collection
time: 0.050; rss: 173MB name resolution
time: 0.004; rss: 173MB lifetime resolution
time: 0.000; rss: 173MB looking for entry point
time: 0.000; rss: 173MB looking for plugin registrar
time: 0.013; rss: 178MB region resolution
time: 0.002; rss: 178MB loop checking
time: 0.002; rss: 178MB static item recursion checking
time: 0.000; rss: 178MB load_dep_graph
time: 0.039; rss: 191MB type collecting
time: 0.000; rss: 191MB variance inference
time: 0.085; rss: 202MB coherence checking
time: 0.065; rss: 203MB wf checking
time: 0.075; rss: 203MB item-types checking
time: 1.521; rss: 241MB item-bodies checking
time: 0.000; rss: 241MB drop-impl checking
time: 0.075; rss: 242MB const checking
time: 0.018; rss: 242MB privacy checking
time: 0.002; rss: 242MB stability index
time: 0.008; rss: 242MB intrinsic checking
time: 0.005; rss: 242MB effect checking
time: 0.021; rss: 242MB match checking
time: 0.014; rss: 242MB liveness checking
time: 0.049; rss: 242MB rvalue checking
time: 0.148; rss: 263MB MIR dump
time: 0.100; rss: 265MB MIR passes
time: 0.136; rss: 265MB borrow checking
time: 0.001; rss: 265MB reachability checking
time: 0.014; rss: 265MB death checking
time: 0.018; rss: 265MB stability checking
time: 0.000; rss: 265MB unused lib feature checking
time: 0.099; rss: 265MB lint checking
time: 0.003; rss: 265MB resolving dependency formats
time: 0.033; rss: 268MB Prepare MIR codegen passes
  time: 0.562; rss: 299MB   translation item collection
  time: 0.019; rss: 303MB   codegen unit partitioning
  time: 0.000; rss: 762MB   write metadata
time: 5.544; rss: 762MB translation
time: 0.001; rss: 762MB assert dep graph
time: 0.000; rss: 762MB serialize dep graph
  time: 2.597; rss: 350MB   llvm function passes [0]
  time: 34.467; rss: 353MB  llvm module passes [0]
  time: 8.496; rss: 355MB   codegen passes [0]
  time: 0.001; rss: 202MB   codegen passes [0]
time: 45.729; rss: 199MB    LLVM passes
  time: 0.247; rss: 194MB   running linker
time: 0.248; rss: 194MB linking

[shahinrostami]

Thank you for the response - I upped to 1024 MB and had the same issue.

It compiled after upping to 2048 MB.

[Manishearth]

fwiw after cargo install clippy I think we got a lot slower to compile. Not a major issue though

[mcarton]

fwiw after cargo install clippy I think we got a lot slower to compile

I’ve noticed that too. It’s much longer now.

[oli-obk]

I'll investigate

[oli-obk]

a cargo test call is not slower than before, because previously clippy was compiled twice, once with --cfg test and once without. I removed that in the same PR where I introduced cargo clippy. Just running cargo build is obviously slower, since it compiles twice.

[crumblingstatue]

I, too, think that the memory usage is excessive. My system "only" has 2 GB of ram, and I had a whole gigabyte ram free for compiling clippy, but rustc failed because it ran out of ram.

This is more likely rustc's fault than clippy's fault though.

[Manishearth]

When you cargo install clippy, it basically links rustc again, which takes up tons of memory. Given the way it's designed that can't be changed.

[crumblingstatue]

When you cargo install clippy, it basically links rustc again, which takes up tons of memory. Given the way it's designed that can't be changed.

I didn't cargo install clippy though. I'm talking about just building cargo-clippy with cargo.

[Manishearth]

By cargo-clippy do you mean the cargo-clippy binary inside clippy? It does the same thing.

[crumblingstatue]

I was actually using https://github.com/arcnmx/cargo-clippy, but now that I see clippy has its own cargo addon now, I'll use that one.

[oli-obk]

With the recent split into bin/plugin/lints, both compilation time and memory usage should be reduced. Not sure what else we can do about this issue.