Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ICE on extremely long string literal (larger than 4G) #61904

Closed
oxalica opened this issue Jun 17, 2019 · 4 comments · Fixed by #61908
Closed

ICE on extremely long string literal (larger than 4G) #61904

oxalica opened this issue Jun 17, 2019 · 4 comments · Fixed by #61908
Labels
A-frontend Area: frontend (errors, parsing and HIR) A-parser Area: The parsing of Rust source code to an AST. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@oxalica
Copy link
Contributor

oxalica commented Jun 17, 2019

I know that having huge literal in source is a bad idea, but it should not cause ICE :(
It looks like something overflowed.

Version: rustc 1.37.0-nightly (4edff843d 2019-06-16) running on x86_64-unknown-linux-gnu

Command, output and backtrace:

$ RUST_BACKTRACE=full rustc ./huge.rs
thread 'rustc' panicked at 'begin <= end (22 <= 21) when slicing `const HELLO: &str = r"
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
hello!!
h`[...]', src/libcore/str/mod.rs:2021:5
stack backtrace:
   0:     0x7fc8de00011b - backtrace::backtrace::libunwind::trace::h912e0d04a7708cf2
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
   1:     0x7fc8de00011b - backtrace::backtrace::trace_unsynchronized::hfb708595cf87b63f
                               at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
   2:     0x7fc8de00011b - std::sys_common::backtrace::_print::ha79f66d09fb9689e
                               at src/libstd/sys_common/backtrace.rs:47
   3:     0x7fc8de00011b - std::sys_common::backtrace::print::hbbf6608241e58d95
                               at src/libstd/sys_common/backtrace.rs:36
   4:     0x7fc8de00011b - std::panicking::default_hook::{{closure}}::h5463a2f4723f6cb3
                               at src/libstd/panicking.rs:198
   5:     0x7fc8ddfffe1e - std::panicking::default_hook::h79a408572fcba4ed
                               at src/libstd/panicking.rs:212
   6:     0x7fc8dbd92351 - rustc::util::common::panic_hook::h318156cfb41f5413
   7:     0x7fc8de000979 - std::panicking::rust_panic_with_hook::hb67cd2b0c77d9895
                               at src/libstd/panicking.rs:479
   8:     0x7fc8de000412 - std::panicking::continue_panic_fmt::he44145e3b8e029cd
                               at src/libstd/panicking.rs:382
   9:     0x7fc8de0002f6 - rust_begin_unwind
                               at src/libstd/panicking.rs:309
  10:     0x7fc8de02e11d - core::panicking::panic_fmt::h7327567cfbeab565
                               at src/libcore/panicking.rs:85
  11:     0x7fc8de030740 - core::str::slice_error_fail::hba2c24f2962448b8
                               at src/libcore/str/mod.rs:0
  12:     0x7fc8db0dd012 - core::str::traits::<impl core::slice::SliceIndex<str> for core::ops::range::Range<usize>>::index::{{closure}}::he3b193b38a91659a
  13:     0x7fc8db0ebf50 - syntax::parse::lexer::StringReader::with_str_from_to::h0d0fa368b8b05ffc
  14:     0x7fc8db0e9916 - syntax::parse::lexer::StringReader::advance_token::hc406a5a0c98fe430
  15:     0x7fc8db0e766d - syntax::parse::lexer::StringReader::try_next_token::habfb2fe795d28efa
  16:     0x7fc8db0e7a9f - syntax::parse::lexer::StringReader::real_token::hfbd846ba27af2f28
  17:     0x7fc8db115ef1 - syntax::parse::lexer::tokentrees::TokenTreesReader::parse_token_tree::h8061afbb3e86c76b
  18:     0x7fc8db1155ce - syntax::parse::lexer::tokentrees::TokenTreesReader::parse_all_token_trees::h8018bcf24cc2951d
  19:     0x7fc8db0e72f5 - syntax::parse::lexer::tokentrees::<impl syntax::parse::lexer::StringReader>::into_token_trees::h35e2e07fb40a0abc
  20:     0x7fc8db0035af - syntax::parse::maybe_file_to_stream::h77846087608aa34a
  21:     0x7fc8db002840 - syntax::parse::maybe_source_file_to_parser::h109680085ff17678
  22:     0x7fc8db0022b4 - syntax::parse::source_file_to_parser::h7859d86f2a1b8e8a
  23:     0x7fc8db00164a - syntax::parse::parse_crate_from_file::h72b9587b203687b8
  24:     0x7fc8dda73bfd - rustc_interface::passes::parse::{{closure}}::h73af6c8d58be2c8e
  25:     0x7fc8dda6e476 - rustc::util::common::time::hedf145db853fee22
  26:     0x7fc8dda12c6f - rustc_interface::passes::parse::h6339ba714c071498
  27:     0x7fc8dda02653 - rustc_interface::queries::Query<T>::compute::h59a144469961cc35
  28:     0x7fc8ddaebb84 - rustc_interface::queries::<impl rustc_interface::interface::Compiler>::parse::hc1befc73f5446e68
  29:     0x7fc8de2f3a71 - rustc_interface::interface::run_compiler_in_existing_thread_pool::h14bf4a38c2714175
  30:     0x7fc8de3650a6 - std::thread::local::LocalKey<T>::with::h1ef421860970c1a1
  31:     0x7fc8de2ff0ae - scoped_tls::ScopedKey<T>::set::hae697baf57baa69a
  32:     0x7fc8de331124 - syntax::with_globals::h2ba7ae4c00aa57d8
  33:     0x7fc8de34fdfd - std::sys_common::backtrace::__rust_begin_short_backtrace::hbacd46537732a9e0
  34:     0x7fc8de01181a - __rust_maybe_catch_panic
                               at src/libpanic_unwind/lib.rs:82
  35:     0x7fc8de2dc2d9 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h5f09d41df1db8d36
  36:     0x7fc8ddfe3e1f - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::h6a25020f908b1369
                               at /rustc/4edff843dd219cf19a5fede6c78c7ce95402e1f5/src/liballoc/boxed.rs:746
  37:     0x7fc8de010500 - <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once::he4b68ab6cba491dc
                               at /rustc/4edff843dd219cf19a5fede6c78c7ce95402e1f5/src/liballoc/boxed.rs:746
  38:     0x7fc8de010500 - std::sys_common::thread::start_thread::hf0f92ea133daa112
                               at src/libstd/sys_common/thread.rs:13
  39:     0x7fc8de010500 - std::sys::unix::thread::Thread::new::thread_start::h9d375b1cdab5f2d3
                               at src/libstd/sys/unix/thread.rs:79
  40:     0x7fc8ddf7eef7 - start_thread
  41:     0x7fc8ddea322f - __GI___clone
  42:                0x0 - <unknown>
query stack during panic:
end of query stack

error: internal compiler error: unexpected panic

note: the compiler unexpectedly panicked. this is a bug.

note: we would appreciate a bug report: https://github.com/rust-lang/rust/blob/master/CONTRIBUTING.md#bug-reports

note: rustc 1.37.0-nightly (4edff843d 2019-06-16) running on x86_64-unknown-linux-gnu

Here is the generator of the source:

out="./huge.rs"
echo 'const HELLO: &str = r"' >$out
yes "hello!!" | head -c 4294967296 >>$out
echo '";fn main() { println!("{}", HELLO); }' >>$out
@Centril Centril added I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ I-nominated T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. A-frontend Area: frontend (errors, parsing and HIR) A-parser Area: The parsing of Rust source code to an AST. labels Jun 17, 2019
@Centril
Copy link
Contributor

Centril commented Jun 17, 2019

cc @petrochenkov @matklad

@matklad
Copy link
Member

matklad commented Jun 17, 2019

Took a quick look. We certainly have a restriction that any file can't be that large, because we use u32 for indexing. And looks like we never try to enforce it anywhere. I think the place to do that would be SourceFile::new.

Do we have any way to panic without this being marked as ICE? I don't want to thread Handler to each SourceFile::new, and this seems like a marginal error where it's totally OK to panic. This is even OK in the IDE context I think: IDE should enforce reasonable max size of the file (something like 10mb, not 4gb) at the virtual file system layer.

The "just panic" fix is here: matklad@8fd5b42

@petrochenkov
Copy link
Contributor

Do we have any way to panic without this being marked as ICE?

FatalError.raise() or span_fatal if the context is available.

Centril added a commit to Centril/rust that referenced this issue Jun 17, 2019
don't ICE on large files

This is an extremely marginal error, so the cost of properly threading
`Handler` everywhere just not seemed justified. However, it's useful
to panic when we create a file, and not when we slice strings with
overflown indexes somewhere in the guts of the compiler.

For this reason, while we provide safe `try_new_source_file`, we don't
change the existing public interface and just panic more or less
cleanly.

closes rust-lang#61904

r? @petrochenkov
@tesuji
Copy link
Contributor

tesuji commented Jun 17, 2019

I tried to compile the same thing in gcc and clang:

#!/bin/sh
printf '#include <stdio.h>\nchar s[] = "' > hello.c
python -c 'print("h" * 4294967296)' >> hello.c
truncate -s -1 hello.c

printf '";' >> hello.c
cat >> hello.c << EOF

int main()
{
  printf("%s\n", s);
  return 0;
}
EOF
gcc hello.c -o hello_gcc
clang hello.c -o hello_clang # crash

With gcc:

% gdb hello_gcc
Reading symbols from a.out...(no debugging symbols found)...done.
(gdb) disas main
Dump of assembler code for function main:
   0x0000000000001130 <+0>:     push   rbp
   0x0000000000001131 <+1>:     mov    rbp,rsp
   0x0000000000001134 <+4>:     lea    rdi,[rip+0x2ec6]        # 0x4001 <s>
   0x000000000000113b <+11>:    call   0x1210 <puts@plt>
   0x0000000000001140 <+16>:    mov    eax,0x0
   0x0000000000001145 <+21>:    pop    rbp
   0x0000000000001146 <+22>:    ret
End of assembler dump.
(gdb) x/s 0x4001
0x4001 <s>:     ""
(gdb) quit

With clang:

% clang hello.c
Stack dump:
0.      Program arguments: /home/lzutao/.local/bin/clang-8 -cc1 -triple x86_64-unknown-linux-gnu -emit-obj -mrelax-all -disable-free -disable-llvm-verifier -discard-value-names -main-file-name hello.c -mrelocation-model static -mthread-model posix -mdisable-fp-elim -fmat
h-errno -masm-verbose -mconstructor-aliases -munwind-tables -fuse-init-array -target-cpu x86-64 -dwarf-column-info -debugger-tuning=gdb -resource-dir /home/lzutao/.local/lib/clang/8.0.0 -internal-isystem /usr/local/include -internal-isystem /home/lzutao/.local/lib/clang/
8.0.0/include -internal-externc-isystem /home/lzutao/.local/include -internal-externc-isystem /usr/lib/gcc/x86_64-linux-gnu/6/include-fixed -internal-externc-isystem /usr/include -internal-externc-isystem /usr/local/include -internal-externc-isystem /usr/include/x86_64-l
inux-gnu -fdebug-compilation-dir /home/lzutao/forked/rust/check -ferror-limit 19 -fmessage-length 135 -fobjc-runtime=gcc -fdiagnostics-show-option -fcolor-diagnostics -o /tmp/hello-27434e.o -x c hello.c -faddrsig
1.      clang-8: error: unable to execute command: Alarm clock
clang-8: error: clang frontend command failed due to signal (use -v to see invocation)
clang version 8.0.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/lzutao/.local/bin
clang-8: note: diagnostic msg: PLEASE submit a bug report to https://bugs.llvm.org/ and include the crash backtrace, preprocessed source, and associated run script.
clang-8: note: diagnostic msg:
********************

PLEASE ATTACH THE FOLLOWING FILES TO THE BUG REPORT:
Preprocessed source(s) and associated run script(s) are located at:
clang-8: note: diagnostic msg: /tmp/hello-943593.c
clang-8: note: diagnostic msg: /tmp/hello-943593.sh
clang-8: note: diagnostic msg:

********************

Meta

% clang --version
clang version 8.0.0
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /home/lzutao/.local/bin
% gcc --version
gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

EDIT: Opened https://bugs.llvm.org/show_bug.cgi?id=42301

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-frontend Area: frontend (errors, parsing and HIR) A-parser Area: The parsing of Rust source code to an AST. I-ICE Issue: The compiler panicked, giving an Internal Compilation Error (ICE) ❄️ T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants