Dec 09 17:34:20.501 ERROR panic: timely:work-0: range end index 1024 out of range for slice of length 54 #5008

realtimclemans · 2020-12-09T17:40:10Z

What version of Materialize are you using?

$ materialized -v

How did you install Materialize?

What was the issue?

Dec 09 17:34:18.858  WARN materialized::version_check: a new version of materialized is available: 0.5.3    
Dec 09 17:34:19.352  WARN dataflow::decode::regex: Line data: {"$schema":"/mediawiki/recentchange/1.0.0","meta":{"uri":"https://ru.wikipedia.org/wiki/%D0%9A%D0%B0%D1%82%D0%B5%D0%B3%D0%BE%D1%80%D0%B8%D1%8F:%D0%90%D0%BB%D1%8C%D0%B1%D0%BE%D0%BC%D1%8B_%D0%BD%D0%BE%D0%B2%D0%BE%D0%B9_%D0%B2%D0%BE%D0%BB%D0%BD%D1%8B","request_id":"caafe655151b8eea362e4f46","id":"fad32410-e9a1-4e8b-9e4f-c16c616232b2","dt":"2020-12-09T17:27:37Z","domain":"ru.wikipedia.org","stream":"mediawiki.recentchange","topic":"eqiad.mediawiki.recentchange","partition":0,"offset":2787358399},"id":403200922,"type":"categorize","namespace":14,"title":"Категория:Альбомы новой волны","comment":"[[:Это не любовь]] добавлена в категорию","timestamp":1607534857,"user":"Diamond Player","bot":false,"server_url":"https://ru.wikipedia.org","server_name":"ru.wikipedia.org","server_script_path":"/w","wiki":"ruwiki","parsedcomment":"<a href=\"/wiki/%D0%AD%D1%82%D0%BE_%D0%BD%D0%B5_%D0%BB%D1%8E%D0%B1%D0%BE%D0%B2%D1%8C\" title=\"Это не любовь\">Это не лю�event: message from source materialize.public.recentchanges_primary_idx cannot be decoded as utf8. Discarding line.    
Dec 09 17:34:20.501 ERROR panic: timely:work-0: range end index 1024 out of range for slice of length 54
   0: materialized::handle_panic
             at /var/lib/buildkite-agent/builds/buildkite-builders-i-0c0231b1f2bdc4566-1/materialize/tests/src/materialized/src/bin/materialized/main.rs:545:65
   1: core::ops::function::Fn::call
             at /usr/local/lib/rustlib/src/rust/library/core/src/ops/function.rs:70:5
   2: std::panicking::rust_panic_with_hook
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:573:17
   3: std::panicking::begin_panic_handler::{{closure}}
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:476:9
   4: std::sys_common::backtrace::__rust_end_short_backtrace
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys_common/backtrace.rs:153:18
   5: rust_begin_unwind
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/panicking.rs:475:5
   6: core::panicking::panic_fmt
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/core/src/panicking.rs:85:14
   7: core::slice::slice_end_index_len_fail
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/core/src/slice/mod.rs:3149:5
   8: <core::ops::range::Range<usize> as core::slice::SliceIndex<[T]>>::index
             at /usr/local/lib/rustlib/src/rust/library/core/src/slice/mod.rs:3338:13
      core::slice::<impl core::ops::index::Index<I> for [T]>::index
             at /usr/local/lib/rustlib/src/rust/library/core/src/slice/mod.rs:3123:9
      <alloc::vec::Vec<T> as core::ops::index::Index<I>>::index
             at /usr/local/lib/rustlib/src/rust/library/alloc/src/vec.rs:2000:9
      dataflow::decode::regex::regex::{{closure}}::{{closure}}::{{closure}}
             at /var/lib/buildkite-agent/builds/buildkite-builders-i-0c0231b1f2bdc4566-1/materialize/tests/src/dataflow/src/decode/regex.rs:47:62
      timely::dataflow::operators::generic::handles::InputHandle<T,D,P>::for_each
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/dataflow/operators/generic/handles.rs:92:13
      dataflow::decode::regex::regex::{{closure}}::{{closure}}
             at /var/lib/buildkite-agent/builds/buildkite-builders-i-0c0231b1f2bdc4566-1/materialize/tests/src/dataflow/src/decode/regex.rs:37:17
      <timely::dataflow::stream::Stream<G,D1> as timely::dataflow::operators::generic::operator::Operator<G,D1>>::unary::{{closure}}::{{closure}}
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/dataflow/operators/generic/operator.rs:402:17
      timely::dataflow::operators::generic::builder_rc::OperatorBuilder<G>::build::{{closure}}::{{closure}}
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/dataflow/operators/generic/builder_rc.rs:123:31
      timely::dataflow::operators::generic::builder_rc::OperatorBuilder<G>::build_reschedule::{{closure}}
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/dataflow/operators/generic/builder_rc.rs:163:26
      <timely::dataflow::operators::generic::builder_raw::OperatorCore<T,L> as timely::scheduling::Schedule>::schedule
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/dataflow/operators/generic/builder_raw.rs:203:9
   9: timely::progress::subgraph::PerOperatorState<T>::schedule
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/progress/subgraph.rs:647:30
  10: timely::progress::subgraph::Subgraph<TOuter,TInner>::activate_child
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/progress/subgraph.rs:330:26
      <timely::progress::subgraph::Subgraph<TOuter,TInner> as timely::scheduling::Schedule>::schedule
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/progress/subgraph.rs:296:17
  11: timely::progress::subgraph::PerOperatorState<T>::schedule
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/progress/subgraph.rs:647:30
  12: timely::progress::subgraph::Subgraph<TOuter,TInner>::activate_child
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/progress/subgraph.rs:330:26
      <timely::progress::subgraph::Subgraph<TOuter,TInner> as timely::scheduling::Schedule>::schedule
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/progress/subgraph.rs:296:17
  13: timely::worker::Wrapper::step::{{closure}}
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/worker.rs:550:57
      core::option::Option<T>::map
             at /usr/local/lib/rustlib/src/rust/library/core/src/option.rs:437:29
      timely::worker::Wrapper::step
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/worker.rs:550:26
  14: timely::worker::Worker<A>::step_or_park
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/worker.rs:235:38
  15: dataflow::server::Worker<A>::run
             at /var/lib/buildkite-agent/builds/buildkite-builders-i-0c0231b1f2bdc4566-1/materialize/tests/src/dataflow/src/server.rs:470:13
  16: dataflow::server::serve::{{closure}}::{{closure}}
             at /var/lib/buildkite-agent/builds/buildkite-builders-i-0c0231b1f2bdc4566-1/materialize/tests/src/dataflow/src/server.rs:231:13
      tokio::runtime::context::enter
             at /cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/runtime/context.rs:72:5
  17: tokio::runtime::handle::Handle::enter
             at /cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.2.22/src/runtime/handle.rs:76:9
      dataflow::server::serve::{{closure}}
             at /var/lib/buildkite-agent/builds/buildkite-builders-i-0c0231b1f2bdc4566-1/materialize/tests/src/dataflow/src/server.rs:225:9
      timely::execute::execute_from::{{closure}}
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/timely/src/execute.rs:299:22
      timely_communication::initialize::initialize_from::{{closure}}
             at /cargo/git/checkouts/timely-dataflow-4c0cc365061cd263/a025087/communication/src/initialize.rs:269:33
      std::sys_common::backtrace::__rust_begin_short_backtrace
             at /usr/local/lib/rustlib/src/rust/library/std/src/sys_common/backtrace.rs:137:18
  18: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
             at /usr/local/lib/rustlib/src/rust/library/std/src/thread/mod.rs:458:17
      <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
             at /usr/local/lib/rustlib/src/rust/library/std/src/panic.rs:308:9
      std::panicking::try::do_call
             at /usr/local/lib/rustlib/src/rust/library/std/src/panicking.rs:373:40
      std::panicking::try
             at /usr/local/lib/rustlib/src/rust/library/std/src/panicking.rs:337:19
      std::panic::catch_unwind
             at /usr/local/lib/rustlib/src/rust/library/std/src/panic.rs:379:14
      std::thread::Builder::spawn_unchecked::{{closure}}
             at /usr/local/lib/rustlib/src/rust/library/std/src/thread/mod.rs:457:30
      core::ops::function::FnOnce::call_once{{vtable.shim}}
             at /usr/local/lib/rustlib/src/rust/library/core/src/ops/function.rs:227:5
  19: <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/alloc/src/boxed.rs:1042:9
      <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/alloc/src/boxed.rs:1042:9
      std::sys::unix::thread::Thread::new::thread_start
             at /rustc/18bf6b4f01a6feaf7259ba7cdae58031af1b7b39/library/std/src/sys/unix/thread.rs:87:17
  20: start_thread
  21: clone

Is the issue reproducible? If so, please provide reproduction instructions.

Please attach any applicable log files.

The text was updated successfully, but these errors were encountered:

benesch · 2020-12-09T17:46:48Z

Thanks for the report. A bit more information would be helpful, though!

It seems to me that you were following the "Create a real-time stream" demo from our getting started docs. Can you confirm? If so, can you share the wikirecent file that caused this error?

If you were not following the demo, what were you doing that triggered the error? I.e., what CREATE SOURCE command did you run?

realtimclemans · 2020-12-09T17:47:57Z

Yes following the quickstart.

…

On Wed, Dec 9, 2020 at 9:47 AM Nikhil Benesch ***@***.***> wrote: Thanks for the report. A bit more information would be helpful, though! It seems to me that you were following the "Create a real-time stream" <https://materialize.com/docs/get-started/#create-a-real-time-stream> demo from our getting started docs. Can you confirm? If so, can you share the wikirecent file that caused this error? If you were not following the demo, what were you doing that triggered the error? I.e., what CREATE SOURCE command did you run? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5008 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AP7ZGZ7FSKQDC6MEWS7F7D3ST6ZZVANCNFSM4UT2NASA> .

benesch · 2020-12-09T17:50:13Z

Great. Can you upload your wikirecent file that triggered this error?

realtimclemans · 2020-12-09T17:52:46Z

wikirecent <https://drive.google.com/file/d/1VzTYOTuFbtMehL-uPtWPBRtIQUepNtZJ/view?usp=drive_web>

…

On Wed, Dec 9, 2020 at 9:50 AM Nikhil Benesch ***@***.***> wrote: Great. Can you upload your wikirecent file that triggered this error? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5008 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AP7ZGZZUZSCKKR4DZDTVSRLST62GJANCNFSM4UT2NASA> .

benesch · 2020-12-09T17:53:12Z

Thanks! Looks like you'll need to grant me permission to access the file though.

realtimclemans · 2020-12-09T17:54:31Z

https://drive.google.com/file/d/1VzTYOTuFbtMehL-uPtWPBRtIQUepNtZJ/view?usp=sharing

…

On Wed, Dec 9, 2020 at 9:53 AM Nikhil Benesch ***@***.***> wrote: Thanks! Looks like you'll need to grant me permission to access the file though. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#5008 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AP7ZGZ5AN3ZGXSNYOBXL6CTST62RPANCNFSM4UT2NASA> .

benesch · 2020-12-09T17:56:43Z

Great, thanks!

benesch · 2020-12-09T21:46:56Z

Thanks again for the report. This will be fixed by #5009.

realtimclemans added the C-bug Category: something is broken label Dec 9, 2020

benesch mentioned this issue Dec 9, 2020

dataflow: avoid out-of-bounds slice when decoding regex #5009

Merged

umanwizard closed this as completed in #5009 Dec 10, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dec 09 17:34:20.501 ERROR panic: timely:work-0: range end index 1024 out of range for slice of length 54 #5008

Dec 09 17:34:20.501 ERROR panic: timely:work-0: range end index 1024 out of range for slice of length 54 #5008

realtimclemans commented Dec 9, 2020 •

edited by benesch

benesch commented Dec 9, 2020

realtimclemans commented Dec 9, 2020 via email

benesch commented Dec 9, 2020

realtimclemans commented Dec 9, 2020 via email

benesch commented Dec 9, 2020

realtimclemans commented Dec 9, 2020 via email

benesch commented Dec 9, 2020

benesch commented Dec 9, 2020

Dec 09 17:34:20.501 ERROR panic: timely:work-0: range end index 1024 out of range for slice of length 54 #5008

Dec 09 17:34:20.501 ERROR panic: timely:work-0: range end index 1024 out of range for slice of length 54 #5008

Comments

realtimclemans commented Dec 9, 2020 • edited by benesch

What version of Materialize are you using?

How did you install Materialize?

What was the issue?

Is the issue reproducible? If so, please provide reproduction instructions.

Please attach any applicable log files.

benesch commented Dec 9, 2020

realtimclemans commented Dec 9, 2020 via email

benesch commented Dec 9, 2020

realtimclemans commented Dec 9, 2020 via email

benesch commented Dec 9, 2020

realtimclemans commented Dec 9, 2020 via email

benesch commented Dec 9, 2020

benesch commented Dec 9, 2020

realtimclemans commented Dec 9, 2020 •

edited by benesch