vector panic when use disk buffer

[z2665]

I install Vector using Helm,and use agent role to collect my logs and send to s3
the heml value hostpath is

  hostPath:
    # persistence.hostPath.path -- Override path used for hostPath persistence
    ## Valid for Agent role, persistence always used for Agent role
    path: "/vector"

the config is

s3_test_logs:
      type: aws_s3
      inputs:
        - enrich_test_log_message
      region: us-west-2
      bucket: test-log-us-west-2
      key_prefix: |-
        test/base/
      compression: gzip
      batch:
        max_bytes: 104857600 # 100MB
        timeout_secs: 180 # 3 minutes
      buffer:
        type: disk
         max_size: 1073741824 # 1GB
      encoding:
        codec: text
      auth:
        assume_role: "role arn"
      tags:
        k8s: ${K8S_CLUSTER_NAME}

I run it ，but get error and crashloop

2022-09-01T12:15:23.835518Z DEBUG sink{component_kind="sink" component_id=s3_test_logs component_type=aws_s3 component_name=s3_test_logs}: vector_buffers::variants::disk_v2::reader: Rolling to next data file.
thread 'vector-worker' panicked at 'reader encountered unrecoverable error: PartialWrite', /project/lib/vector-buffers/src/topology/channel/receiver.rs:64:18
stack backtrace:
   0: rust_begin_unwind
             at ./rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/panicking.rs:498:5
   1: core::panicking::panic_fmt
             at ./rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/core/src/panicking.rs:107:14
   2: core::result::unwrap_failed
             at ./rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/core/src/result.rs:1613:5
   3: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   4: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
   6: <vector_buffers::topology::channel::receiver::BufferReceiverStream<T> as futures_core::stream::Stream>::poll_next
   7: <vector::utilization::Utilization<S> as futures_core::stream::Stream>::poll_next
   8: <futures_util::stream::stream::filter::Filter<St,Fut,F> as futures_core::stream::Stream>::poll_next
   9: <futures_util::stream::stream::map::Map<St,F> as futures_core::stream::Stream>::poll_next
  10: <stream_cancel::combinator::TakeUntilIf<S,F> as futures_core::stream::Stream>::poll_next
  11: <futures_util::stream::stream::flatten::Flatten<St,<St as futures_core::stream::Stream>::Item> as futures_core::stream::Stream>::poll_next
  12: <futures_util::stream::stream::FlatMap<St,U,F> as futures_core::stream::Stream>::poll_next
  13: <vector_core::stream::partitioned_batcher::PartitionedBatcher<St,Prt,KT> as futures_core::stream::Stream>::poll_next
  14: <futures_util::stream::stream::filter_map::FilterMap<St,Fut,F> as futures_core::stream::Stream>::poll_next
  15: <vector_core::stream::concurrent_map::ConcurrentMap<St,T> as futures_core::stream::Stream>::poll_next
  16: <futures_util::stream::stream::filter_map::FilterMap<St,Fut,F> as futures_core::stream::Stream>::poll_next
  17: <futures_util::stream::stream::ready_chunks::ReadyChunks<St> as futures_core::stream::Stream>::poll_next
  18: <tokio::future::poll_fn::PollFn<F> as core::future::future::Future>::poll
  19: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  20: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  21: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  22: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
  23: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
  24: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  25: tokio::runtime::task::harness::Harness<T,S>::poll
  26: std::thread::local::LocalKey<T>::with
  27: tokio::runtime::thread_pool::worker::Context::run_task
  28: tokio::runtime::thread_pool::worker::Context::run
  29: tokio::macros::scoped_tls::ScopedKey<T>::set
  30: tokio::runtime::thread_pool::worker::run
  31: tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut
  32: tokio::runtime::task::harness::Harness<T,S>::poll
  33: tokio::runtime::blocking::pool::Inner::run
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
2022-09-01T12:15:24.228149Z ERROR sink{component_kind="sink" component_id=s3_test_logs component_type=aws_s3 component_name=s3_test_logs}: vector::topology: An error occurred that Vector couldn't handle.
2022-09-01T12:15:24.228235Z  INFO vector: Vector has stopped.

how can I solve this problem

[spencergilbert]

👋

Hi @z2665, can you please share your full config?

[z2665]

I solved the problem after consulting my colleague, the problem occurred when I only gave the vector memory limit 500Mi, triggered OOMKILL, and after the vector was pulled up again it would repeatedly report this error and fall into crashloop.
Based on issues 13178 I upgraded to version 0.23 to avoid this problem.