Skip to content

Conversation

@Duansg
Copy link
Member

@Duansg Duansg commented Dec 9, 2025

What's changed?

Close #3871

Background

When using one-time tasks or private collectors, if a single metric collection returns multiple data entries, the Collector will call responseSyncJobData after all metrics are collected to bundle all results into a single List. ArrowUtil.serializeMetricsData then serializes this List into an Arrow stream containing multiple VectorSchemaRoot objects.Upon receiving the data, the Manager invokes ArrowUtil.deserializeMetricsData. In unpatched versions, this triggers an Unexpected end of input exception, as shown below:

Page waiting timeout

error_arrow

Console error

Caused by: java.io.IOException: Unexpected end of input. Missing schema.
	at org.apache.arrow.vector.ipc.ArrowStreamReader.readSchema(ArrowStreamReader.java:207)
	at org.apache.arrow.vector.ipc.ArrowReader.initialize(ArrowReader.java:178)
	at org.apache.arrow.vector.ipc.ArrowReader.ensureInitialized(ArrowReader.java:171)
	at org.apache.arrow.vector.ipc.ArrowReader.getVectorSchemaRoot(ArrowReader.java:68)
	at org.apache.hertzbeat.common.util.ArrowUtil.deserializeMultipleRoots(ArrowUtil.java:98)
	... 40 common frames omitted

Mock error

 @look org.apache.hertzbeat.common.util.ArrowUtilTest#testSerializeAndDeserializeMetricsData

Reasons

ArrowStreamReader pre-reads data during deserialization, causing positional offsets when consecutively reading multiple Arrow streams and resulting in an Unexpected end of input exception.

Modification details

  1. Improved the serialization/deserialization logic of ArrowUtil by adding length-prefixed data, ensuring each Arrow stream can be read with precision.
  2. Developed the reproducible test case ArrowUtilTest and passed the test.

Checklist

  • I have read the Contributing Guide
  • I have written the necessary doc or comment.
  • I have added the necessary unit tests and all cases have passed.

Add or update API

  • I have added the necessary e2e tests and all cases have passed.

After the repair

success_arrow Success_detail

@Duansg Duansg requested review from Calvin979 and tomsun28 December 9, 2025 08:58
@Duansg Duansg added this to the 1.8.0 milestone Dec 9, 2025
Copy link
Member

@tomsun28 tomsun28 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@tomsun28 tomsun28 merged commit dc51ce7 into apache:master Dec 9, 2025
3 checks passed
@Duansg Duansg deleted the fix-3871 branch December 9, 2025 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Failed to add Prometheus monitor task

2 participants