Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: SPU fails with "not enough for records" #2509

Closed
nacardin opened this issue Jul 27, 2022 · 2 comments
Closed

[Bug]: SPU fails with "not enough for records" #2509

nacardin opened this issue Jul 27, 2022 · 2 comments
Labels
bug Something isn't working no-stale Opt-out of closing issue due to no activity SPU SPU related Stability/Scalability
Milestone
0.10.4

Comments

@nacardin
Copy link
Contributor

What happened
No response from SPU when producing

Describe the setup

  • Fluvio Cloud 0.9.31

How to reproduce it (as minimally and precisely as possible)
Not sure

Log output
SPU:

2022-07-27T00:01:34.995869Z ERROR sc_dispatch_loop{socket=14}:handle_update_replica_request{sc_sink=fd(14)}: fluvio_spu::control_plane::dispatcher: error storage Log validation error: not enough for records
2022-07-27T00:01:34.995896Z ERROR sc_dispatch_loop{socket=14}:handle_update_replica_request{sc_sink=fd(14)}: fluvio_spu::control_plane::dispatcher: error storage Log validation error: not enough for records
2022-07-27T00:01:34.995902Z ERROR sc_dispatch_loop{socket=14}:handle_update_replica_request{sc_sink=fd(14)}: fluvio_spu::control_plane::dispatcher: error storage Log validation error: not enough for records
2022-07-27T00:01:34.995907Z ERROR sc_dispatch_loop{socket=14}:handle_update_replica_request{sc_sink=fd(14)}: fluvio_spu::control_plane::dispatcher: error storage Log validation error: not enough for records
@nacardin nacardin added the bug Something isn't working label Jul 27, 2022
@sehz
Copy link
Contributor

sehz commented Jul 27, 2022

@galibey Can you look into this?

@github-actions
Copy link

Stale issue message

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Oct 3, 2022
@morenol morenol added no-stale Opt-out of closing issue due to no activity and removed no-issue-activity labels Oct 3, 2022
@morenol morenol reopened this Oct 3, 2022
@sehz sehz added this to the 0.10.4 milestone Dec 17, 2022
@nacardin nacardin mentioned this issue Dec 17, 2022
Closed
@sehz sehz mentioned this issue Dec 28, 2022
Closed
bors bot pushed a commit that referenced this issue Dec 28, 2022
@sehz
feat: debug storage offsets (#2905)
a58561b 
changes related to issue #2509 
- Updated Storage CLI to validate log entries
- Use anyhow in the storage validation to simplify error handling.
- Internal refactoring of API to simplify validation handling such as `next` -> `try_next`
@sehz sehz mentioned this issue Dec 29, 2022
Closed
bors bot pushed a commit that referenced this issue Dec 30, 2022
@sehz
fix: recover from invalid segments (#2909)
30219ae 
resolves #2509.

Recover from invalid logs due to incorrect batch encoding.  It does this by setting the log size to the last correct batches.
Tested with data from corrupted segments.   

- More clean and simplification of the Log validator.
- Migrated more API to use anyhow
bors bot pushed a commit that referenced this issue Dec 30, 2022
@sehz
fix: recover from invalid segments (#2909)
86ddbd1 
resolves #2509.

Recover from invalid logs due to incorrect batch encoding.  It does this by setting the log size to the last correct batches.
Tested with data from corrupted segments.   

- More clean and simplification of the Log validator.
- Migrated more API to use anyhow
@bors bors bot closed this as completed in e68dcca Dec 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working no-stale Opt-out of closing issue due to no activity SPU SPU related Stability/Scalability
Projects
None yet
Milestone
0.10.4
4 participants
@sehz @nacardin @morenol @galibey