Skip to content

librbd: rbd_aio_write_with_crc32c store CRC32C with initial value -1 to match msgr2 validation#66377

Merged
idryomov merged 1 commit intoceph:mainfrom
baum:rbd_aio_write_with_crc32c_initial_fix
Nov 27, 2025
Merged

librbd: rbd_aio_write_with_crc32c store CRC32C with initial value -1 to match msgr2 validation#66377
idryomov merged 1 commit intoceph:mainfrom
baum:rbd_aio_write_with_crc32c_initial_fix

Conversation

@baum
Copy link
Copy Markdown
Contributor

@baum baum commented Nov 23, 2025

librbd: rbd_aio_write_with_crc32c store CRC32C with initial value -1 to match msgr2 validation

Fix runtime error, using test command:

   sudo dd if=/dev/zero bs=32k of=/dev/nvme0n1 count=1

The error log:

2025-11-23T11:24:10.512+0000 7f30f4ec0640  1 --2- [v2:192.168.13.2:6802/3444906816,v1:192.168.13.2:6803/3444906816] >> 192.168.13.3:0/3916714748 conn(0x527d400 0x728f700 crc :-1 s=THROTTLE_DONE pgs=2038703 gs=2038723 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_read_frame_epilogue_main bad segment crc calculated=1136411986 expected=4294967295

msg2 validation (src/msg/async/frames_v2.cc:47):
Calls bufferlist::crc32c(-1) with initial value -1
uint32_t crc = segment_bl.crc32c(-1); // Uses initial value -1 (0xFFFFFFFF)

The cached CRC is stored with initial value 0, but msg2 calculates with initial value -1 (0xFFFFFFFF).

see #65901

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands

You must only issue one Jenkins command per-comment. Jenkins does not understand
comments with more than one command.

@baum baum requested a review from a team as a code owner November 23, 2025 14:43
@github-actions github-actions bot added the rbd label Nov 23, 2025
@baum baum requested a review from idryomov November 23, 2025 14:47
Copy link
Copy Markdown
Contributor

@idryomov idryomov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly the kind of thing I had in mind when suggesting to add a test in #65901 (comment).

If we are going to implicitly assume msgr2 here, that should at least be mentioned in librbd.h file where the public API is defined.

@baum baum force-pushed the rbd_aio_write_with_crc32c_initial_fix branch 3 times, most recently from aad0da7 to 6d60365 Compare November 23, 2025 23:45
@idryomov
Copy link
Copy Markdown
Contributor

The cached CRC is stored with initial value 0, but msg2 calculates with initial value -1 (0xFFFFFFFF).

The bufferlist code should be able to take a CRC for one initial value and transform it into a CRC for another initial value without repeating the full calculation (i.e. efficiently, without even looking at the data buffer). This means that as long as the precomputed CRC a) isn't manipulated with beforehand and b) gets stored with the correct initial value, the fact that there is a difference between msgr1 and msgr2 should be transparent to the user (except for the computational overhead of possibly needing to adjust the precomputed CRC).

If we are going to implicitly assume msgr2 here, that should at least be mentioned in librbd.h file where the public API is defined.

So after taking a deeper look, I don't think msgr2 or anything about its internals is worth mentioning there. We just need to clearly state what initial value is expected (or consider accepting it as a separate argument to be as flexible as possible).

@baum baum changed the title librbd: rbd_aio_write_with_crc32c store CRC32C with initial value -1 to match msg2 validation librbd: rbd_aio_write_with_crc32c store CRC32C with initial value -1 to match msgr2 validation Nov 25, 2025
@baum baum force-pushed the rbd_aio_write_with_crc32c_initial_fix branch from 6d60365 to 4c422a7 Compare November 25, 2025 07:53
Fix runtime error, using test command:
   sudo dd if=/dev/zero bs=32k of=/dev/nvme0n1 count=1

The error log:
   2025-11-23T11:24:10.512+0000 7f30f4ec0640  1 --2- [v2:192.168.13.2:6802/3444906816,v1:192.168.13.2:6803/3444906816] >> 192.168.13.3:0/3916714748 conn(0x527d400 0x728f700 crc :-1 s=THROTTLE_DONE pgs=2038703 gs=2038723 cs=0 l=1 c_cookie=0 s_cookie=0 reconnecting=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_read_frame_epilogue_main bad segment crc calculated=1136411986 expected=4294967295

Ceph msgr2 validation (ceph/src/msg/async/frames_v2.cc:47):
   uint32_t crc = segment_bl.crc32c(-1);  // Uses initial value -1

Co-authored-by: Ilya Dryomov <idryomov@gmail.com>
Signed-off-by: Alexander Indenbaum <aindenba@redhat.com>
@baum baum force-pushed the rbd_aio_write_with_crc32c_initial_fix branch from 4c422a7 to 36d99e3 Compare November 26, 2025 08:06
@idryomov
Copy link
Copy Markdown
Contributor

jenkins test make check

1 similar comment
@idryomov
Copy link
Copy Markdown
Contributor

jenkins test make check

@idryomov idryomov merged commit 7e93afe into ceph:main Nov 27, 2025
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants