Skip to content

libvncserver: avoid colour map alignment issues#710

Open
marcofortina wants to merge 1 commit into
LibVNC:masterfrom
marcofortina:fix/colour-map-alignment
Open

libvncserver: avoid colour map alignment issues#710
marcofortina wants to merge 1 commit into
LibVNC:masterfrom
marcofortina:fix/colour-map-alignment

Conversation

@marcofortina
Copy link
Copy Markdown
Contributor

Summary

Fixes the pointer aliasing/alignment issue in rfbSendSetColourMapEntries().

The previous code built a wire buffer as char[], then cast it to rfbSetColourMapEntriesMsg * and uint16_t *. That can create unaligned accesses on strict-alignment architectures and also relies on type-punning through incompatible pointer types.

Changes

  • Write the SetColourMapEntries header directly into the byte buffer.
  • Write 16-bit wire values through a small memcpy() helper instead of casting the byte buffer to uint16_t *.
  • Keep the existing wire layout and endian conversion behavior.
  • Add a NULL check for the dynamic buffer allocation path used when nColours > 256.

Validation

Tested with a minimal local CMake configuration:

cmake -S . -B build-668-patch \
  -DWITH_EXAMPLES=OFF \
  -DWITH_TESTS=ON \
  -DWITH_OPENSSL=OFF \
  -DWITH_GNUTLS=OFF \
  -DWITH_GCRYPT=OFF \
  -DWITH_SDL=OFF \
  -DWITH_GTK=OFF \
  -DWITH_QT=OFF \
  -DWITH_FFMPEG=OFF \
  -DWITH_XCB=OFF \
  -DWITH_LIBSSHTUNNEL=OFF \
  -DWITH_SYSTEMD=OFF \
  -DCMAKE_BUILD_TYPE=Debug
cmake --build build-668-patch --parallel 1
ctest --test-dir build-668-patch --output-on-failure

Result:

100% tests passed, 0 tests failed out of 5

Notes

Closes #668.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

pointer aliasing/alignment issue in rfbSendSetColourMapEntries()

2 participants