Checksum state files written to disk #7586

Closed
dakrone opened this Issue Sep 4, 2014 · 0 comments

Projects

None yet

2 participants

@dakrone
Member
dakrone commented Sep 4, 2014

We write a number of state files to disk, including persistent cluster settings, the state of indices and so forth. In order to detect disk corruption in these files we should also write the checksum of these files to disk. On read we can verify the integrity of these files if a checksum file is present.

@dakrone dakrone self-assigned this Sep 4, 2014
@s1monw s1monw assigned s1monw and unassigned dakrone Oct 6, 2014
@s1monw s1monw added a commit to s1monw/elasticsearch that referenced this issue Oct 15, 2014
@s1monw s1monw [CORE] Add cluster and index state checksums
This commit adds checksumming for cluster and index states. It moves
from a plain XContent based on-disk format to a more structured binary
format including a header and footer as well as a CRC32 checksum for
each of these files. Since previous versions didn't write any format
identifier etc. this commit adds a file extension `.st` for states
that have header/footer and checksum.
This commit also moves over to write temporary files and moves them into
place with an atomic move operation. This commit also serializes and
deserializes the states without materilazing them into opaque memory.

Closes #7586
8dddac3
@s1monw s1monw added a commit that referenced this issue Oct 15, 2014
@s1monw s1monw [CORE] Add cluster and index state checksums
This commit adds checksumming for cluster and index states. It moves
from a plain XContent based on-disk format to a more structured binary
format including a header and footer as well as a CRC32 checksum for
each of these files. Since previous versions didn't write any format
identifier etc. this commit adds a file extension `.st` for states
that have header/footer and checksum.
This commit also moves over to write temporary files and moves them into
place with an atomic move operation. This commit also serializes and
deserializes the states without materilazing them into opaque memory.

Closes #7586
c399509
@s1monw s1monw closed this in #8010 Oct 15, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment