Please sign in to comment.
make corrupted file handling more robust
Summary: On detecting corruption in a backed up file, stop reading any further. Calculate how many bytes are being lost. Output this information in LOG messages and updated the bytes lost counter. StdFile::readNext can only work for framed files. The dead code for reading from non-framed files was anyway wrong - it allocated only 4K for a message line while there are many messages longer than that. I removed this dead code. I haven't yet added checksums - what the task asks for. I will not close the task after this diff Test Plan: 1/ Make sure that the regular code when the backup file is not corrupted works. 2/ Force backup. corrupt backup file by changing frame size field. move the scribe-server to sending_buffer state. The backup file is removed, as much data that could be sent is sent and log and counters have the loss information. [Tue May 25 23:22:35 2010] "WARNING: Corruption Data Loss -14 bytes in /tmp/corr/foo/foo_00000" scribe_overall:bytes lost: 28 scribe_overall:received good: 7 foo:received good: 7 scribe_overall:retries: 154 foo:bytes lost: 28 <=== foo:retries: 154 scribe_overall:sent: 3 3/ same as 2/ but when uploading the backup file there is an error. The backup file is left as it is. In this situation when the backup file was being read the LOG will contain info that x bytes were lost. But the bytes lost counter won't go up. The bytes lost counter only goes up when the corrupted file is being deleted. DiffCamp Revision: 118317 Reviewed By: groys CC: agiardullo, pkhemani, groys, scribe-dev@lists Tasks: Revert Plan: OK git-svn-id: svn+ssh://tubbs/svnapps/fbomb/branches/scribe-os/fbcode/scribe@28770 2248de34-8caa-4a3c-bc55-5e52d9d7b73a
- Loading branch information...
Showing with 114 additions and 55 deletions.