The current implementation contains the bug in that the metadata has a last blockId maintained along with only a mapping of blockId and the MD5 checksum of it. Thus there is no way that blocks can be consumed in the order in which they were added to the stream thus not providing a sequential access pattern for the blocks. For the correction, a list of blocks can be used to fix it. Two cases need to be carefully looked into
- PutBlock with locking
- PutBlock without locking
For case 1, only one writer can append blocks at any time thus there is no race condition. However in case 2, multiple writers may be writing to the stream at the same time and there can be interleaving between the writers. It is not necessary that the last blockId should strictly increase and this interleaving is perfectly fine. So proper handling of race condition should be done i.e. atomic updates for the last blockId, list of blocks and mapping of blockId to the checksum as a whole should be done.
The current implementation contains the bug in that the metadata has a last blockId maintained along with only a mapping of blockId and the MD5 checksum of it. Thus there is no way that blocks can be consumed in the order in which they were added to the stream thus not providing a sequential access pattern for the blocks. For the correction, a list of blocks can be used to fix it. Two cases need to be carefully looked into
For case 1, only one writer can append blocks at any time thus there is no race condition. However in case 2, multiple writers may be writing to the stream at the same time and there can be interleaving between the writers. It is not necessary that the last blockId should strictly increase and this interleaving is perfectly fine. So proper handling of race condition should be done i.e. atomic updates for the last blockId, list of blocks and mapping of blockId to the checksum as a whole should be done.