Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MCOL-498 The knob to disable disk space preallocation for segment files. #738

Merged
merged 13 commits into from Apr 24, 2019
Merged

MCOL-498 The knob to disable disk space preallocation for segment files. #738

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
1d9f47a
MCOL-498. Segment files extension uses fallocate() now to optimize lo…
drrtuy Feb 28, 2018
81fe7fa
MCOL-498. Add the knob to disable segment|dict file preallocation. Di…
drrtuy Mar 6, 2018
7cf0d55
MCOL-498: Fill up the block with NULLs when CS touches for the first …
drrtuy Mar 15, 2018
6db8b1f
MCOL-498: Fill up the last used block with empty values, whilst doing…
drrtuy Mar 20, 2018
8037af5
MCOL-498 Fill up next block with empty values if insert values up to …
drrtuy Mar 25, 2018
46a46aa
MCOL-498 Support for non dict compressed columns.
drrtuy Aug 20, 2018
29becc2
MCOL-498 Passed test100.
drrtuy Dec 14, 2018
cbdcdb9
MCOL-498 Add DBRootX.PreallocSpace setting in the XML. Dict files ext…
drrtuy Dec 20, 2018
abf7ef8
MCOL-498 Changes made according with review suggestions.
drrtuy Jan 20, 2019
ecbf6b7
MCOL-498 Returned changes in we_colbufcompressed.* b/c they are necce…
drrtuy Jan 21, 2019
bc3c780
MCOL-498 Revived unit tests for writeengine/shared and add new tests
drrtuy Feb 20, 2019
22c0c98
MCOL-498 Reduced number of blocks created for abbreviated extents
drrtuy Apr 3, 2019
723ca53
Merge branch 'develop' into MCOL-498_5
LinuxJedi Apr 24, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Prev Previous commit
Next Next commit
MCOL-498: Fill up the block with NULLs when CS touches for the first … 
…time it with INSERT..VALUES.
  • Loading branch information
@drrtuy
drrtuy committed Apr 22, 2019
commit 7cf0d55dd0d43c24a0a405c050b7fbbbe77863fe
2 changes: 1 addition & 1 deletion writeengine/shared/we_fileop.cpp
View file
Open in desktop
Expand Up @@ -1879,7 +1879,7 @@ int FileOp::initDctnryExtent(
std::ostringstream oss;
std::string errnoMsg;
Convertor::mapErrnoToString(savedErrno, errnoMsg);
oss << "FileOp::initColumnExtent(): fallocate(" << currFileSize <<
oss << "FileOp::initDctnryExtent(): fallocate(" << currFileSize <<
", " << writeSize << "): errno = " << savedErrno <<
": " << errnoMsg;
logging::Message::Args args;
Expand Down
26 changes: 25 additions & 1 deletion writeengine/wrapper/we_colop.cpp
View file
Open in desktop
Expand Up @@ -1519,6 +1519,7 @@ void ColumnOp::setColParam(Column& column,
* rowIdArray - the array of row id, for performance purpose, I am assuming the rowIdArray is sorted
* valArray - the array of row values
* oldValArray - the array of old value
* bDelete - yet
* RETURN:
* NO_ERROR if success, other number otherwise
***********************************************************/
Expand All @@ -1533,6 +1534,8 @@ int ColumnOp::writeRow(Column& curCol, uint64_t totalRow, const RID* rowIdArray,
char charTmpBuf[8];
uint64_t emptyVal;
int rc = NO_ERROR;
bool fillUpWEmptyVals = false;
bool fistRowInBlock = false;

while (!bExit)
{
Expand All @@ -1551,8 +1554,18 @@ int ColumnOp::writeRow(Column& curCol, uint64_t totalRow, const RID* rowIdArray,
return rc;

bDataDirty = false;
// MCOL-498 We got into the next block, so the row is first in that block
// - fill the block up with NULLs.
if ( curDataFbo != -1 && !bDelete )
fillUpWEmptyVals = true;
}

// MCOL-498 CS hasn't touched any block yet,
// but the row fill be the first in the block.
fistRowInBlock = ( !(curRowId % (BYTE_PER_BLOCK / curCol.colWidth)) ) ? true : false;
if( fistRowInBlock && !bDelete )
fillUpWEmptyVals = true;

curDataFbo = dataFbo;
rc = readBlock(curCol.dataFile.pFile, dataBuf, curDataFbo);

Expand Down Expand Up @@ -1676,8 +1689,19 @@ int ColumnOp::writeRow(Column& curCol, uint64_t totalRow, const RID* rowIdArray,

// take care of the cleanup
if (bDataDirty && curDataFbo >= 0)
{
if ( fillUpWEmptyVals )
{
emptyVal = getEmptyRowValue(curCol.colDataType, curCol.colWidth);
int writeSize = BYTE_PER_BLOCK - ( dataBio + curCol.colWidth );
// MCOL-498 Add the check though this is unlikely at the moment of writing.
if ( writeSize )
setEmptyBuf( dataBuf + dataBio + curCol.colWidth, writeSize, emptyVal, curCol.colWidth );
fillUpWEmptyVals = false;
fistRowInBlock = false;
}
rc = saveBlock(curCol.dataFile.pFile, dataBuf, curDataFbo);

}
return rc;
}

Expand Down
4 changes: 2 additions & 2 deletions writeengine/wrapper/we_colop.h
View file
Open in desktop
Expand Up @@ -267,7 +267,7 @@ class ColumnOp : public DbFileOp
bool bDelete = false);

/**
* @brief Write row(s) for update and delete @Bug 1886,2870
* @brief Write row(s) for delete @Bug 1886,2870
*/
EXPORT virtual int writeRows(Column& curCol,
uint64_t totalRow,
Expand All @@ -277,7 +277,7 @@ class ColumnOp : public DbFileOp
bool bDelete = false);

/**
* @brief Write row(s) for update and delete @Bug 1886,2870
* @brief Write row(s) for update @Bug 1886,2870
*/
EXPORT virtual int writeRowsValues(Column& curCol,
uint64_t totalRow,
Expand Down