Skip to content

Commit ed9fda2

Browse files
committed
Speed up CSV import by not querying the stream position
Avoid querying the position in the text stream using Qt's pos() function to update the progress dialog. Instead keep track of the stream position manually. This is possible here because we don't ever seek in the file. In result, this speeds up the CSV import dramatically.
1 parent a03a901 commit ed9fda2

File tree

1 file changed

+3
-1
lines changed

1 file changed

+3
-1
lines changed

src/csvparser.cpp

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -140,9 +140,11 @@ CSVParser::ParserResult CSVParser::parse(csvRowFunction insertFunction, QTextStr
140140
};
141141
FieldBufferDealloc dealloc(record);
142142

143+
qint64 bufferPos = 0;
143144
while(!stream.atEnd())
144145
{
145146
sBuffer = stream.read(m_nBufferSize).toUtf8();
147+
bufferPos += sBuffer.length();
146148
auto sBufferEnd = sBuffer.constEnd();
147149

148150
for(auto it = sBuffer.constBegin(); it != sBufferEnd; ++it)
@@ -276,7 +278,7 @@ CSVParser::ParserResult CSVParser::parse(csvRowFunction insertFunction, QTextStr
276278

277279
if(m_pCSVProgress && parsedRows % 100 == 0)
278280
{
279-
if(!m_pCSVProgress->update(stream.pos()))
281+
if(!m_pCSVProgress->update(bufferPos))
280282
return ParserResult::ParserResultCancelled;
281283
}
282284
}

0 commit comments

Comments
 (0)