Incremental writing of rows is very slow

[ShaheedHaque]

I'm using the following software:

$ sudo -H pip3 list | egrep 'odfpy|ods'
odfpy                         1.4.1               
pyexcel-ods                   0.6.0               

and the following test program:

#!/bin/env python3
from datetime import datetime

from pyexcel_io import writer

FILENAME = 'tmp.ods'
LIBRARIES = ['pyexcel-ods', 'pyexcel-ods3', 'pyexcel-odsw']
LIBRARY = LIBRARIES[0]
ROW = ['c' + str(c) for c in range(153)]


with writer.Writer('ods', library=LIBRARY) as file_stream:
    file_stream.open(FILENAME)
    page_stream = file_stream.writer.create_sheet('mysheet')
    #
    # Write 10, 100, 1000, ... rows.
    #
    row_count = 10
    for multiplier in range(4):
        before = datetime.utcnow()
        for row in range(row_count):
            ROW[0] = 'r' + str(row)
            page_stream.write_row(ROW)
        after = datetime.utcnow()
        print('row_count', row_count, 'seconds', (after - before).total_seconds())
        before = after
        row_count *= 10
    page_stream.close()
    after = datetime.utcnow()
    print('page_stream close', 'seconds', (after - before).total_seconds())
    before = after
after = datetime.utcnow()
print('file_stream close', 'seconds', (after - before).total_seconds())
before = after

and see the following output:

$ ./ods.py 
row_count 10 seconds 0.019138
row_count 100 seconds 0.191849
row_count 1000 seconds 2.262572
row_count 10000 seconds 23.984489
page_stream close seconds 5.154405
file_stream close seconds 27.650097

As you can see, it takes about 26s to write_row() 11110 times, and then about 33s to close the page_stream and file_stream. Is this as expected? Is there anything I can do to speed it up? I actually need to get to a row_count somewhere circa 200k (and possibly more), and that is clearly going to take quite a while with these numbers.

[ShaheedHaque]

I forgot to say, the memory consumption (on Ubuntu, current release) peaks with top showing a resident set size of 3.2GB. That feels like a clue!