Tab as delimiter

[0xDAFE]

Hi there,

Tested version: pantables 0.12.2

I would like to use pantable with inline content and the csv-kwargs options to use TAB / "\t" as a delimiter for the columns (see example document).
pantable-example.md.txt

Throwing in some debug log statements in pantable.py

def read_data(include, data, encoding=None, csv_kwargs={}):
    """Parse CSV table.

    `include`: path to CSV file or None. This is prioritized first.

    `data`: str of CSV table.

    Return None when the include path is invalid.
    """
    import logging
    logging.basicConfig(level=logging.DEBUG)
    log = logging.getLogger()
    log.debug("csv_kwargs: " + str(csv_kwargs))
    if (csv_kwargs["delimiter"] == "\t"):
        log.debug("Correct tab delimiter")
        log.debug(data.encode('unicode_escape'))
        log.debug("Header 1\tHeader2\tHeader3\nX\tY\tZ".encode('unicode_escape'))
    elif (csv_kwargs["delimiter"] == "\\t"):
        log.debug("Escaped delimiter!")
    
    with (io.StringIO(data) if include is None else io.open(str(include), encoding=encoding)) as f:
        table_list = list(csv.reader(f, **csv_kwargs))
        log.debug(table_list)

    if not table_list:
        raise EmptyTableError

    return table_list

pandoc -F pantable -o TEST.html pantable-example.md
DEBUG:root:csv_kwargs: {'delimiter': '\t'}
DEBUG:root:Correct tab delimiter
DEBUG:root:b'Header 1    Header 2    Header 3\\nX   Y   Z'
DEBUG:root:b'Header 1\\tHeader2\\tHeader3\\nX\\tY\\tZ'
DEBUG:root:[['Header 1    Header 2    Header 3'], ['X   Y   Z']]
pantable: alignment string is too long, truncated.
DEBUG:root:csv_kwargs: {'delimiter': ','}
DEBUG:root:[['Header 1', 'Header 2', 'Header 3'], ['X', 'Y', 'Z']]

What I can conclude is that the tabs seem to be getting lost at some point passing around pandoc and the filters but I cannot put my finger on where exactly.

Do you have an idea how to fix this and use tab-delimited data?

[0xDAFE]

Ok, the answer was right in front of me. In case anyone comes across this issue through search engines:
The pandoc documentation clearly states the option
-p, --preserve-tabs

Preserve tabs instead of converting them to spaces. (By default, pandoc converts tabs to spaces before parsing its input.) Note that this will only affect tabs in literal code spans and code blocks. Tabs in regular text are always treated as spaces.