v4 add custom delimiter for csv.

[tobhv]

Hello, v4 has reduced the amount of dependencies of libs. In order to profit optimally I want to try to move to csv only. However I am missing the spot how I can customize the delimiter and or support multiple different delimiters in version v4. Ideally the correct separator is automatically detected based on the language the user has and something the user can override during import or export.

[matthewhegarty]

django-import-export is based on tablib. I had a quick look at the tablib source to see how they handle custom delimiters, and it doesn't look to me that it is directly configurable, although it seems there is some code to detect delimiter. Perhaps if you created a subclass of a data format it would be possible. You'd have to read their docs and source to find out.

django-import-export doesn't allow for custom delimiters at present, but we could maybe support it if it was implemented in tablib.

[tobhv]

hi. from what i found it seems configurable in tablib:

import tablib
dataset = tablib.Dataset()
dataset.append(['Column1', 'Column2', 'Column3'])
dataset.append(['bla', 'bla2', 'bla3'])
with open('outputsemicolon.csv', 'w', newline='') as f:
    f.write(dataset.export('csv', delimiter=';'))

#during import delimiter is autodetected it seems:
with open('inputsemicolon.csv', 'r') as fh:
    imported_data = tablib.Dataset().load(fh, format='csv')
    print(imported_data)

#but it can also be specified as an argument
with open('inputcomma.csv', 'r') as fh:
    imported_data = tablib.Dataset().load(fh, format='csv', delimiter=',')
    print(imported_data)

source: https://tablib.readthedocs.io/en/stable/formats.html
or do you mean a different way of configuring it ?

[matthewhegarty]

Ah yes, looks like you can do it in tablib, although we don't support it in django-import-export. I guess it could be configurable. This would be the starting point.

[tobhv]

yes. simple example below:

from django.conf import settings
from import_export.formats.base_formats import TextFormat


class SCSV(TextFormat):
    TABLIB_MODULE = "tablib.formats._csv"
    CONTENT_TYPE = "text/csv"

    def export_data(self, dataset, **kwargs):
        return dataset.export('csv', delimiter=';')

in order to support one delimiter this code is fine. for example by moving the delimiter value to settings.
to support both some more work needed and will check on how import is working out. thanks again.

[tobhv]

and with working imports:

import tablib
from django.conf import settings
from import_export.formats.base_formats import TextFormat


class SCSV(TextFormat):
    TABLIB_MODULE = "tablib.formats._csv"
    CONTENT_TYPE = "text/csv"

    def create_dataset(self, in_stream, **kwargs):
        dataset = super().create_dataset(in_stream, **kwargs)
        dataset = tablib.Dataset().load(in_stream, format='csv', delimiter=';')
        return dataset

    def export_data(self, dataset, **kwargs):
        return dataset.export('csv', delimiter=';')

and then it looks like some more definition work is needed for various formats on each field if one moves to csv, but also this is configurable by defining each field appropriately in resources ?
date format in my case exports to 22/01/2024. but importing only works using format 2024-01-22.