Import data with foreign key

[DayDotMe]

Hello and first of all, thanks for this package !

I'm having issue when trying to import data with a foreign key field. The code doesnt raise any error, but nothing is saved in database. I've tried using custom initialize as in the documentation, however though I obtain correct values, no object is saved in database. When trying the same code with entities without foreign keys, everything goes fine.

Please find attached my function code. Thanks in advance for any help.
ps : here is what print(row) prints : (types and values are as expected)
['SCHNEIDER ELECTRIC', 'A9L16577', 'v8', 'Z130', 'B', 'aaa', 1, 'dummy', <Groupe: Serveurs>]
`
def import_xls(item, form):

    if form.is_valid():
        if request.FILES['file'].content_type.find('spreadsheet') == -1:
            messages.add_message(request, messages.ERROR, 'Le fichier n\'est pas un tableur !')
            return
        fields = []
        temp = item._meta.get_fields()
        for field in temp:
            if not field.name.startswith("article") and not field.name == 'id' and not isinstance(field,ManyToManyField)and not isinstance(field, ManyToManyRel):
                fields.append(field.name)
        if item == ArticleHW or item == ArticleSW: #no foreign keys, everything goes fine
            try:
                request.FILES['file'].save_to_database(model=item, mapdict=fields)
                messages.add_message(request, messages.SUCCESS, 'Enregistrement : Succès')
            except ValueError:
                messages.add_message(request, messages.ERROR, 'Erreur : Le nombre de champs ne correspond pas !')
        elif item == Equipement:
            try:
                def set_group(row):
                    group = Groupe.objects.get(id=int(row[8]))
                    row [8] = group
                    print(row)
                    return row
                #this is not saving 
                request.FILES['file'].save_to_database(model=item, mapdict=fields, initializer=set_group)           
                messages.add_message(request, messages.SUCCESS, 'Enregistrement : Succès')
            except ValueError as e:
                print(e)
                messages.add_message(request, messages.ERROR, 'Erreur : Le nombre de champs ne correspond pas !') `

[chfw]

Are you able to check your log file see what cause it to fail?

pyexcel-io v0.4.x line:
https://github.com/pyexcel/pyexcel-io/blob/master/pyexcel_io/database/importers/django.py#L51

and pyexcel-io v0.3.x line:

https://github.com/pyexcel/pyexcel-io/blob/v0.3.x/pyexcel_io/database/django.py#L68

[DayDotMe]

God thank you for answer, it was a problem with the excel file formatting. However it's a bit of a shame to have to go through pyexcel logs to find this kind of error. Anyway thanks again for your work and your answer.

[chfw]

Hmm... what if the save_to_database throw a exception in the first place? would it help you find the problem early?

[DayDotMe]

Yes to me it would definitely be a great feature.
For instance I added a length check to save_as() in core.py and the corresponding error message to warn the user if he tries to import files with more fields than in the model and I'll probably add importers messages.

[DayDotMe]

Hi again. I still have trouble with these imports and this time nothing is logged by django.py
Actually, when I try to save in database, django throws a value error with this message :
"<Equipement: SIEMENS>" needs to have a value for field "id" before this many-to-many relationship can be used
Below is my code:

           try:
                def set_group(row):
                    try: # if group exists its assigned, if not i create a new one, no issue there
                        group = Groupe.objects.filter(name__contains=str(row[8]))[0]
                    except IndexError:
                        group = Groupe(name=str(row[8]))
                        group.save()
                    row[8] = group
                    return row
                request.FILES['file'].save_to_database(model=item, mapdict=fields, initializer=set_group)
                messages.add_message(request, messages.SUCCESS, 'Enregistrement : Succès')
            except ValueError as e:
                messages.add_message(request, messages.ERROR, 'Erreur : ' + str(e) )

Thanks in advance for any help.

[chfw]

This means that equipment need to be saved in the database before the database give an id back.

[DayDotMe]

Yes I know, but equipment should be saved by save_to_database (item is of type Equipment).

[chfw]

Alright, in this case where nothing is left, it is assumed that the item is created. However, the item was not there in the database.

I would suggest that to set a break point in the django.py, right after the close() method. And then try upload items again. step through the code and see.

[chfw]

pyexcel-io v0.4.1 with explicit exception is released.

[DayDotMe]

I was in a rush so I have written a workaround but I will debug this, try v0.4.1 and update

[DayDotMe]

New issue with pyexcel-io, here is my debugger content if it can be of any use to you.

I tried commenting this line, but then it'll throw an attribute error: "module 'pyexcel' has no attribute 'save_as' ". In fact, a coworker already faced this error with v0.4.0 of pyexcel though we installed it and django-excel exactly in the same way.

[chfw]

You are reproducing this problem here. Are you able to try logging it? At the meantime, I checked the package files pyexcel 0.5.0 and pyexcel-io 0.4.x and the packages are not having the same issue as this one: pyexcel/pyexcel-io#37, where an extra folder 'pyexcel' accidentally appeared in pyexcel-io.