"Errors: SUCCESS: 6 attribute(s) added. ERROR: field with index 4 is not the same!" when adding columns to vectors

[qgib]

Author Name: Giovanni Manghi (@gioman)
Original Redmine Issue: 8481
Affected QGIS version: master
Redmine category:vectors

---

Add a column in a PostGIS or Spatialite vector (or Shapefile, but does not seems to happen always in this case), then toggle editing to save edits.

The new status bar will pop up, and if you go read the message you will see something like

@errors: SUCCESS: 6 attribute(s) added. ERROR: field with index 4 is not the same!@

The edit is usually successful anyway, sometimes it is needed to toggle editing again.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

This is related to definitions of field types in the QGIS provider implementation which do not match the field type which is created by the driver in the background.

Therefore it is essentially to have a list of affected providers / field type pairs.

OGR Provider : Date is fixed in d758340
OGR Provider : DateTime is not supported by all OGR drivers. E.g. Shapefile does not support it. Therefore, this fieldtype needs special attention and we need to find eigher a way to query OGR drivers if they support this type or create and maintain a blacklist/whitelist.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

OGR Provider : DateTime Blacklisted this datatype for ShapeFile in a8a1090. Please help to extend the list for other formats which do not support this field type. Querying OGR for support is unfortunately not possible according to #gdal.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

For what it can matter I noticed that on a postgis layer there is no error just when adding "numeric" and "decimal" fields (cannot test real and double because of #17247) and it gives the error when choosing any other type.

In my opinion among the list of blockers this issue is the really important one as it is a regression of a pretty basic (q)qgis function. Any chance to have this fixed (in time)?

thanks in advance

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Giovanni Manghi wrote:

For what it can matter I noticed that on a postgis layer there is no error just when adding "numeric" and "decimal" fields (cannot test real and double because of #17247) and it gives the error when choosing any other type.

on SL the error shows when adding integer/double column types, but not "text".

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

2f2e088 solves this partially

The other half of the problem is, that the [qgis] spatialite provider indicates, that you can define length and precision of these fields between 0 and 20, while sqlite does in fact not support this. Therefore afterwards any column will have a default length (0) and default precision (0) assigned.

I would propose to remove the option to define these (useless) options from the spatialite drivers.

For now: As long as you don't change them (i.e. leave them set to 0) you should not get any warnings.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Hi Matthias

For now: As long as you don't change them (i.e. leave them set to 0) you should not get any warnings.

In SL I get the error also when creating integer fields even when leaving 0 as width, does the above patch fix this case? Thanks!

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Giovanni Manghi wrote:

In SL I get the error also when creating integer fields even when leaving 0 as width, does the above patch fix this case? Thanks!

Exactly

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Matthias Kuhn wrote:

Giovanni Manghi wrote:

In SL I get the error also when creating integer fields even when leaving 0 as width, does the above patch fix this case? Thanks!

Exactly

sweet. I guess that fixing postgresql is a different beast(?). thanks again.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Fixed the trivial ones in a9c05d7

Remaining problems are:

• Postgres: Field length and precision are not correctly read from the database. (For @char@ type with configurable length)
• Spatialite: Field length and precision are configurable although they should not be. (Easy fix, but I need confirmation that the length is really not used by anything)

Giovanni, could you also test the other providers, that the changes did not introduce any errors which have been masqueraded by the add attribute dialog behavior before. Thank you very much.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Giovanni, could you also test the other providers, that the changes did not introduce any errors which have been masqueraded by the add attribute dialog behavior before. Thank you very much.

Hi Matthias, for the moment I re-tested postgis, SL and shapes.

Postgis:
now I see the error only when adding a char (text, fixed length) column

SL:
I see no errors when adding columns and leave width/precision to 0, but when adding an integer column I see that the values stored are still decimal

SHP:
no problems

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• fixed_version_id was changed from Version 2.0.0 to Future Release - High Priority

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Fixed the spatialite problems in 249526b

No other value than 0 is allowed for integer and real columns.
I'm pretty confident, that this is ok. Length / precision settings will be lost anyway when saving the fields to the provider according to https://www.sqlite.org/datatype3.html#storageclasses

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

I was hesitating to backport this to the 2.0 fixes but if there is demand, I can do so. The task itself is easy.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

Is this still an issue?

Looking at my earlier comments, I didn't take care of this one: "Postgres: Field length and precision are not correctly read from the database. (For char type with configurable length)"

Can somebody confirm this (or are there others affected)

---

• status_id was changed from Open to Feedback

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Matthias Kuhn wrote:

Is this still an issue?

Looking at my earlier comments, I didn't take care of this one: "Postgres: Field length and precision are not correctly read from the database. (For char type with configurable length)"

Can somebody confirm this (or are there others affected)

when I try to add a "char" column on a PostGIS layer, on save I get

Could not commit changes to layer sele

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 53 is not the same!

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• fixed_version_id was changed from Future Release - High Priority to Version 2.2
• status_id was changed from Feedback to Open

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Giovanni Manghi wrote:

Matthias Kuhn wrote:

Is this still an issue?

Looking at my earlier comments, I didn't take care of this one: "Postgres: Field length and precision are not correctly read from the database. (For char type with configurable length)"

Can somebody confirm this (or are there others affected)

when I try to add a "char" column on a PostGIS layer, on save I get

Could not commit changes to layer sele

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 53 is not the same!

PostGIS:

still getting error when saving a newly created char column, example

Could not commit changes to layer evora

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 8 is not the same!

after discarding the edits the column seems that can be updated without errors

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Giovanni Manghi wrote:

Giovanni Manghi wrote:

Matthias Kuhn wrote:

Is this still an issue?

Looking at my earlier comments, I didn't take care of this one: "Postgres: Field length and precision are not correctly read from the database. (For char type with configurable length)"

Can somebody confirm this (or are there others affected)

when I try to add a "char" column on a PostGIS layer, on save I get

Could not commit changes to layer sele

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 53 is not the same!

PostGIS:

still getting error when saving a newly created char column, example

Could not commit changes to layer evora

Errors: SUCCESS: 1 attribute(s) added.
ERROR: field with index 8 is not the same!

after discarding the edits the column seems that can be updated without errors

a recent commit by Jürgen has fixed this.

---

• status_id was changed from Open to Closed
• resolution was changed from to fixed/implemented

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

I have seen this again, on master, working on a shapefile

Errors: SUCCESS: 1 attribute(s) added.

ERROR: field with index 8 is not the same!

---

• status_id was changed from Closed to Reopened
• fixed_version_id was changed from Version 2.2 to Version 2.4
• resolution was changed from fixed/implemented to

[qgib]

Author Name: Jürgen Fischer (@jef-n)

---

Giovanni Manghi wrote:

I have seen this again, on master, working on a shapefile

Errors: SUCCESS: 1 attribute(s) added.

ERROR: field with index 8 is not the same!

And which types did you use?

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

And which types did you use?

will try to replicate again and let you know.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

And which types did you use?

just got it when after adding an integer column to a point file, and filling it with field calculator and the rand function.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

and just seen this on a PostGIS layer, with QGIS master

Errors: SUCCESS: 1 attribute(s) added.

ERROR: the count of fields is incorrect after addition/removal of fields!

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

When updating this issue, please provide details not only about the provider, but also the data type you used, this considerably increases the chances of having the issue fixed ;-)

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

Matthias Kuhn wrote:

When updating this issue, please provide details not only about the provider, but also the data type you used, this considerably increases the chances of having the issue fixed ;-)

Hi Matthias,

will do it. Anyway I have already done it (sometimes), see for example in #17248 (comment)

cheers!

PS
the problem with this issue is that is not easily replicable.

[qgib]

Author Name: Jürgen Fischer (@jef-n)

---

Giovanni Manghi wrote:

will do it. Anyway I have already done it (sometimes), see for example #17248 (comment)

Well, the problem is the mapping between QgsField (name, type, precision) to a provider specific type on creationg of the field and the retrieval of field definitions from a provider and their mapping back to QgsField. After the field is created, the provider retrieves the new field definitions and checks it the QgsField created from that, matches what you originally entered. A mismatch produces the error message.

For instance the postgres provider didn't care to retrieve the length of character fields and therefore each character field with a given length constraint would produce that error - because the provider wouldn't reflect the length constraint in the QgsField it reported (and in turn not limit edit widget in forms to that given length later and lead to a commit error if the user entered a value exceeding that length). That's what the related commit above fixes.

The spatialite fix however removes the ability to specify a length constraint for float and integer field in spatialite databases as spatialite would ignore and drop constraint anyway and lead to the same error message - although it's actually a different problem.

With other types and/or providers there might be other different mismatches (the name might change case or be truncated to a certain length, types might change (eg. when a integer width exceeds some limit, it might be turned into a real type). So to reproduce we need to know what combination was actually used (point file and integer doesn't really cut it ;)).

But reporting and fixing another combination that produces this error wouldn't 'fix' this ticket. Maybe we should close it in favor of individual tickets for each failing combination we find.

[qgib]

Author Name: Matthias Kuhn (@m-kuhn)

---

maybe we should put some more details into the log containing dumps of the specified and provider reported field definitions, so there is a bit more information about what failed.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

It is a few days now that I don't see anymore this kind of messages, before it was quite common.

Maybe we can lower the priority of this ticket?

---

• status_id was changed from Reopened to Feedback

[qgib]

Author Name: Jürgen Fischer (@jef-n)

---

Giovanni Manghi wrote:

It is a few days now that I don't see anymore this kind of messages, before it was quite common.

Maybe we can lower the priority of this ticket?

As said to me this are individual issues - although there is always the same error message popping up. So I close this and open new ones when a new combinations where creation and retrieval of types disagree is identified.

[qgib]

Author Name: Giovanni Manghi (@gioman)

---

• status_id was changed from Feedback to Closed