Discard Empty Fields: behaviour, default status, name

[qgib]

Author Name: Simon Dedman (@SimonDedman)
Original Redmine Issue: 19071
Affected QGIS version: 3.0.3
Redmine category:data_provider/delimited_text_

---

Imagine a csv with columns:
lat, long, 2016_data, 2017_data, 2018_data, Total
Where a selection of the 2016:18 cells are blank.
If one imports this as a layer and doesn't notice the 'discard empty fields' option is on (by default, 2.18.20 & 3.0.3), Qgis will not discard empty FIELDS, as defined elsewhere (e.g. layer properties) as being the columns of data, but will discard empty CELLS.... and then slide all cells left to fill the blanks, such that e.g.:

lat, long, 2016_data, 2017_data, 2018_data, Total
la1 lo1 2 =blank= 3 5

would become
lat, long, 2016_data, 2017_data, 2018_data, Total
la1 lo1 2 3 5 =blank=

and therefore plotting Total would show you that there are no data when you can see from your original csv that there certainly are.
This may be the correct implementation for fields for other file types but is very much counterintuitive and bad for csvs. Please could:

1. 'discard empty fields' be turned off by default, either for csvs or all
2. 'discard empty fields' be renamed to e.g. 'remove blank cells', ideally with explanatory tooltip
3. This behaviour be corrected so blank fields i.e. columns of data are removed but not cells, or rather not with the post-removal rearrangement of other fields' data.

Thanks.

[qgib]

Author Name: Alexander Bruy (@alexbruy)

---

• operating_system was changed from Win10 to
• category_id was changed from Processing/Core to Data Provider/Delimited Text

[alexbruy]

@ccrook what is your opinion on this?

[Pedro-Murteira]

Tested on versions 3.22.4 and 3.24.0. "Discard Empty Fields" seems to be turned off by default. Did you come across this issue in more recent versions of QGIS? @SimonDedman

[github-actions]

The QGIS project highly values your report and would love to see it addressed. However, this issue has been left in feedback mode for the last 14 days and is being automatically marked as "stale".
If you would like to continue with this issue, please provide any missing information or answer any open questions. If you could resolve the issue yourself meanwhile, please leave a note for future readers with the same problem and close the issue.
In case you should have any uncertainty, please leave a comment and we will be happy to help you proceed with this issue.
If there is no further activity on this issue, it will be closed in a week.

[github-actions]

While we hate to see this happen, this issue has been automatically closed because it has not had any activity in the last 42 days despite being marked as feedback. If this issue should be reconsidered, please follow the guidelines in the previous comment and reopen this issue.
Or, if you have any further questions, there are also further support channels that can help you.