-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
filter on joined table broken #27463
Comments
Author Name: Giovanni Manghi (@gioman) Why tagging this as a "regression"? Did it worked as expected in 2.18 or another LTR release? 1 to many joins in QGIS are called relations, and you manage them in the project options. Could you please attach a sample project and data to demonstrate the problem? Thanks.
|
Author Name: Tobias Wendorff (Tobias Wendorff) Giovanni Manghi wrote:
See other report. I thought a regression is limited to the current major version.
I wasn't able to get it to work, but I'll figure it out in the next days :-)
After building an example, I've figured out that the problem occurs only SOMETIMES, if I'm joining against delimited text files! I'll build a sample project of course. |
Author Name: Tobias Wendorff (Tobias Wendorff) My god! After some more trial & error I've figured out something REALLY unexpected. +You will not trust your eyes+: the join works, but the filter does NOT work, if one of the column headers starts with a digit!
Check the demo file attached (join already created) Edit: typo (join works, filter doesn't)
|
Author Name: Giovanni Manghi (@gioman)
Confused, again. The Join in your sample project works. I also tried (on master) to create another join using as join field "1_1" from "demo_table" and it works (even if it does not make sense as there is nothing in "1_1" that can be joined in "demo_join"). Also here #27463 (comment) you state that you see the problem when joining delimited text layers, but in the sample you attached you use json, which is not a delimited text layer.
does this apply only to columns being used to define the join, or is just enough to be any column in the joined vector/table (anyway it seems I'm not confirming, see above)? Please clarify. Thanks. |
Author Name: Tobias Wendorff (Tobias Wendorff) Giovanni Manghi wrote:
Sorry, fixed it: the join works, but filter does not.
I first thought it's a problem of CSV. Then I did some more debugging and figured out that every file with an attribute table with having a number at first position of the column name makes the filter non working. Again: join works, filter doesn't.
It apply to any column. I'm joining "gitter_id_100m" (which has no leading number), the hoin works, but the filter doesn't. |
Author Name: Andrea Giudiceandrea (@agiudiceandrea) I've made some tests with QGIS 2.18 and now the issue reported by Tobias is clearer to me: normally, when you try to filter a layer (using the "Filter..." option from it's contextual menu) that is joined with a table, the "Filter on joined fields" window appears and if you click on "Yes" then: a virtual layer is created and added to the map and the Query Builder window of the virtual layer opens up with the fields of the original layer and the also the joined fields listed but, if the layer or the joined table contains a field whose name starts with a digit, then the virtual layer is not created (although the "Filter on joined fields" window appears and "Yes" is clicked) and the Query Builder window that opens up is that of the original layer so the joined fields aren't listed I can confirm this strange behavior in QGIS 2.18.19. |
Author Name: Andrea Giudiceandrea (@agiudiceandrea) It seems that SQL has problems with layer/table or field names whose first character is a digit. To handle this case (and also the additional case of white spaces in layer/table or field names), I think we can enclose layer/table and field names in double quotes (or square brackets) in the SQL query string generated by QgsVirtualLayerDefinitionUtils::fromJoinedLayer [1] during the virtual layer creation process. [1] https://github.com/qgis/QGIS/blob/master/src/core/qgsvirtuallayerdefinitionutils.cpp |
Author Name: Giovanni Manghi (@gioman) Andrea Giudiceandrea wrote:
yeah I imagined that the problem could be that. Is too much asking for another patch? |
Author Name: Andrea Giudiceandrea (@agiudiceandrea) Giovanni Manghi wrote:
There's another problem I realized only now that I have tested the tool... It seems (with QGIS 2.18.23 and 3.3.0) that the Filter tool for joined layers doesn't work as expected even with normal fields/tables/layers names (i.e. not starting with a digit or containing spaces): It's strange to me that Tobias hasen't noticed this bigger problem. Could you, Tobias and/or Giovanni, confirm this? |
Author Name: Giovanni Manghi (@gioman) Andrea Giudiceandrea wrote:
I have just tested this scenario on master/linux and can't confirm. |
Author Name: Giovanni Manghi (@gioman)
let me rephrase this: What I see is that the joined attributes do NOT show in the filter dialog even if the tables do have "normal" column names, i.e. nothing starting with a number. So it seems to me that the description of this issue must be re-written: filtering of joined layers is not working at all, regardless the naming of the columns. Will wait for confirmation before doing that. |
Author Name: Tobias Wendorff (Tobias Wendorff) Andrea Giudiceandrea wrote:
Hehe, I've only tested, if the filter comes up or not. Thanks for finding this out :-) Giovanni Manghi wrote:
I can't reproduce this. For me, the joined attributes only do NOT show in the filter dialog if the table has at least ONE column name starting with a number. But be aware: I've joined a dataset without geometry to a base table, which has geometry. It might be different, if you join two layers containing a geometry. |
Author Name: Andrea Giudiceandrea (@agiudiceandrea) My system is: Windows 7 64 bit Given data In the map window:
A join is added to testlayer (layer properties/joins window): Joinlayer: testtable | Join field: tableid | Target field: layerid Test projects (for 2.18 and 3) and layer files attached. Steps
Expected behavior ( see https://vimeo.com/123287077 and https://github.com/qgis/QGIS/blob/master/src/app/qgisapp.cpp#L9372-L9399 )
Actual behavior (on my system)
a) NOT OK : +no feature is drawn in the map+ although testlayer (virtual) +attribute table window shows 4 features+ (total: 4, filtered: 4, selected: 0) with the related values of fields from testlayer and joined fields from testtable
a) NOT OK : +no feature is drawn in the map+ and testlayer (virtual) +attribute table window shows no features+ (total: 4, filtered: 0, selected: 0) In the case of at least one field name starting with a digit (but also also containing spaces and also for layer names) as reported by Tobias, this is what happens: Actual behavior (on my system)
Partial workaround A partial workaround to the geometry missing bug is to edit the virtual layer settings (from the virtual layer contextual menu "Edit virtual layer"): in the Query section there is the sql query string
just add t.geometry among the fields to select:
and then click Add or OK to overwrite the virtual layer settings.
|
Author Name: Andrea Giudiceandrea (@agiudiceandrea)
|
Author Name: Giovanni Manghi (@gioman) Andrea Giudiceandrea wrote:
yeah, I confirm this observations... is safe to say that this functionality does not work as it should be on both 2.18 and 3.* Could any of you raise this matter in the developers mailing list? thanks!
I've tested this behavior on several systems. QGIS3 (even dev-nightlies) have this problem. There are no logs, about a virtual layer being created or not. to new description: https://issues.qgis.org/issues/19636#note-13 old description: I've tested this behavior on several systems. QGIS3 (even dev-nightlies) have this problem. There are no logs, about a virtual layer being created or not.
|
Author Name: Andrea Giudiceandrea (@agiudiceandrea) Giovanni Manghi wrote:
I'll write a report asap in qgis-developer with a proposed patch. |
Author Name: Jürgen Fischer (@jef-n)
https://issues.qgis.org/issues/19636#note-13 old description: I've tested this behavior on several systems. QGIS3 (even dev-nightlies) have this problem. There are no logs, about a virtual layer being created or not. to new description: old description: I've tested this behavior on several systems. QGIS3 (even dev-nightlies) have this problem. There are no logs, about a virtual layer being created or not. |
Author Name: Andrea Giudiceandrea (@agiudiceandrea) [QGIS-Developer] "Filter on joined fields" and Virtual layers not working as expected |
Author Name: Nyall Dawson (@nyalldawson)
|
Author Name: Andrea Giudiceandrea (@agiudiceandrea) Applied in changeset 05fda10.
|
Author Name: Tobias Wendorff (Tobias Wendorff)
Original Redmine Issue: 19636
Affected QGIS version: 3.3(master)
Redmine category:vectors
new description:
#27463 (comment)
old description:
When creating a table join using "joins" in "layer properties", you can't filter the resulting table. QGIS informs that it's needed to build a virtual layer to make filtering work. But even if you select "yes", the joined fields don't appear in attribute table. Also, it doesn't matter if you create a virtual join in memory or not. Sure, you create a join using "processing", but this creates a new dataset. I'm also missing a "drop none-joined rows" in here... this seems to be only available from "processing" (like 1-to-many)
I've tested this behavior on several systems. QGIS3 (even dev-nightlies) have this problem. There are no logs, about a virtual layer being created or not.
The text was updated successfully, but these errors were encountered: