Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The queries are not working correctly for multiple tags #9923

Closed
richtera opened this issue Sep 29, 2018 · 5 comments
Closed

The queries are not working correctly for multiple tags #9923

richtera opened this issue Sep 29, 2018 · 5 comments
Assignees
Labels
bug [triage] something behaving unexpectedly server / core Issues relating to the server or core of Ghost

Comments

@richtera
Copy link

richtera commented Sep 29, 2018

Welcome to Ghost's GitHub repo! 👋🎉

Issue Summary

SQL statements not correctly generated for tags. When a post has multiple tags or you're querying for multiple tags the resulting sql will not be correct. My Issue #9916 got closed but this is not working correctly as far as I can see.

To Reproduce

I have this routing:

routes:
  /units/:
    controller: channel
    filter: tags:unit
    template: units
    order: slug

collections:
  /unit/:
    permalink: /unit/{slug}/
    filter: tags:unit
    type: custom
    frontPageTemplate: units
    template:
      - unit

  /:
    permalink: /{slug}/
    filter: tags:-[article,unit,activity]
    template:
      - index

taxonomies:
  tag: /tag/{slug}/
  author: /author/{slug}/

The SQL being generated is:

select `posts`.*
from `posts`
	left outer join `posts_tags` on `posts_tags`.`post_id` = `posts`.`id`
	left outer join `tags` on `posts_tags`.`tag_id` = `tags`.`id`
where (`posts`.`status` = 'published'
	and `posts`.`page` = false)
	and (`tags`.`slug` not in ('article', 'unit', 'activity'))
group by `posts`.`id`
order by CASE WHEN posts.status = 'scheduled' THEN 1 WHEN posts.status = 'draft' THEN 2 ELSE 3 END ASC,
	CASE WHEN posts.status != 'draft' THEN posts.published_at END DESC,
        posts.updated_at DESC,
	posts.id DESC

Some of the sample posts have the following tags:

Post 1: unit-special, unit
Post 2: activity-unit-special, activity
Post 3: article-unit-special, article
Post 4:

I tried this with single special tags but that didn't work either. Basically the join should be multiple or an inner sub query rather than doing a group by to stitch it back together. Using the group by will consider each tag in turn and cannot do an AND or NOT OR across the different rows in the group by.

Technical details:

When going to the home page all 4 articles are considered and then filtered out. In the actual system I have about 2300 posts and the filter will filter out the first 50 and display no posts on my home page.

  • Ghost Version: 2.1.4
  • Node Version: 8.12
  • Browser/OS: chrome mac
  • Database: jawsdb mysql
@kirrg001 kirrg001 self-assigned this Oct 1, 2018
@kirrg001
Copy link
Contributor

kirrg001 commented Oct 1, 2018

Hey again!

https://docs.ghost.org/docs/dynamic-routing#section-filter

Can you pls read this section and adapt your routes.yaml? Currently, you have to use 100% inverse filters when using multiple collections.

@kirrg001 kirrg001 added the needs:info [triage] Blocked on missing information label Oct 1, 2018
@richtera
Copy link
Author

richtera commented Oct 1, 2018

That's precisely what I am trying to do. The inverse filter tags:-[tag1,tag2,tag3] does not generate the correct SQL for posts that have multiple tags themselves since it uses a join and a group by. It would need to use a sub-select across tags for it to work.
NOTE: Every tag other than "tag1", "tag2" and "tag3" will pass through as a TRUE.

@kirrg001
Copy link
Contributor

kirrg001 commented Oct 1, 2018

Yeah can confirm. The actual SQL query is not correct when the post has multiple tags.

@kirrg001 kirrg001 added bug [triage] something behaving unexpectedly server / core Issues relating to the server or core of Ghost and removed needs:info [triage] Blocked on missing information labels Oct 1, 2018
@kirrg001
Copy link
Contributor

kirrg001 commented Oct 7, 2018

I'll close this issue in favor of #9950. It's the same report. I've left a comment and linked your report.

Ghost uses GQL since years and GQL generates the SQL queries for Ghost. These queries are broken for relations, tracked e.g. here.

Dynamic Routing uses NQL only to match JSON objects, but not to generate the SQL queries. That's why you are running into this bug, which is tracked since 2016.

We have to finish up the NQL implementation and replace GQL with NQL in Ghost (in the model layer).

NQL will take care of the challenge to generate the correct SQL queries for relations.

Thanks for your report 👋

@kirrg001 kirrg001 closed this as completed Oct 7, 2018
kirrg001 added a commit that referenced this issue Dec 11, 2018
refs #10105, closes #10108, closes #9950, refs #9923, refs #9916, refs #9574, refs #6345, refs #6309, refs #6158, refs TryGhost/GQL#16

- removed GQL dependency
- replaced GQL with our brand new NQL implementation
- fixed all known filter limitations
- GQL suffered from some underlying filter bugs, which NQL tried to fix
- the bugs were mostly in how we query the database for relation filtering
- the underlying problem was caused by a too simple implementation of querying the relations
- mongo-knex has implemented a more robust and complex filtering mechanism for relations
- replaced logic in our bookshelf filter plugin
- we pass the custom, default and override filters from Ghost to NQL, which then are getting parsed and merged into a mongo JSON object. The mongo JSON is getting attached by mongo-knex.

NQL: https://github.com/NexesJS/NQL
mongo-knex: https://github.com/NexesJS/mongo-knex
@kirrg001
Copy link
Contributor

This bug was fixed in Ghost 2.8.0.

daniellockyer pushed a commit to TryGhost/framework that referenced this issue Jun 15, 2021
refs #10105, closes #10108, closes TryGhost/Ghost#9950, refs TryGhost/Ghost#9923, refs TryGhost/Ghost#9916, refs TryGhost/Ghost#9574, refs TryGhost/Ghost#6345, refs TryGhost/Ghost#6309, refs TryGhost/Ghost#6158, refs TryGhost/GQL#16

- removed GQL dependency
- replaced GQL with our brand new NQL implementation
- fixed all known filter limitations
- GQL suffered from some underlying filter bugs, which NQL tried to fix
- the bugs were mostly in how we query the database for relation filtering
- the underlying problem was caused by a too simple implementation of querying the relations
- mongo-knex has implemented a more robust and complex filtering mechanism for relations
- replaced logic in our bookshelf filter plugin
- we pass the custom, default and override filters from Ghost to NQL, which then are getting parsed and merged into a mongo JSON object. The mongo JSON is getting attached by mongo-knex.

NQL: https://github.com/NexesJS/NQL
mongo-knex: https://github.com/NexesJS/mongo-knex
daniellockyer pushed a commit to TryGhost/framework that referenced this issue Jun 15, 2021
refs #10105, closes #10108, closes TryGhost/Ghost#9950, refs TryGhost/Ghost#9923, refs TryGhost/Ghost#9916, refs TryGhost/Ghost#9574, refs TryGhost/Ghost#6345, refs TryGhost/Ghost#6309, refs TryGhost/Ghost#6158, refs TryGhost/GQL#16

- removed GQL dependency
- replaced GQL with our brand new NQL implementation
- fixed all known filter limitations
- GQL suffered from some underlying filter bugs, which NQL tried to fix
- the bugs were mostly in how we query the database for relation filtering
- the underlying problem was caused by a too simple implementation of querying the relations
- mongo-knex has implemented a more robust and complex filtering mechanism for relations
- replaced logic in our bookshelf filter plugin
- we pass the custom, default and override filters from Ghost to NQL, which then are getting parsed and merged into a mongo JSON object. The mongo JSON is getting attached by mongo-knex.

NQL: https://github.com/NexesJS/NQL
mongo-knex: https://github.com/NexesJS/mongo-knex
daniellockyer pushed a commit to TryGhost/framework that referenced this issue Jun 15, 2021
refs #10105, closes #10108, closes TryGhost/Ghost#9950, refs TryGhost/Ghost#9923, refs TryGhost/Ghost#9916, refs TryGhost/Ghost#9574, refs TryGhost/Ghost#6345, refs TryGhost/Ghost#6309, refs TryGhost/Ghost#6158, refs TryGhost/GQL#16

- removed GQL dependency
- replaced GQL with our brand new NQL implementation
- fixed all known filter limitations
- GQL suffered from some underlying filter bugs, which NQL tried to fix
- the bugs were mostly in how we query the database for relation filtering
- the underlying problem was caused by a too simple implementation of querying the relations
- mongo-knex has implemented a more robust and complex filtering mechanism for relations
- replaced logic in our bookshelf filter plugin
- we pass the custom, default and override filters from Ghost to NQL, which then are getting parsed and merged into a mongo JSON object. The mongo JSON is getting attached by mongo-knex.

NQL: https://github.com/NexesJS/NQL
mongo-knex: https://github.com/NexesJS/mongo-knex
daniellockyer pushed a commit to TryGhost/framework that referenced this issue Jun 15, 2021
refs #10105, closes #10108, closes TryGhost/Ghost#9950, refs TryGhost/Ghost#9923, refs TryGhost/Ghost#9916, refs TryGhost/Ghost#9574, refs TryGhost/Ghost#6345, refs TryGhost/Ghost#6309, refs TryGhost/Ghost#6158, refs TryGhost/GQL#16

- removed GQL dependency
- replaced GQL with our brand new NQL implementation
- fixed all known filter limitations
- GQL suffered from some underlying filter bugs, which NQL tried to fix
- the bugs were mostly in how we query the database for relation filtering
- the underlying problem was caused by a too simple implementation of querying the relations
- mongo-knex has implemented a more robust and complex filtering mechanism for relations
- replaced logic in our bookshelf filter plugin
- we pass the custom, default and override filters from Ghost to NQL, which then are getting parsed and merged into a mongo JSON object. The mongo JSON is getting attached by mongo-knex.

NQL: https://github.com/NexesJS/NQL
mongo-knex: https://github.com/NexesJS/mongo-knex
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug [triage] something behaving unexpectedly server / core Issues relating to the server or core of Ghost
Projects
None yet
Development

No branches or pull requests

2 participants