Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exclude statuses from index that is not searchable by local user #12038

Closed
wants to merge 1 commit into from
Closed

Exclude statuses from index that is not searchable by local user #12038

wants to merge 1 commit into from

Conversation

tribela
Copy link
Contributor

@tribela tribela commented Oct 1, 2019

Currently, searchable_by field contains all related accounts that contains remote user.
But remote user cannot perform a search on local instance.

This PR makes that ES index only contain statuses that searchable by at least one local user.
This slightly increase DB query at insert a status, But decrease ES volume massively(because most of remote status doesn't containing a mention to local user)

@tribela tribela changed the title Index statuses that only searchable by local user Exclude status from index that is not searchable by local user Oct 1, 2019
@tribela tribela changed the title Exclude status from index that is not searchable by local user Exclude statuses from index that is not searchable by local user Oct 1, 2019
@@ -141,7 +141,7 @@ def searchable_by(preloaded = nil)
ids += preloaded.reblogs[id] || []
end

ids.uniq
Account.where('id in (?)', ids.uniq).filter(&:local?).map(&:id)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an n+1 query which is why the indexing becomes slower. It also loads all account objects into memory before filtering them with Ruby instead of only selecting local accounts with SQL.

Instead, you can change the crutch definitions in chewy/statuses_index.rb to filter for local accounts in the first place.

For example:

data = ::Mention.where(status_id: collection.map(&:id)).where(account: Account.local).pluck(:status_id, :account_id)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants