Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use separate workers to process imports, retry failures #5207

Merged
merged 1 commit into from
Oct 3, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions app/workers/import/relationship_worker.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# frozen_string_literal: true

class Import::RelationshipWorker
include Sidekiq::Worker

sidekiq_options queue: 'pull', retry: 8, dead: false

def perform(account_id, target_account_uri, relationship)
from_account = Account.find(account_id)
target_account = ResolveRemoteAccountService.new.call(target_account_uri)

return if target_account.nil?

case relationship
when 'follow'
FollowService.new.call(from_account, target_account.acct)
when 'block'
BlockService.new.call(from_account, target_account)
when 'mute'
MuteService.new.call(from_account, target_account)
end
rescue ActiveRecord::RecordNotFound
true
end
end
56 changes: 12 additions & 44 deletions app/workers/import_worker.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,63 +12,31 @@ class ImportWorker
def perform(import_id)
@import = Import.find(import_id)

case @import.type
when 'blocking'
process_blocks
when 'following'
process_follows
when 'muting'
process_mutes
Import::RelationshipWorker.push_bulk(import_rows) do |row|
[@import.account_id, row.first, relationship_type]
end

@import.destroy
end

private

def from_account
@import.account
end

def import_contents
Paperclip.io_adapters.for(@import.data).read
end

def import_rows
CSV.new(import_contents).reject(&:blank?)
end

def process_mutes
import_rows.each do |row|
begin
target_account = ResolveRemoteAccountService.new.call(row.first)
next if target_account.nil?
MuteService.new.call(from_account, target_account)
rescue Mastodon::UnexpectedResponseError, HTTP::Error, OpenSSL::SSL::SSLError
next
end
end
end

def process_blocks
import_rows.each do |row|
begin
target_account = ResolveRemoteAccountService.new.call(row.first)
next if target_account.nil?
BlockService.new.call(from_account, target_account)
rescue Mastodon::UnexpectedResponseError, HTTP::Error, OpenSSL::SSL::SSLError
next
end
def relationship_type
case @import.type
when 'following'
'follow'
when 'blocking'
'block'
when 'muting'
'mute'
end
end

def process_follows
import_rows.each do |row|
begin
FollowService.new.call(from_account, row.first)
rescue Mastodon::NotPermittedError, ActiveRecord::RecordNotFound, Mastodon::UnexpectedResponseError, HTTP::Error, OpenSSL::SSL::SSLError
next
end
end
def import_rows
CSV.new(import_contents).reject(&:blank?)
end
end