Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add `tootctl preview_cards remove` #11320

Merged
merged 5 commits into from Jul 28, 2019

Conversation

@mayaeh
Copy link
Collaborator

commented Jul 15, 2019

This changes will be change the remove thumbnails of previews from automatic execution by the Sidekiq scheduler to manual execution by the tootctl command.

After #11304, object storage’s rate-limit is triggered when Scheduler::PreviewCardsCleanupScheduler is executed in some object storage environments.

スクリーンショット 2019-07-15 22 43 31

スクリーンショット_2019-07-15_22_46_12

This is similar to #8187.

So I made tootctl preview_cards remove command referring to tootctl media remove.

There are multiple if statements.
I think that it is not a good writing style.

Please let me know if there is a better way of writing.


TODO

  • Remove unused file
  • Fix if/elsif blocks
  • Add output to confirm when the number of days is less than 2 weeks

@mayaeh mayaeh changed the title Add `tootctl preview-cards remove` Add `tootctl preview_cards remove` Jul 15, 2019

@mayaeh mayaeh force-pushed the mayaeh:feature_tootctl_preview_cards branch from 3997dc3 to a4d4dd5 Jul 15, 2019

lib/mastodon/preview_cards_cli.rb Outdated Show resolved Hide resolved
config/sidekiq.yml Show resolved Hide resolved
@Gargron

This comment has been minimized.

Copy link
Member

commented Jul 15, 2019

After #11304, object storage’s rate-limit is triggered when Scheduler::PreviewCardsCleanupScheduler is executed in some object storage environments.

It might be possible to add a rate-limit system to Sidekiq jobs. While Scheduler::PreviewCardsCleanupScheduler is an obvious culprit here, at certain number of Sidekiq threads/workers and certain workloads (a lot of incoming posts with media?) the object-storage rate-limit could be triggered by other jobs as well. Something to think about.

@mayaeh mayaeh changed the title Add `tootctl preview_cards remove` [WIP] Add `tootctl preview_cards remove` Jul 16, 2019

mayaeh added some commits Jul 16, 2019

fix code style again
Add exclude case where image_file_name is blank
@mayaeh

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 16, 2019

While Scheduler::PreviewCardsCleanupScheduler is an obvious culprit here, at certain number of Sidekiq threads/workers and certain workloads (a lot of incoming posts with media?) the object-storage rate-limit could be triggered by other jobs as well.

I think that it isn't happening on my server so far.
But, as you say, I think it can happen.

@mayaeh mayaeh changed the title [WIP] Add `tootctl preview_cards remove` Add `tootctl preview_cards remove` Jul 16, 2019


if time_ago > 2.weeks.ago
prompt.say "\n"
prompt.say('The preview cards less than the past two weeks will not be re-acquired even when needed.')

This comment has been minimized.

Copy link
@mayaeh

mayaeh Jul 16, 2019

Author Collaborator

Is this text easy to understand? I do not have any confidence.

This comment has been minimized.

Copy link
@Gargron

Gargron Jul 17, 2019

Member

Media remove does not ask for confirmation. Do you think it's necessary here?

This comment has been minimized.

Copy link
@mayaeh

mayaeh Jul 18, 2019

Author Collaborator

Yes, I think it is necessary.

In preview cards, I think that only those older than 2 weeks will be re-acquired.
https://github.com/tootsuite/mastodon/blob/master/app/services/fetch_link_card_service.rb#L25

On the other hand, I can not find the date limit for re-acquisition of attached media.

Therefore, I think that it is necessary to output a warning if the number of days less than 2 weeks is specified in this command.

@mayaeh mayaeh requested a review from Gargron Jul 17, 2019

@Gargron Gargron merged commit 0d80f68 into tootsuite:master Jul 28, 2019

2 checks passed

build-and-test Workflow: build-and-test
Details
codeclimate All good!
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.