Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Search limit #418

Merged
merged 3 commits into from
Jun 30, 2023
Merged

Conversation

fabricionaweb
Copy link
Contributor

@fabricionaweb fabricionaweb commented May 15, 2023

The idea here is to limit the number of queries to be made between each running.

The current feature excludeRecentSearch can already filter off some results, so combined with a limit feature we can have more flexibility.

Example of my usage:

  • My library has 440 items
  • excludeRecentSearch set to 5 days
  • searchLimit set to 100
  • searchCadence set to 1 day

Every day the search runs, it picks only the 100 that have not be done before

The user needs to pay attemption of their values or this can keep items out of the search, for example, if the excludeRecentSearch is too short for the limit, it will be start to pick items again...

fixes #432

@mmgoodnow
Copy link
Collaborator

mmgoodnow commented May 18, 2023

i actually removed a limit flag when I added periodic searches, because it felt like it didn't really make sense. However I do see this use case and have wished for something similar.

what if we had a way for cross-seed to automatically balance?

example:

searchCadence 1 day, excludeRecentSearch is 6 months (~180 days), we're running for the first time. 1000 torrents are eligible to run.

therefore, we do one 180th of the torrents, plus some number for good measure.

one problem with this approach would be that as we progress through the 6 months, less and less torrents will be eligible to run. when there are 89 torrents left, a 180th of the eligible torrents would round to 0. so the first time, we'd do like 5, and towards the end we'd be doing less. So it's not fully balanced.

Another approach: we count the total number of torrents, ignoring excludeRecentSearch, then divide that number by 180. Then, we run that number of torrents today (rounding up). this would probably be the most balanced? and it does seem feasible…

@yammes08
Copy link

i actually removed a limit flag when I added periodic searches, because it felt like it didn't really make sense. However I do see this use case and have wished for something similar.

what if we had a way for cross-seed to automatically balance?

example:

searchCadence 1 day, excludeRecentSearch is 6 months (~180 days), we're running for the first time. 1000 torrents are eligible to run.

therefore, we do one 180th of the torrents, plus some number for good measure.

one problem with this approach would be that as we progress through the 6 months, less and less torrents will be eligible to run. when there are 89 torrents left, a 180th of the eligible torrents would round to 0. so the first time, we'd do like 5, and towards the end we'd be doing less. So it's not fully balanced.

Another approach: we count the total number of torrents, ignoring excludeRecentSearch, then divide that number by 180. Then, we run that number of torrents today (rounding up). this would probably be the most balanced? and it does seem feasible…

I was chatting to fab about this - that last idea sounds like it would work. So, you could just have an enableSearchLimit flag, true or false, and if true it does the calculation you describe?

@mmgoodnow
Copy link
Collaborator

So, you could just have an enableSearchLimit flag, true or false, and if true it does the calculation you describe?

yeah i think so.

mmgoodnow
mmgoodnow previously approved these changes Jun 30, 2023
src/cmd.ts Outdated Show resolved Hide resolved
src/config.template.docker.cjs Outdated Show resolved Hide resolved
src/config.template.cjs Outdated Show resolved Hide resolved
@mmgoodnow mmgoodnow merged commit 2544077 into cross-seed:master Jun 30, 2023
1 check failed
@mmgoodnow
Copy link
Collaborator

Thank you for the contribution!

@fabricionaweb fabricionaweb deleted the search-limit branch December 10, 2023 23:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

excludeRecentSearch API usage is too bursty
3 participants