Avoid hitting the rate limit. Don't let it fail because of it.

[muisje]

Hey,

I tried this out with Google's free AI API, but i hit the rate limit and it failed to translated. I was using Gemini 3.1 Flash Lite which has a RMP of 15. Which it hit after 16 sentences. And then it failed because it got the message of the rate limit.

Lingarr.Server.Exceptions.TranslationException: Translation failed for subtitle line
 ---> Lingarr.Server.Exceptions.TranslationException: Retry limit reached after TooManyRequests.
 ---> System.Net.Http.HttpRequestException: Gemini API request failed with status TooManyRequests: {
  "error": {
    "code": 429,
    "message": "You exceeded your current quota, please check your plan and billing details. For more information on this error, head to: https://ai.google.dev/gemini-api/docs/rate-limits. To monitor your current usage, head to: https://ai.dev/rate-limit. \n* Quota exceeded for metric: generativelanguage.googleapis.com/generate_content_free_tier_requests, limit: 15, model: gemini-3.1-flash-lite\nPlease retry in 165.781679ms.",
    "status": "RESOURCE_EXHAUSTED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.Help",
        "links": [
          {
            "description": "Learn more about Gemini API quotas",
            "url": "https://ai.google.dev/gemini-api/docs/rate-limits"
          }
        ]
      },
      {
        "@type": "type.googleapis.com/google.rpc.QuotaFailure",
        "violations": [
          {
            "quotaMetric": "generativelanguage.googleapis.com/generate_content_free_tier_requests",
            "quotaId": "GenerateRequestsPerMinutePerProjectPerModel-FreeTier",
            "quotaDimensions": {
              "location": "global",
              "model": "gemini-3.1-flash-lite"
            },
            "quotaValue": "15"
          }
        ]
      },
      {
        "@type": "type.googleapis.com/google.rpc.RetryInfo",
        "retryDelay": "0s"
      }
    ]
  }
}

   at Lingarr.Server.Services.Translation.GoogleGeminiService.TranslateWithGeminiApi(String message, CancellationToken cancellationToken) in /src/Lingarr.Server/Services/Translation/GoogleGeminiService.cs:line 199
   at Lingarr.Server.Services.Translation.GoogleGeminiService.TranslateAsync(String text, String sourceLanguage, String targetLanguage, List`1 contextLinesBefore, List`1 contextLinesAfter, CancellationToken cancellationToken) in /src/Lingarr.Server/Services/Translation/GoogleGeminiService.cs:line 141
   --- End of inner exception stack trace ---
   at Lingarr.Server.Services.Translation.GoogleGeminiService.TranslateAsync(String text, String sourceLanguage, String targetLanguage, List`1 contextLinesBefore, List`1 contextLinesAfter, CancellationToken cancellationToken) in /src/Lingarr.Server/Services/Translation/GoogleGeminiService.cs:line 148
   at Lingarr.Server.Services.SubtitleTranslationService.TranslateSubtitleLine(TranslateAbleSubtitleLine translateAbleSubtitle, CancellationToken cancellationToken) in /src/Lingarr.Server/Services/SubtitleTranslationService.cs:line 110
   --- End of inner exception stack trace ---
   at Lingarr.Server.Services.SubtitleTranslationService.TranslateSubtitleLine(TranslateAbleSubtitleLine translateAbleSubtitle, CancellationToken cancellationToken) in /src/Lingarr.Server/Services/SubtitleTranslationService.cs:line 125
   at Lingarr.Server.Services.SubtitleTranslationService.TranslateSubtitles(List`1 subtitles, TranslationRequest translationRequest, Boolean stripSubtitleFormatting, Int32 contextBefore, Int32 contextAfter, CancellationToken cancellationToken) in /src/Lingarr.Server/Services/SubtitleTranslationService.cs:line 73
   at Lingarr.Server.Jobs.TranslationJob.Execute(TranslationRequest translationRequest, CancellationToken cancellationToken) in /src/Lingarr.Server/Jobs/TranslationJob.cs:line 197

Mutliple things can be done to succesfully translate with this.

First thing is to continue at the suggested time 'Please retry in 165.781679ms.'.

Secondly is by trying to avoid hitting the limit by delaying requests per minute.

Third is to try to translate more than one line per request. It hit the rate limit, but the 586 / 250K TPM wasn't hit at all

And besides that if it failed it could be continued at a later moment in time continuing where it left of.

This isn't the fastest way to translate a subtitle, but it would make sure it gets translated

[muisje]

Ah i didn't look well enough, saw some options in the settings for what i suggested, but this could be automated by looking at the response. And it still shouldn't fail because of hitting the rate limit.

[muisje]

Well it's not exactly what i suggested.

What's in the config is delay to retry after hitting the rate limit, after that it will hit the rate limit again. This could be handled in a smarter way to avoid hitting it in a consistent matter, not by exponentially scaling the delay after hitting a rate limit

[rowanfuchs]

Currently translation batching is a feature that covers this issue and tweaking the retry settings should be able to cover this. I am not yet feeling a specific service tweak to circumvent these settings yet.