Bazarr downloads and overwrites existing/manually downloaded subtitles regardless of setting

[csjoh]

Describe the bug
Bazarr upgrades existing/manually downloaded subtitles even if the setting to do this is unchecked (disabled, i.e. do NOT upgrade existing subtitles).

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Settings' then 'Subtitles'
2. Click on 'Upgrade Previously Downloaded Subtitles', which will upgrade subtitles previously downloaded by Bazarr.
3. Keep the checkbox for 'Upgrade Manually Downloaded or Translated Subtitles' unchecked, which in my mind should mean "Leave existing subtitle files for which there is no download record, alone"
4. Click 'Save'.
5. Wait for Bazarr to download a subtitle file and upgrade an existing/manually downloaded/imported subtitle.

Expected behavior
Subtitles imported into Radarr/Sonarr as part of the release imported, should be treated as "manually downloaded" and be left alone by Bazarr. This does not happen. Disabling the functionality to upgrade subtitles that Bazarr has already downloaded seems to be the only fix for this so far.

Software:

• Bazarr Version: 1.0.2
• Sonarr Version: 3.0.6.1342
• Radarr Version: 4.0.4.5922
• Operating System: Linux-5.10.28-Unraid-x86_64-with
• Python Version: 3.9.7

[morpheus65535]

Subtitles imported by Sonarr/Radarr aren't considered by Bazarr in the upgrade process. In fact, they are like any other subtitles present on disk: if Bazarr can index them and match a language, it will be considered during the missing subtitles calculation. Bazarr will never try to replace subtitles that haven't been downloaded or uploaded through the UI or automatic download functionnalities.

[csjoh]

Yet still they are upgraded. Just today a movie was downloaded that included various .srt files for Norwegian, English and Polish. Radarr imported the .srt files, and less than half an hour later, I was notified that the English subtitles were upgraded by Bazarr.

[morpheus65535]

Bazarr CAN'T do that. It's simply impossible. Bazarr rely on the history table to know what can or can't be upgraded. This table doesn't include subtitles "found" on disk.

If you look at history, you'll see that this is a normal download and not an upgrade. Why did it download a English subtitles? Well probably because your Sonrr/Radarr is improperly configured to import subtitles. Maybe because they are imported but improperly named.

[csjoh]

Yeah, but it did.

The movie downloaded was The House (2022), imported to Radarr at 10:02:

/media/Film/The House (2022)/The.House.2022.1080p.WEBRip.x265-RARBG.mp4 along with three subtitle files for Norwegian, English and Polish:
/media/Film/The House (2022)/The.House.2022.1080p.WEBRip.x265-RARBG.en.srt
/media/Film/The House (2022)/The.House.2022.1080p.WEBRip.x265-RARBG.no.srt
/media/Film/The House (2022)/The.House.2022.1080p.WEBRip.x265-RARBG.pl.srt

They were all renamed correctly by Radarr:
The House (2022) WEBRip-1080p.mp4
The House (2022) WEBRip-1080p.en.srt
The House (2022) WEBRip-1080p.no.srt
The House (2022) WEBRip-1080p.pl.srt

File system listing:

# ls -la
total 1594084
drwxrwxrwx 1 root   root           6 Feb  7 10:30 ./
drwxrwxrwx 1 nobody users      57344 Feb  2 13:56 ../
-rw-rw-r-- 1 nobody users      61236 Feb  7 10:30 The\ House\ (2022)\ WEBRip-1080p.en.srt
-rw-rw-r-- 1 nobody users 1632094570 Jan 21 10:31 The\ House\ (2022)\ WEBRip-1080p.mp4
-rw-rw-r-- 1 nobody users      56150 Feb  7 10:02 The\ House\ (2022)\ WEBRip-1080p.no.srt
-rw-rw-r-- 1 nobody users      52144 Feb  7 10:02 The\ House\ (2022)\ WEBRip-1080p.pl.srt

As you can see, the Norwegian and Polish subtitle files have the timestamp set to when the files were imported. The English subtitle file has its modification date/time set to 10:30, when Bazarr downloaded the updated/upgraded file.

The movie file has the release date set as its modification date, which trips me up sometimes but is generally a nice thing for reports and things like that.

Radarr's import settings:

Bazarr's Subtitle options:

My Telegram notification alerting me that Bazarr downloaded new subtitles:
Bazarr notification (10:30)
The House (2022) : English subtitles downloaded from supersubtitles with a score of 95.83%.

I don't want to sound like I'm picking a fight, but there is something weird going on. Maybe there's a bug, maybe I've done something wrong in my configuration. Either way I'd like to figure this out, and ideally also let others who may experience the same thing find a solution.

[morpheus65535]

As I said earlier, Bazarr cannot upgrade a subtitles that it doesn't have download itself (or that you've uploaded through the UI).

The only possible cause of your issue that I can think of is that Bazarr got notified of mp4 import and indexed it before Radarr imported the subtitles file. So Bazarr think there's no subtitles file alongside the movie and download an English subtitles accordingly to what's requested in the languages profile assigned to this movie.

However, I highly doubt that Radarr doing a mv on a subtitles file would take more than some milliseconds so it's very unlikely to happen.

[morpheus65535]

By the way, here's the query to determine what subtitles could be upgraded: https://github.com/morpheus65535/bazarr/blob/master/bazarr/get_subtitle.py#L1438-L1454

Also, subtitles not downloaded or uploaded by Bazarr aren't in that table but only indexed in table_movies subtitles column.

[csjoh]

As I said earlier, Bazarr cannot upgrade a subtitles that it doesn't have download itself (or that you've uploaded through the UI).

The only possible cause of your issue that I can think of is that Bazarr got notified of mp4 import and indexed it before Radarr imported the subtitles file. So Bazarr think there's no subtitles file alongside the movie and download an English subtitles accordingly to what's requested in the languages profile assigned to this movie.

However, I highly doubt that Radarr doing a mv on a subtitles file would take more than some milliseconds so it's very unlikely to happen.

Yeah, that seems unlikely, but this is not the first time this has happened though. Most downloads include subs in separate files and I try to use the subs that accompany the releases as they are usually synced properly and (mostly) from the original source. There have been quite a few instances where Bazarr has downloaded "upgraded" subtitle files and overwritten existing ones.

However, you seem insistent that this cannot happen and I say it does happen, so we'll just agree to disagree.

I'll disable the upgrade subs feature completely and hopefully avoid further incidents that way.

Thanks for your time on this issue and thank you for developing Bazarr, I highly appreciate it.

[morpheus65535]

TBH, I highly doubt that disabling "upgrade subtitles" will fix this issue. Please update this issue to confirm that. If the issue is still present, we could temporarely add a sleep in the right place to give Radarr some time to move the subtitles file. I agree with you that if you have access to subtitles files include with the movie, they shouldn't get overwrited. I'll be happy to provide a fix if we can make sure that's the cause of the issue.

[csjoh]

Disabling "upgrade subtitles" did solve the issue, it seems. Another release was imported with accompanying subtitles, and they were not overwritten.

[morpheus65535]

One success could be circumstantial. Once you'll have 5 succesful import, re-enable upgrade subtitles and wait for 5 unsuccesful import (ie: subtitles overwriten). Then we'll know there's a high probability of this settings to be the cause of your issue (although it's almost impossible, like I explained before).

[csjoh]

I now have had 5 successful imports where no imported subtitles have been overwritten. I'll re-enable upgrade subtitles and report back once I've had 5 new imports. Stay tuned! :)

[morpheus65535]

I can't wait! ;-)

[csjoh]

Yeah, bad news. Bazarr overwrote the subs for a few recently downloaded titles, so there seems to be something wrong. The checkbox does in fact do the oppposite of what the label says it should do.

It might just be my system, but I kind of doubt that. I'll just leave the setting off for the time being.

[morpheus65535]

Can you show me a screenshot of your history please? I don't care about the series name but only about the icon to the left of the suspicious entries.

[csjoh]

Sure:

[morpheus65535]

I guess the problematic download are those Polish subtitles? If it's the case, they aren't upgrade but normal download. Recycle icon means upgrade.

[csjoh]

No, usually only English subs are included with the release, and the Norwegian and Polish subs are downloaded by Bazarr, and that usually works really well.

The second title is listed 5 times (as 2, 3, 5, 6 and 7), and the English subtitles that Bazarr claims was downloaded 6 days ago were in fact imported with the release, and then upgraded yesterday even though Bazarr didn't download the file in the first place.. Yesterday's upgrade is thus incorrect as Bazarr didn't have any history for that file.

Now that I think of it, the first title in that list had an English subtitle file included with the release (it's a Norwegian-language film), and that was also downloaded for the first time (as far as Bazarr is aware) 3 hours ago while it was in fact imported with the release by Radarr on January 18th 2022.

[morpheus65535]

If it's in the history as downloaded, it's because it's been downloaded. Put your mouse over the "i" circled icon and you'll see from what provider. Bazarr does not "claims" to have downloaded subtitles it doesn't. This subtitles was download and had a score of 94.17%. Why it's been downloaded? I would need a debug log to confirm it but, for sure, it was considered as missing.

[csjoh]

In that case, Bazarr needs to look a little harder for existing subtitle files. The movie was imported as "Movie file (year) WEBDL-1080p.mp4" and the English sub file as "Movie name (year) WEBDL-1080p.en.srt" as specified in Radarr's settings. I believe Bazarr looks for subtitle files named the same as the movie file?

I can enable debugging in Bazarr and Radarr and then have Radarr download something and then we'll have a look at what Bazarr does afterwards.

[morpheus65535]

You're right, if the filename match (excluding langue and extension), Bazarr will match it to the movie and it will get indexed.

Of course, having an uncensored debug log could help. You can send it to me through PM on Discord if you want or just drop it here.

[morpheus65535]

No news, good news.