Marked as Missing even when the folder is correct

[Ltek]

LOTS of problem with Bulk import and manually trying to fix what Radarr marks as 'Missing'

1. Marks as missing even when folder is correct.
2. Marks as missing and doesnt correct even when you MANUALLY set the folder on the movie.
3. When the folder is not exactly as Radarr think it should be,Radrarr cannot find it, marks as missing
   ... example: user doesnt have 'The' on the folder, Radarr cant figure that out.

To Reproduce
see above

Expected behavior
Should find movies when they exist.
Should not be marked as missing when you point the folder manually!
Should not expect THE at the front of movies
Should not get confused if THE on the front doesnt exist

Screenshots
included 2:
Charlotte's Web... one of MANY movies that exist, the fold has been set manually and Radarr still shows as missing
The Incredibles ... I left the THE on the folder to show you even when Radarr sees what it is looking for it still doesnt work

Desktop (please complete the following information):
Synology NAS
Version
0.2.0.1120
Mono Version
5.8.0.108 (tarball Sat Feb 17 22:03:12 UTC 2018)

Debug Logs
Debug log includes me manually changing/setting the path and also a full 'rescan' being ran


radarr.debug.txt

[Ltek]

This in log when I did a rescan of a single movie...
Error
EventAggregatorDiskScanService failed while processing [MovieUpdatedEvent]: Source and destination can't be the same /volume1/kids movies/WALL·E (2008)/8:06pmRenameMovieFileService[tt0910970][WALL·E (2008)]'s movie folder changed to: /volume1/kids movies/WALL·E (2008)8:06pmRefreshMovieServiceUpdating Info for WALL·E

... why would it try to process the movie in some way? Should simply add it!

and this...

	EventAggregator	DiskScanService failed while processing [MovieUpdatedEvent]: Could not find a part of the path '/volume1/movies/Avengers, The (2012)'.	8:11pm
	RenameMovieFileService	[tt0848228][The Avengers (2012)]'s movie folder changed to: /volume1/movies/The Avengers (2012)	8:11pm
	RefreshMovieService	Updating Info for The Avengers

... HUH? Radarr CHANGED the folder (added 'The' - wtf!) then tells us it cant find it - DUH! Why did it change the folder? I didnt ask it to. So, still shows as Missing UNTIL I re-ran the manual scan then it finds it
= not a smart system. Sorry I'm frustrated - it is simply not working properly and I've been trying to figure this out for months.

[fryfrog]

For movies that you've corrected the path to what is literally right, do a refresh and see if it shows up. If it doesn't, turn logging up to debug or trace and see what you see. These issues are typically caused by file formats Radarr doesn't know about (like .iso or ?BVDM? folders) or ownership/permissions issues.

I haven't used Bulk Import, but my understanding is that what ever path it finds during import is what is used. Do you maybe have static paths disabled, and so it tries to "correct" them?

[fryfrog]

Also, you should use {Movie TitleThe} in your folder/file naming pattern if that is what you're after.

If you're literally dropping the word "The" from the name entirely... well, you're wrong and it's going to cause you problems all over place.

[support]

We use the issue tracker exclusively for bug reports and feature requests. However, this issue appears to be a support request. Please hop over onto our Discord or Subreddit

[fryfrog]

To be sure, parts of this are kind of a bug report and kind of a feature request. But I think until we get your fixable problems squared away, you won't be able to refine this into a useful bug report or feature request. :)

[Ltek]

This IS a bug report. Is that a real bot or just a dev not wanting to fix this issue? I have tried to set MULTIPLE settings and manually point Radarr to the correct path and it still doesnt work.

I never had this problem with Couch...worked perfectly with the same exact structure!

all my movies are MP4 or MKV - plus if they have a 'the' it is in the format <name, the>
... so it should work fine if Radarr is working proper, which is not

[fryfrog]

That is me deciding that what you need is some help figuring out what is wrong. Most of what you describe as not working does work, so we need to figure out what is going on in your setup.

Do you want to work toward that or leave a bug report open that doesn't have enough useful details to ever be solved?

Its okay to keep using CP, it has a handful of features that aren't in Radarr and likely won't be for a long time. Not the problems you're having, but stuff like custom folder/file paths and automated adding of new movies and probably a few more things I don't remember.

[galli-leo]

Please also post debug logs of when you tried your bulk import as well as disk scan.

Additionally, it seems like you have automatic folder rename enabled. I strongly advise to turn that feature off.

[Ltek]

Did you look at the log I posted? Its the debug log -- from when I did a full disk scan, bulk import (which Radarr says it find nothing new) and multiple, manual changes to movies pointing them to the folder (which Radarr still shows as missing even though the path is correct)

I spent 2 hours doing all of this, PLEASE read the log I attached

[fryfrog]

I have no idea how I just scrolled past w/o seeing the log, sorry!

[v0.2.0.1120] System.IO.DirectoryNotFoundException: Could not find a part of the path '/volume1/kids movies/WALL E (2008)'.
  at System.IO.__Error.WinIOError (System.Int32 errorCode, System.String maybeFullPath) [0x00207] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/__error.cs:188 
  at System.IO.FileSystemEnumerableIterator`1[TSource].HandleError (System.Int32 hr, System.String path) [0x00006] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/filesystemenumerable.cs:564 
  at System.IO.FileSystemEnumerableIterator`1[TSource].CommonInit () [0x00054] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/filesystemenumerable.cs:296 
  at System.IO.FileSystemEnumerableIterator`1[TSource]..ctor (System.String path, System.String originalUserPath, System.String searchPattern, System.IO.SearchOption searchOption, System.IO.SearchResultHandler`1[TSource] resultHandler, System.Boolean checkHost) [0x000d6] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/filesystemenumerable.cs:264 
  at System.IO.FileSystemEnumerableFactory.CreateFileNameIterator (System.String path, System.String originalUserPath, System.String searchPattern, System.Boolean includeFiles, System.Boolean includeDirs, System.IO.SearchOption searchOption, System.Boolean checkHost) [0x00009] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/filesystemenumerable.cs:48 
  at System.IO.Directory.InternalGetFileDirectoryNames (System.String path, System.String userPathOriginal, System.String searchPattern, System.Boolean includeFiles, System.Boolean includeDirs, System.IO.SearchOption searchOption, System.Boolean checkHost) [0x00000] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/directory.cs:834 
  at System.IO.Directory.InternalGetFiles (System.String path, System.String searchPattern, System.IO.SearchOption searchOption) [0x00000] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/directory.cs:651 
  at System.IO.Directory.GetFiles (System.String path, System.String searchPattern, System.IO.SearchOption searchOption) [0x00038] in /spksrc/native/mono/work-native/mono-5.8.0.108/mcs/class/referencesource/mscorlib/system/io/directory.cs:638 
  at NzbDrone.Common.Disk.DiskProviderBase.GetFiles (System.String path, System.IO.SearchOption searchOption) [0x00048] in C:\projects\radarr-usby1\src\NzbDrone.Common\Disk\DiskProviderBase.cs:177 
  at NzbDrone.Core.MediaFiles.RenameMovieFileService.RenameMoviePath (NzbDrone.Core.Movies.Movie movie, System.Boolean shouldRenameFiles) [0x0008a] in C:\projects\radarr-usby1\src\NzbDrone.Core\MediaFiles\RenameMovieFileService.cs:167 
  at NzbDrone.Core.MediaFiles.DiskScanService.Scan (NzbDrone.Core.Movies.Movie movie) [0x00000] in C:\projects\radarr-usby1\src\NzbDrone.Core\MediaFiles\DiskScanService.cs:75 
  at NzbDrone.Core.MediaFiles.DiskScanService.Handle (NzbDrone.Core.Movies.Events.MovieUpdatedEvent message) [0x00000] in C:\projects\radarr-usby1\src\NzbDrone.Core\MediaFiles\DiskScanService.cs:198 
  at NzbDrone.Core.Messaging.Events.EventAggregator.PublishEvent[TEvent] (TEvent event) [0x000c3] in C:\projects\radarr-usby1\src\NzbDrone.Core\Messaging\Events\EventAggregator.cs:56

^ For that one, can you remove the weird dot from the file name on disk and in that movie and see if it work?

[fryfrog]

Log in a pastebin, I found it easier to grok.

[fryfrog]

You've got a couple that are like DVD folder structures...

The Matrix (1999)/VTS_01_2.VOB

[fryfrog]

That is a long log, I made it about a third of the way through w/o seeing anything stand out besides what I mentioned above. Can you tell us some example movies that aren't working so I can look for them specifically?

Also, again... this is still currently support, not a bug :p

[fryfrog]

And give us some ls -alh output from a few of the folders for movies it isn't detecting.

[Ltek]

I cleared the log then did some work... manually set a few movie folders and then forced them to 'sync' individually. Some worked, some did not. None of them auto-detected as 'Downloaded' until I forced a rescan (not sure why this would be necessary? I just told it exactly where the movie was -- should it not automatically rescan itself?)

Look at the end of the log... "Big Hero 6"
The path was 100% correct from the start

1. multiple manual scans ='missing'
2. manually chose the same folder... scan = 'missing'
3. manually set the WRONG folder.
4. manually set the CORRECT (same as in step 1 above)... scan = 'downloaded'
   .... Nothing else changed. Nothing.
   radarr.debug.txt

[Ltek]

During this process I found another bug..

Wiped out my entire library in Radarr...
I decided to do a bulk edit of the 'Missing' movies... and delete them, then try a bulk import. I made sure to ONLY have the button to filter the 'Missing' selected and the count at the bottom showed 49. But it proceeded to delete ALL 200+ movies

I rebooted the NAS to give it a clean start, Radarr is now importing the movies but look like the same 49 'missing' movies are still not being found.

I checked one, and the path still looking for... /volume1/movies/The Avengers (2012)
when the path is actually... /volume1/movies/Avengers, The (2012)

[fryfrog]

There is something weird going on in your setup and figuring it out over an issue is tedious. Can you hop in Discord and do this in real time at some point?

It also sounds like you're using a Synology, does your support Docker? If it does, I'd suggest that route instead of the native package.

[Ltek]

I don't think it is my setup. I removed all the movies again... removed the movie paths, set it to use a Nightly and then restarted.

When I choose a path, Radarr selects it just fine... and even shows the folders being found in the log... but then fails to "see" any movies.

I confirmed a bug where Radarr pukes (stops, 100% fails) when it tries to parse a folder named #Recycle ...
I delete the folder and amazing, Radarr 'sees' all 174 movies.

Debug log attached

NzbDroneErrorPipeline	Request Failed: Access to the path '/volume1/movies/#recycle' is denied.

radarr.debug.txt

I'll open a new bug report for this one

UPDATE... I rescanned my main folder and ALL the movies scanned properly... I suspect the problem I was having is a direct affect of the bug with Radarr not being able to handle child folder names with # in them.

[fryfrog]

I'll be honest, it has got to be your setup. Everyone else uses this just fine. You're one person having issues. And past experience with others having problems says it is an issue in your setup.

Hop on Discord and do this real time. Trying to do this in issue comments is tediously frustrating.

[fryfrog]

18-9-28 16:55:01.0|Debug|X509CertificateValidationPolicy|Certificate validation for https://127.0.0.1:5001/webapi/DownloadStation/task.cgi?_sid=60Ulyc88cSSv6CBLBN02462&api=(removed)&version=1&method=list&additional=detail%2Ctransfer failed.

You're also trying to use https w/ your download client, which appears to be Download Station.

So do you see how it really seems likely to be a setup issue? :)

[Ltek]

"You're also trying to use https w/ your download client, which appears to be Download Station."
... what does DL Station have to do with Radarr Scanning existing movie folders? Pretty sure it doesnt affect that at all?

As I stated, the bug looks pretty clear to me... Radarr doesnt know how to handle folders with a # at the front (maybe anywhere in the name)... When it encounters it it fails out and stops scanning.

I verified this with 2 separate root folders. Once I removed the child folder with the # Radarr works perfectly... yes, perfectly -- all issues I was having no longer exist when those folders are removed.

Synology uses a hidden folder named "#recycle" as trash collection. This is built-into the OS. It can be turned off but then there is no way to undo a file deletion. Not a good option

the Fix is for Radarr to properly handle folders with a # in the name.

I can do this on Discord (which I'm not yet a member) but all you need to do is t create a child folder with a # at the front of the name... you should see the exact same problem. Should take you 2 minutes to repro - far less time than getting on Discord.

[fryfrog]

Radarr handles folders w/ # in their name just fine, look at issue #3063 where I showed you that isn't the problem. I mean, the problem is literally in the log file you posted. What it doesn't appear to handle well is complete lack of access to that folder. Radarr has a recycle bin too, so you're not totally screwed by not having it. It'd be worth figuring out if totally bailing on an access error is the right thing or not.

The Download Station incorrectly using https doesn't have anything to do w/ your problem, it just shows more issues with your setup.

I'm glad your issue is resolved, if you have the time to add something about #recycle folder to the Wiki, that would be great.

[Ltek]

You're saying when Radarr has perms to the Root folder but doesn't have permission to a Child folder it completely stops searching the rest of the Root folder? ... Either this is a bug or poor design. Radarr should simply skip folders it doesn't have permissions to and raise the error in the log - and move on to folders it can parse.

BTW, I checked DL Station perms and also ran "test" in Radarr on the connection... both are fine. No error during test. So, if Radarr is throwing error trying to connect we have another bug either it is sending perms incorrectly or the 'test' function isn't working (saying tested fine when it doesn't)
Of Note: When I disable SSL in Radarr the 'test' Fails. But succeeds when SSL is enabled ... this is all as it should work. SSL is required for all of my NAS connections.

[fryfrog]

You're saying when Radarr has perms to the Root folder but doesn't have permission to a Child folder it completely stops searching the rest of the Root folder? ... Either this is a bug or poor design. Radarr should simply skip folders it doesn't have permissions to and raise the error in the log - and move on to folders it can parse.

I think we've finally nailed it down to the actual bug. You should edit the 2nd issue you made, both the title and the details and we can re-open it and name it a bug. If you have the time, it'd be great to reproduce it in a variety of ways and see if there are any differences. Like, just a single movie w/ wrong permissions instead of a weirdly named/permissioned #recycle folder created by Synology.

Also, it'd be good to give a stat #recycle so we can see exactly what that folder was doing.

For your SSL issue, if it works... its fine and that is great. My guess is you're using a self signed certificate or if not, you've signed it incorrectly. If Radarr ever decides to enforce certificate validity, it'd stop working. What happens when you load that https url in your browser? Does it warn you that the certificate isn't valid?

If you turn SSL off, you'd of course need to point at the non SSL version of Download Station. That might be a different port in the config or http vs. https. I don't use DS, so I'm not sure. That is probably why your testing of it failed. Unless you've literally disabled / blocked it from running non-ssl on Synology itself?

[fryfrog]

Also, it'd be super interesting to see if you end up w/ the same problem in Sonarr!

[Ltek]

I've been running Sonarr for about 3 years, no issues. both of my Sonarr managed folders have #recycle folders and have since day 1.

[fryfrog]

Awesome, hopefully that means the fix already exists and just needs to be pulled or adapted.