Title limit is 13775? Can't scan missing in 15k library but can in 12k library.

[Whixey]

A library of 12k titles worked. Later, with same and then a fresh install, a scan of a library of 15k titles was unsuccessful after multiple attempts.

I have tried to find hard some limits or max titles in the source but I'm not quite sure what I'm looking for. It looks like the title limit might be 15k and the scan missing limit might be 13775? I only say this because there are more than 15k titles but gaps only reports exactly 15k titles and when you go to missing tab and then back to title tab it then reverts to 13775. Which is odd.

This is a vanilla docker install with single plex source.

The error is that when I start the scan for missing, the page loads new elements for a second then loads back the gap icon like nothing ever happened.

What else should I provide/try?

TIA

[Whixey]

yeah,
grep -o -i tmdbId ownedMovies.json | wc -l
gave 13775.

[Whixey]

When working with the logs- this happened when piping or redirecting docker logs for the container.

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.sun.xml.bind.v2.runtime.reflect.opt.Injector (jar:file:/usr/app/gaps.jar!/BOOT-INF/lib/jaxb-impl-2.3.0.jar!/) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int)
WARNING: Please consider reporting this to the maintainers of com.sun.xml.bind.v2.runtime.reflect.opt.Injector
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

[JasonHHouse]

Did you try clearing the data in configuration and rerunning? It might be something that isn't saving correctly.

One limit I could think of off the top of my head is memory usage since that's a large plex server, that's a lot of movies and even more movies in collections to find.

If you can send the full docker log, that would help a ton.

[Whixey]

I'll attach that log. I did not think to watch the plex vms resources during this since its on a different box. I will do that right after this post.

yeah, the only permutation I did not try was a completely fresh container. after failing the first time with 15 I checked the image for updates (none) then deleted the existing plex share and re-added that and re-ran to same effect. Next I deleted the plex share again and then I just moved my {local} to a different name then stopped, removed, created, started the docker and in the app added the library and rescanned, again gave similar local data and failures.

if it helps this is docker in virtualbox.

concerning memory I watched side by side with docker stats and the base system memory was near constant at 30% also the peak, the vm 0.1% also peak and gaps was fluctuating between 300-450MB. even when adding the initial library the process seemed well fed.

I tried to profile stuff through docker and got lost down a rabbit hole. nothing I discovered led me to believe it was a specific variable limit or what have you, but again, I may not know what I should be looking for.

logs
docker logs [gaps] > gaps.logs
gaps.log.gz

also please do not judge me for not changing the local folder!

[Whixey]

I checked plex. Unless I missed it and now I can't trigger that event again it is fine. I did not see any OOM messages and the performance averages for the last few hours were all the same. Plex is installed directly to a vm and has 16GB.

[Whixey]

It works now. Just looks like 13775 is the limit. (I forgot to re-add the TMDB key)

[Whixey]

Now that its running- the memory stays at 400MB, occasionally dropping to 399.9MB.

[Whixey]

it finished. it works on the 13775 but is unaware of anything past that.

[JasonHHouse]

Are all 15k movies matched to TMDB/IMDB movies? I have sports and other things that don't match and are ignored by gaps. The same happens for movies like fan-made Star Wars Project 4k77 movies.

[Whixey]

Yes.

EDIT: Or at least all 15k in plex fetch posters/ratings and all 13775 in gaps have those posters. So maybe not all have tmdb as -1 and valid imdb?