[Support] Autotagging still not working

[DrMartinus]

What's your question or issue?

I am writing in continuation of this post: [https://github.com//discussions/9902#discussion-8295206]
At some point the classifier stopped again working. I tried to start it again yesterday. In the log, I get these messages:

[2026-04-06 18:25:43,022] [DEBUG] [paperless.classifier] Gathering data from database... [2026-04-06 18:26:55,429] [DEBUG] [paperless.classifier] 7745 documents, 3085 tag(s), 568 correspondent(s), 199 document type(s). 57 storage path(s) [2026-04-06 18:27:10,561] [DEBUG] [paperless.classifier] Vectorizing data... [2026-04-06 18:27:10,561] [DEBUG] [paperless.classifier] Vectorizing data... [2026-04-06 19:49:18,052] [DEBUG] [paperless.classifier] Training tags classifier... [2026-04-07 00:41:53,700] [DEBUG] [paperless.classifier] Training correspondent classifier... [2026-04-07 02:00:05,646] [INFO] [paperless.tasks] Emptying trash of all expired documents [2026-04-07 02:00:06,823] [INFO] [paperless.tasks] Deleted 0 documents from trash

and at the NAS, where I initiated the process via SSH, I get this message:

/usr/local/bin/document_create_classifier: line 17: 6180 Killed s6-setuidgid paperless python3 manage.py document_create_classifier “$@”

I wonder if this could have to do with some operations which are scheduled to take place around 2am at the NAS (automatic software updates, virus scan, etc.). I restarted the classifier again this morning, according to the last log entries there should be a chance that it will finish before 2am the next day, if the NAS activities caused the problem. I will keep an eye on RAM consumption.

I still wonder: why did it stop classifying (because it used to work after going through the process previously, but stopped after days or weeks of operation), and why doesn't it try to restart? Why does it say (after it had worked for a while):
[paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.
There should be sufficient RAM (16 GB, of which about 50% is being used during the process, however I couldn't check at 2 am and don't know if it had built up during the process).

And I sincerely apologize, if there is another post written by me about the same issue or question besides the one referred to above (which is locked and hence cannot be continued by me, or can it?), I looked for it and didn't find any, else I would have added this report there.

What have you tried?

No response

Paperless-ngx version

2.20.10

Host OS

Asustor NAS AS6104T

Installation method

Docker - official image

System status

No response

Relevant logs or output

No response

[stumpylog]

The Killed in your SSH output is the Linux OOM killer terminating the process -- no error message, just SIGKILL. Your NAS maintenance tasks (virus scan, software updates) running around 2am pushed total RAM over the limit while the classifier was mid-training.

The "model does not exist" message afterward is a side effect: the process was killed before it could write the model file to disk, leaving paperless with nothing to load on subsequent runs.

To work around this, schedule the classifier task outside your maintenance window using PAPERLESS_TRAIN_TASK_CRON, or if your NAS allows it, limit the memory available to the virus scanner to leave more headroom for the classifier.

[DrMartinus]

Thank you for your reply, that clarifies some doubts I had. However, I find the RAM consumption quite heavy. I started another run this morning, the first time it was killed just after two hours, the second time is still running, but RAM is at 80-90%, hence I expect it being killed anytime soon. And it will take longer than 24 hours, most likely (after 8 hours, it is still training the tags classifier). I would have to suspend all scheduled tasks, I need to check which ones I can suspend.
Some ideas: The first time it went through with the tags classification (if I get that right), it was already doing the correspondents classification, but it has always to start again from 0 if it fails somewhere during the entire process of classification. Wouldn't it be wise, particularly as it takes that long and consumes that much RAM, to divide the classification into three or four different tasks, which run one after the other, and each builds its database and finishes it independent of where the other processes are?
Probably, it will be too much to aks, but I will never get paperless to work the way it is supposed to work, if I encounter these issues again and again. And it is really tedious to add tags etc. to each document individually (I know, I can do that also for multiple docs, but this benefits me only as long as I add a bunch of older documents with the same background; once paperless has got all those documents, I will add new ones one by one, each requiring a new correspondent, path, tags and what more.
Just FYI: When paperless does not classify documents, 75% or more of the NAS's RAM is free, which means that paperless consumes more than 10 GB for the classification. That may be necessary given the amount of the docs, tags etc., but I'm sure the handling of RAM can be managed more economically.

[DrMartinus]

I just saw that this discussion has been automatically closed because it was marked as answered. Well, I didn't do it. For me, the main question ist, ow to avoid paperless being killed. There was a suggestion what to try, but this suggestion has not been the solution. The process which I ran yesterday had almost finished, it went up to [2026-04-07 21:47:38,131] [DEBUG] [paperless.classifier] Training storage paths classifier..., which is, I guess, the last training step, but then it was killed about 5 hours before the scheduled maintenance by the NAS. The RAM problem was only caused by paperless then, because from what I can see, no other process was started. And it troubles me to know that I might be unable to ever use a very important feature of paperless (=autotagging). Considering that paperless can, according to my observation, not work to its full potential on a NAS with 16 GB RAM (BTW, the NAS has even a 128 GB SSD Cache installed), should become a real problem to its developers, for many NAS's can't have more RAM than that, many have even less. But it seems there are many instances where paperless works flawlessly, and autotagging was running for a while on my NAS as well, but stopped for unknown reasons to do so.
The question remains: what can I do to make this process successful? What do others do? Try it a hundred times? Well, I am at my fourth attempt now (since autotagging stopped working), let's see how far this goes. And why doesn't it work right from the beginning of its installation? Or why does it break every now and then?
Considering that the process takes around 14 to 18 hours to finish (according to last observations), I won't be doing it much more often, but rather look for an alternative, which I do not really want because paperless-ngx seems to me a very good tool for what it does.

[shamoon]

That is a very long time. I guess one question is if you really need 3k tags with auto matching? While the time to train is surprising that does seem like quite a challenging dataset

[DrMartinus]

Well, the documents are of a wide variety and of different walks of life. They shall be accessible not only to me, but to others (in the family) as well. If you look for a document, you need a way to find it, and for me that's the tags. Searching in the documents takes quite a long time (up to a minute or even more), and if the keyword you search for is too common, you might end up with hundreds of documents and won't find the one you are looking for.
Currently, I am hoping that it will go through to the end, it's just at the last step and RAM seems to be sufficient. But for that I had to stop a few more apps which are usually running on the NAS (they don't consume much RAM, but altogether it's about 1.5 GB, which seems good to have for paperless-ngx). I still think that paperless should be able to work with less than 12 GB (exclusively reserved for paperless), but probably the more the better. Unfortunately, the NAS has already double the officially allowed RAM (and handles it just fine), hence I guess that I can't add any more to it.

[DrMartinus]

Ok, it's done, it took about ten hours and 20 minutes in total, and that only because I stopped all other software that could be stopped on the NAS. If I hadn't, it would have taken around 14 to 16 hours and probably not ended properly. How can I make sure that the classifier keeps the data? I hope additional training will be much faster end less RAM consuming, or will it always be more or less the same?