timeout set too low in ./occ

[arch-user-france1]

The classifier doesn't work anymore

A node process starts as it should but shortly after it reports 'classifier timed out'

I used occ to run the classifier manually

[arch-user-france1]

recently an automatic run classified some (many)
Did the model update?

[bonswouar]

I believe the model hasn't been updated since august.
You don't have any other log (or other output when running manually)? That could be one specific image that causes the timeout, maybe related to #98, how much free ram do you have?

Seeing those issues maybe we should add a command option to have a verbose output when running the classifier for debugging purpose, what do you think @marcelklehr ?

[arch-user-france1]

It's a very little chance to work. There's no Iasue with RAM (3GB free and 11GB swap, includes zram of min. 2GB)

I don't know where the logs should be - it's made very poor (no output..)

Gratulation for the 100. issue...

[arch-user-france1]

Wait... How do I run it with node-js?

[arch-user-france1]

I ran classifier_imagenet

[arch-user-france1]

Hmmm wait...

It's in red: Failed to classify images
And in white: Classifier processes timeout

Actually, there's almost 1GB RAM free when I see a node process but it only uses 6% CPU and there's a swappiness process using almost same % CPU

[arch-user-france1]

Now the classifier process runs
Maybe there's really a little problem with a timeout setting. Uh the process got killer by the occ script again... It looks like the timeout is really too short (idk what timeout)

[arch-user-france1]

Uh actually it is #98

[arch-user-france1]

[arch-user-france1]

This is a new problem!!! The first version I had worked also when there was no really free RAM

Edit: I remember that it worked at 2. try, the first try had Sams error

[bonswouar]

Alright I'm just gonna close this one for clarity

[arch-user-france1]

Please remove duplicate and reopen

I don't have the RAM issue anymore

[bonswouar]

Ok the ram problem might be unrelated to your initial problem.
Although you said earlier it did run, did it work then? What have you changed since?
Is your error still Cannot find module 'fs/promises'? Did you change node's path in the config?

[arch-user-france1]

It seems to work but then the process dies because it's parrent (the ./occ command) decides to stop because of 'timeout'

[arch-user-france1]

I ran classifier_imagenet

This is just an error when I try running it manually with node. I don't even know what I'm doing wrong

[bonswouar]

I fear timeout could mean anything, what's the output exactly, the same as your screenshot (Cannot find module 'fs/promises')?

EDIT : Ok thanks for the screenshot, what is your syntax to run it manually?

[arch-user-france1]

I ran node src/classifier_imagenet.js (full-)path-to-picture

Am I doing this correct?

[bonswouar]

No haha :)
It's ./apps/recognize/bin/node apps/recognize/src/classifier_imagenet.js (full-)path-to-picture

[arch-user-france1]

This is what happens when I just run it with occ: https://cloud.privacyy.ch/index.php/s/DE4ZSRx67k2BnXy

[arch-user-france1]

Uh that was a hit
I was downloading something - didn't have much free RAM and now the classifier..
It's currently filling up my RAM

It looks like it's just a timeout wich is too small

[arch-user-france1]

Ok Where the results:

Allocation exceeds free memory 4 times (ONLY A WARNING NOT ERROR)

[arch-user-france1]

[bonswouar]

Maybe try to run it manually on the last image that timed out to be sure there is not any other problem : /var/www/data/privacyy/files/Talk/-59267722...
If it's still the 10% Warnings I'll close this issue for duplicate if it's ok for you

[arch-user-france1]

It's not duplicate not Ok for m
I only have to know where this timeout is set
Then I fix it

[bonswouar]

Timeout is set to 10 seconds per image (50 in pure JS mode), it shouldn't take more. The warning may indicates there is a memory problem and thus it probably takes more time than it should

EDIT : bad values, those are for face service, imagenet values are 12s / 480s

[arch-user-france1]

I tried to check the picture but it doesn't exist
And I don't know wich picture the first is - I can't scroll. May use the command | head but idk if it would work

[arch-user-france1]

Timeout is set to 10 seconds per image (50 in pure JS mode), it shouldn't take more. The warning may indicates there is a memory problem and thus it probably takes more time than it should

No on my system it has to put serveral things to RAM like some Apache cache files so it takes more for the 1. picture than for the others (I guess lol)

[arch-user-france1]

Can you give me the path of this file so I can mess around?
I just need it
I got already the script wich has the timeout message

[arch-user-france1]

Yes I found it!!! lib/Service/ClassifyImagenetService.php

[arch-user-france1]

I recommend to make it higher
It will fix everything of your needs on low-RAM

[arch-user-france1]

@marcelklehr take notice of it please and decide wether it should be rised or not

[arch-user-france1]

Hmm it's now classifying with 5Seconds/Picture

It is totally worth it to at least provide it at the start of the application

[bonswouar]

You could leave the issue open, your timeout change will be reverted next time you update.

I don't know what's the best way for that, we could make it configurable but that's a bit tricky for any user to know which timeout value he'd want..
Another way could be to make a tiny benchmark (only one image for example) during install and set a custom timeout value based on his duration. @marcelklehr, thoughts?

[arch-user-france1]

Could you remove the duplicate-thing? Thanks... There's no duplicate around

Yeah, prob. there can/should be a setting in src/.../admin... (Nextcloud settings)
Or maybe just change the timeout for the first 5 pictures to 480 (what I have)

Marcelklehr wasn't here this day.. Let's just wait

[marcelklehr]

Seeing those issues maybe we should add a command option to have a verbose output when running the classifier for debugging purpose, what do you think @marcelklehr ?

Yeah, completely moving the debug output to the nextcloud log was a bad move it seems. I'm not sure how best to have debug output at both ends and keep the code DRY.

I don't know what's the best way for that, we could make it configurable but that's a bit tricky for any user to know which timeout value he'd want.

I'm happy to increase the timeout, even if that means it's gonna be much higher. It was only ever meant as a way to avoid waiting on the process forever. I just haven't had the time to make the change.

[bonswouar]

Alright, if a high timeout value isn't a problem let's go for that!