Manual job start --> "unhandeld exception" #208

majordomus0 · 2020-01-01T14:52:57Z

Hi, I have a new issue after having successfully installed facerecognition

My system is a Raspberry Pi 4 with 4GB of RAM, running RaspBian 10 Buster.
Nextcloud 17.0.2.1 with manually installed FaceRecognition App, becauses it was not available via the App Store.
swap file size is 2GB, as Matias told me before
Edit: wrote my swap file was 4GB, actually it was 2GB as matias told me

After installing facerecog. with Matias' help successfully, and after having activated analysis in the NC settings GUI, nothing happened after 24 h , it told me "analysis is disabled", so I decided to manually start the process with

root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:background_job

After executing this command, I get this error:

1/10 - Executing task CheckRequirementsTask (Check all requirements)
An unhandled exception has been thrown:
TypeError: Return value of OCA\FaceRecognition\Helper\MemoryLimits::getTotalMemoryLinux() must be of the type int, float returned in /var/www/nextcloud/apps/facerecognition/lib/Helper/MemoryLimits.php:82
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/Helper/MemoryLimits.php(56): OCA\FaceRecognition\Helper\MemoryLimits::getTotalMemoryLinux()
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/CheckRequirementsTask.php(82): OCA\FaceRecognition\Helper\MemoryLimits::getAvailableMemory()
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(115): OCA\FaceRecognition\BackgroundJob\Tasks\CheckRequirementsTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, false, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}root@nextcloudpi:/var/www/nextcloud#

Anybody knows how to fix this? Again, I am no linux/programming pro and would appreciate any type of help, also didn't find any similar threads in the support area of the repository

These are my settings in the nextcloud GUI

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

matiasdelellis · 2020-01-01T15:09:00Z

Hi @majordomus0

After installing facerecog. with Matias' help successfully, and after having activated analysis in the NC settings GUI, nothing happened after 24 h , it told me "analysis is disabled", so I decided to manually start the process with

Now the operation is 100% manual. Therefore they must execute it manually. or within a cron or systemd Timers. I have pending to write a wiki about this. 😞

swap file size is 2GB, as Matias told me before
Edit: wrote my swap file was 4GB, actually it was 2GB as matias told me

You can make the swap file of the size you want. The interesting thing about using a file is just trying different sizes. When you have a size that works, you can activate it before the analysis, and deactivate it at the end. Keep in mind that having it continuously activated can slow your system. 😉

 1/10 - Executing task CheckRequirementsTask (Check all requirements)
An unhandled exception has been thrown:
TypeError: Return value of OCA\FaceRecognition\Helper\MemoryLimits::getTotalMemoryLinux() must be of the type int, float returned in /var/www/nextcloud/apps/facerecognition/lib/Helper/MemoryLimits.php:82

Seems dup of #199

Please share the content of your /proc/meminfo file.. 😉

majordomus0 · 2020-01-01T15:20:47Z

Thanks for your answer!

Now the operation is 100% manual. Therefore they must execute it manually. or within a cron or
systemd Timers. I have pending to write a wiki about this. 😞

Just to clarify that I understood it right:
By manually starting the process with background:job , the automatic analysis will be disabled "forever" and needs to be configured manually from then?
If yes, I will reset my system after having fixed all errors to have back the auto-mode :-)

Should I have been more patient and wait longer until the program would have started analysis?

This is the content of my meminfo file:

MemTotal:        3999784 kB
MemFree:         3118864 kB
MemAvailable:    3523316 kB
Buffers:           60524 kB
Cached:           468128 kB
SwapCached:            0 kB
Active:           499428 kB
Inactive:         232764 kB
Active(anon):     203964 kB
Inactive(anon):    28452 kB
Active(file):     295464 kB
Inactive(file):   204312 kB
Unevictable:          16 kB
Mlocked:              16 kB
HighTotal:       3264512 kB
HighFree:        2596716 kB
LowTotal:         735272 kB
LowFree:          522148 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                12 kB
Writeback:             0 kB
AnonPages:        203604 kB
Mapped:            96412 kB
Shmem:             28880 kB
Slab:              79948 kB
SReclaimable:      36072 kB
SUnreclaim:        43876 kB
KernelStack:        2680 kB
PageTables:         4248 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1999892 kB
Committed_AS:    4258088 kB
VmallocTotal:     245760 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
Percpu:              752 kB
CmaTotal:         262144 kB
CmaFree:          222168 kB

matiasdelellis · 2020-01-02T14:28:46Z

Should I have been more patient and wait longer until the program would have started analysis?

The detection of the analysis status is a bit archaic. It tells you stopped, because due to the last error it was never executed yet.

Now I do tests .. 😉

matiasdelellis · 2020-01-02T15:14:47Z

Hi @majordomus0

Your /proc/meminfo file is valid. 😕
I guess it's a error in the conversion of types in php, which is very strange. 😞 But considering that we talk about an ARM microprocessor everything can be. 😅

It's just a test .. but you can try applying this patch? 😉

diff --git a/lib/Helper/MemoryLimits.php b/lib/Helper/MemoryLimits.php
index a4deb6a..707de16 100644
--- a/lib/Helper/MemoryLimits.php
+++ b/lib/Helper/MemoryLimits.php
@@ -79,7 +79,7 @@ class MemoryLimits {
                }
                fclose($fh);
                $memKb = intval($mem);
-               return $memKb * 1024;
+               return intval($memKb * 1024);
        }

        /**

majordomus0 · 2020-01-02T21:43:56Z

After reading up a couple of hours how to patch files on linux/raspbian, it worked out for me :-)

Now the process seems to work after patching the file with the --ignore-whitespace command.
This looks good at first :-) but..

each analysis of the image stops with the following error:

root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:background_job

1/10 - Executing task CheckRequirementsTask (Check all requirements)

Seems that you configured1907 MB of memory, however, this exceedsdetected as maximum for PHP: 1024MB.
We will ignore the FaceRecognition settings, and we will limit to that.
2/10 - Executing task CheckCronTask (Check that service is started from either cron or from command)
3/10 - Executing task LockTask (Acquire lock so that only one background task can run)
4/10 - Executing task DisabledUserRemovalTask (Purge all the information of a user when disable the analysis.)
5/10 - Executing task StaleImagesRemovalTask (Crawl for stale images (either missing in filesystem or under .nomedia) and remove them from DB)
6/10 - Executing task CreateClustersTask (Create new persons or update existing persons)
Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:

have 1000 faces already processed (you have 0),

have 100 images (you have 19),

or you need to have 95% of you images processed (you have 0.00%)
7/10 - Executing task AddMissingImagesTask (Crawl for missing images for each user and insert them in DB)
Finding missing images for user ncp
8/10 - Executing task EnumerateImagesMissingFacesTask (Find all images which don't have faces generated for them)
9/10 - Executing task ImageProcessingTask (Process all images to extract faces)
NOTE: Starting face recognition. If you experience random crashes after this point, please look FAQ at https://github.com/matiasdelellis/facerecognition/wiki/FAQ
Processing image /media/USBdrive/ncdata/ncp/files/Photos/IMG_20191125_152132.jpg
Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
Processing image /media/USBdrive/ncdata/ncp/files/IMG_20200101_003456.jpg
Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
Processing image /media/USBdrive/ncdata/ncp/files/Photos/IMG_20191125_181719.jpg
Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
Processing image /media/USBdrive/ncdata/ncp/files/IMG_20191225_140524.jpg
Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
Processing image /media/USBdrive/ncdata/ncp/files/IMG_20191224_171227.jpg
Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
Processing image /media/USBdrive/ncdata/ncp/files/Photos/Coast.jpg
Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
Processing image /media/USBdrive/ncdata/ncp/files/Photos/Nut.jpg
Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
Processing image /media/USBdrive/ncdata/ncp/files/Photos/Hummingbird.jpg

matiasdelellis · 2020-01-02T23:09:03Z

Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length

It is memory problem.. Do you have the swap file activated? Note that it does not keep activated between reboots..

On the other hand:

this exceedsdetected as maximum for PHP: 1024MB.

Remember that you must configure more memory to PHP.. 😉

https://www.php.net/manual/en/ini.core.php#ini.sect.resource-limits
Assign at least 2 GB

... and finally:

Seems that you configured1907 MB of memory

I recommend that you configure 1Gb in the application settings panel .. Higher value will become very slow for the raspberry .. and this value already gives very good results .. 😉

majordomus0 · 2020-01-03T00:31:21Z

First of all, let me say thank you again for helping and the quick answers!

After raising my php memory to 3GB (changed the ini in both folders cli+fpm and restarted), and assigning 2GB to the swap file and activating it, I get a

Processing image /media/USBdrive/ncdata/ncp/files/Photos/IMG_20191224_171227.jpg
Error during background task execution
If error is not transient, this means that core component of face recognition is not working properly
and that quantity and quality of detected faces and person will be low or suboptimal.
You probably want to file an issue (please include exception below) to: https://github.com/matiasdelellis/facerecognition/issues

In ImageProcessingTask.php line 186:
                                                                               
  Not enough memory to run face recognition! Please look FAQ at https://githu  
  b.com/matiasdelellis/facerecognition/wiki/FAQ

As you told me, I configured 1GB in the Nextcloud application panel of FaceRecog

This is a pity, because I thought 4GB of ram on a machine, even if it's a raspi (it's the new model 4) would be sufficient :-(
One last possibility which I haven't tried out yet is to enable compressed ram / zRam , via the nextcloudpi GUI, do you think this would make a difference?

Just for info, this is my free -m output

root@nextcloudpi:/var/www/nextcloud# free -m
total used free shared buff/cache available
Mem: 3906 338 3273 21 294 3424
Swap: 2047 0 2047

Do you have any other ideas how to solve this besides buying a stronger machine?
Bought a Raspi4 with a SSD to de-google my life and NC is running on it like a charm besides face recog....but hey in your introduction you wrote that one needs to make the hands dirty to get it running so let's go on :-)

matiasdelellis · 2020-01-03T02:21:29Z

It is still a memory problem, but seeing your numbers of free output found it very strange. It seems that you have 5 GB free, which is more than enough. 😕

First check memory_limit:

[matias@nube nextcloud]$ php -i | grep memory_limit
memory_limit => 2048M => 2048M

Try two more things.. First execute the command with more debug and show me the complete log again.

root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:background_job -vvv

Then, you can try to force use a smaller image size to analysis..

root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:background_job -vvv --max_image_area 786432

With this it uses 1024x768 images, and you can try even smaller. But that would not be a real solution. 😞

As you told me, I configured 1GB in the Nextcloud application panel of FaceRecog

This configuration is a bit empirical, but it obviously depends on the hardware and we don't test it in ARM. Dlib is probably not optimized for ARM and use more memory.

One last possibility which I haven't tried out yet is to enable compressed ram / zRam , via the nextcloudpi GUI, do you think this would make a difference?

Sincerely I dont know. You can try.

Do you have any other ideas how to solve this besides buying a stronger machine?
Bought a Raspi4 with a SSD to de-google my life and NC is running on it like a charm besides face recog....but hey in your introduction you wrote that one needs to make the hands dirty to get it running so let's go on :-)

I also think that today is a good machine.. but the last rappi I used was 1A, so I don't have good memories .. haha.
Hopefully we will make it work well with this hardware. 😉

majordomus0 · 2020-01-03T11:38:04Z

Hi Matias

Sincerely I dont know. You can try.
Ok, compressed ram didn't make a difference

This is my output after using the -vvv parameter

pi@nextcloudpi:~ $ sudo -i
root@nextcloudpi:~# free -m
              total        used        free      shared  buff/cache   available
Mem:           3906         262        3393          19         249        3502
Swap:          2047           0        2047
root@nextcloudpi:~# php -i | grep memory_limit
memory_limit => 3000000000 => 3000000000
root@nextcloudpi:~# cd /var/www/nextcloud/
root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:background_job -vvv
1/10 - Executing task CheckRequirementsTask (Check all requirements)
	Found 2047 MB available to PHP.
	Applying the memory limit to 1144MB configured in settings.
2/10 - Executing task CheckCronTask (Check that service is started from either cron or from command)
3/10 - Executing task LockTask (Acquire lock so that only one background task can run)
4/10 - Executing task DisabledUserRemovalTask (Purge all the information of a user when disable the analysis.)
yielding
5/10 - Executing task StaleImagesRemovalTask (Crawl for stale images (either missing in filesystem or under .nomedia) and remove them from DB)
	Skipping stale images removal for user ncp as there is no need for it
6/10 - Executing task CreateClustersTask (Create new persons or update existing persons)
	Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
	* have 1000 faces already processed (you have 0),
	* have 100 images (you have 12),
	* or you need to have 95% of you images processed (you have 0.33%)
yielding
7/10 - Executing task AddMissingImagesTask (Crawl for missing images for each user and insert them in DB)
	Skipping full image scan for user ncp
8/10 - Executing task EnumerateImagesMissingFacesTask (Find all images which don't have faces generated for them)
yielding
9/10 - Executing task ImageProcessingTask (Process all images to extract faces)
	NOTE: Starting face recognition. If you experience random crashes after this point, please look FAQ at https://github.com/matiasdelellis/facerecognition/wiki/FAQ
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/Nut.jpg
	Image scaled from 2000x1333 to 1148x765 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/IMG_20191224_171212.jpg
	Image scaled from 4000x3000 to 1083x812 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/Nextcloud Community.jpg
	Image scaled from 3000x2000 to 1148x765 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/Coast.jpg
	Image scaled from 1100x734 to 1148x766 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/Hummingbird.jpg
	Image scaled from 2000x1333 to 1148x765 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/IMG_20191125_181719.jpg
	Image scaled from 4000x3000 to 1083x812 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/IMG_20191217_190511.jpg
	Image scaled from 4000x3000 to 1083x812 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/IMG_20191213_071154.jpg
	Image scaled from 4000x3000 to 1083x812 (since max image area is 878906 pixels^2)
	Faces found: 0. Image will be skipped because of the following error: std::bad_array_new_length
	Exception: std::bad_array_new_length in /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:229
Stack trace:
#0 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(229): CnnFaceDetection->detect('/media/myCloudD...')
#1 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php(171): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->findFaces(Object(CnnFaceDetection), Object(OCA\FaceRecognition\Db\Image))
#2 /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php(120): OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute(Object(OCA\FaceRecognition\BackgroundJob\FaceRecognitionContext))
#3 /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php(138): OCA\FaceRecognition\BackgroundJob\BackgroundService->execute(0, true, NULL, NULL)
#4 /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php(255): OCA\FaceRecognition\Command\BackgroundCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#5 /var/www/nextcloud/3rdparty/symfony/console/Application.php(908): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#6 /var/www/nextcloud/3rdparty/symfony/console/Application.php(269): Symfony\Component\Console\Application->doRunCommand(Object(OCA\FaceRecognition\Command\BackgroundCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#7 /var/www/nextcloud/3rdparty/symfony/console/Application.php(145): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#8 /var/www/nextcloud/lib/private/Console/Application.php(214): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#9 /var/www/nextcloud/console.php(97): OC\Console\Application->run()
#10 /var/www/nextcloud/occ(11): require_once('/var/www/nextcl...')
#11 {main}
10/10 - Executing task UnlockTask (Release obtained lock)
You have new mail in /var/mail/root
root@nextcloudpi:/var/www/nextcloud#

majordomus0 · 2020-01-03T11:41:40Z

Just for your Info:

After raising my Memory limit in my Nextcloud to 1.9 Gigabyte, the output is a little bit different.


root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:reset_all
Reset successfully done
root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:background_job -vvv
1/10 - Executing task CheckRequirementsTask (Check all requirements)
	Found 2047 MB available to PHP.
	Applying the memory limit to 1907MB configured in settings.
2/10 - Executing task CheckCronTask (Check that service is started from either cron or from command)
3/10 - Executing task LockTask (Acquire lock so that only one background task can run)
4/10 - Executing task DisabledUserRemovalTask (Purge all the information of a user when disable the analysis.)
yielding
5/10 - Executing task StaleImagesRemovalTask (Crawl for stale images (either missing in filesystem or under .nomedia) and remove them from DB)
	Skipping stale images removal for user ncp as there is no need for it
6/10 - Executing task CreateClustersTask (Create new persons or update existing persons)
	Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
	* have 1000 faces already processed (you have 0),
	* have 100 images (you have 0),
	* or you need to have 95% of you images processed (you have 0.00%)
yielding
7/10 - Executing task AddMissingImagesTask (Crawl for missing images for each user and insert them in DB)
	Finding missing images for user ncp
	Found /ncp/files/Nextcloud.png
	Found /ncp/files/Photos/Coast.jpg
	Found /ncp/files/Photos/Hummingbird.jpg
	Found /ncp/files/Photos/IMG_20191125_152132.jpg
	Found /ncp/files/Photos/IMG_20191125_181719.jpg
	Found /ncp/files/Photos/IMG_20191213_071154.jpg
	Found /ncp/files/Photos/IMG_20191217_190511.jpg
	Found /ncp/files/Photos/IMG_20191224_171212.jpg
	Found /ncp/files/Photos/IMG_20191224_171227.jpg
	Found /ncp/files/Photos/Nextcloud Community.jpg
	Found /ncp/files/Photos/Nut.jpg
	Found /ncp/files/testimage.jpg
yielding
8/10 - Executing task EnumerateImagesMissingFacesTask (Find all images which don't have faces generated for them)
yielding
9/10 - Executing task ImageProcessingTask (Process all images to extract faces)
	NOTE: Starting face recognition. If you experience random crashes after this point, please look FAQ at https://github.com/matiasdelellis/facerecognition/wiki/FAQ
yielding
	Processing image /media/USBdrive/ncdata/ncp/files/Photos/Nextcloud Community.jpg
	Image scaled from 3000x2000 to 1482x988 (since max image area is 1464843 pixels^2)
Error during background task execution
If error is not transient, this means that core component of face recognition is not working properly
and that quantity and quality of detected faces and person will be low or suboptimal.
You probably want to file an issue (please include exception below) to: https://github.com/matiasdelellis/facerecognition/issues

In ImageProcessingTask.php line 186:
                                                                               
  [RuntimeException]                                                           
  Not enough memory to run face recognition! Please look FAQ at https://githu  
  b.com/matiasdelellis/facerecognition/wiki/FAQ                                
                                                                               

Exception trace:
 () at /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/Tasks/ImageProcessingTask.php:186
 OCA\FaceRecognition\BackgroundJob\Tasks\ImageProcessingTask->execute() at /var/www/nextcloud/apps/facerecognition/lib/BackgroundJob/BackgroundService.php:120
 OCA\FaceRecognition\BackgroundJob\BackgroundService->execute() at /var/www/nextcloud/apps/facerecognition/lib/Command/BackgroundCommand.php:138
 OCA\FaceRecognition\Command\BackgroundCommand->execute() at /var/www/nextcloud/3rdparty/symfony/console/Command/Command.php:255
 Symfony\Component\Console\Command\Command->run() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:908
 Symfony\Component\Console\Application->doRunCommand() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:269
 Symfony\Component\Console\Application->doRun() at /var/www/nextcloud/3rdparty/symfony/console/Application.php:145
 Symfony\Component\Console\Application->run() at /var/www/nextcloud/lib/private/Console/Application.php:214
 OC\Console\Application->run() at /var/www/nextcloud/console.php:97
 require_once() at /var/www/nextcloud/occ:11

face:background_job [-u|--user_id USER_ID] [-M|--max_image_area MAX_IMAGE_AREA] [-t|--timeout TIMEOUT]

root@nextcloudpi:/var/www/nextcloud#

matiasdelellis · 2020-01-03T14:31:19Z

root@nextcloudpi:~# free -m
              total        used        free      shared  buff/cache   available
Mem:           3906         262        3393          19         249        3502
Swap:          2047           0        2047
root@nextcloudpi:~# php -i | grep memory_limit
memory_limit => 3000000000 => 3000000000

According to this you have the memory_limit applied and 5 GB of memory free. Should be enough. 😞

Do you have any way to monitor memory usage? Personally I recommend netdata, but I don't know how it works with raspberry. Maybe you can open another ssh session in parallel and see it in htop as memory behaves while executing the command. This to corroborate that you are really running out of memory..

You can directly deactivate the memory_limit using -1. At least as test. But you can't control the kernel behavior, maybe the nextcloudpi image has some configuration that we don't know.

Then, you can try to force use a smaller image size to analysis..

root@nextcloudpi:/var/www/nextcloud# sudo -u www-data php occ face:background_job -vvv --max_image_area 786432

Also try this (If it fails reduce the size to 640*480=499200‬) to make sure that with smaller image size it works correctly. (Literally memory usage depends linearly on it.)

matiasdelellis · 2020-01-03T14:39:31Z

You can also know if is the kernel that kills the application by viewing its messages. Use dmesg -w

matiasdelellis · 2020-02-06T00:35:57Z

Hi @majordomus0
Please, I invite you to try again with master. The memory calculation was drastically changed.

matiasdelellis · 2020-03-12T13:34:47Z

I understand that it was fixed in the last release. If the problem still persists please reopen this issue.
Regards,
Matias

matiasdelellis mentioned this issue Jan 6, 2020

Force cast to integer in conversion after multiplying by a number.. #211

Merged

matiasdelellis closed this as completed Mar 12, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Manual job start --> "unhandeld exception" #208

Manual job start --> "unhandeld exception" #208

majordomus0 commented Jan 1, 2020 •

edited by matiasdelellis

matiasdelellis commented Jan 1, 2020

majordomus0 commented Jan 1, 2020 •

edited

matiasdelellis commented Jan 2, 2020

matiasdelellis commented Jan 2, 2020

majordomus0 commented Jan 2, 2020 •

edited

matiasdelellis commented Jan 2, 2020

majordomus0 commented Jan 3, 2020 •

edited

matiasdelellis commented Jan 3, 2020

majordomus0 commented Jan 3, 2020

majordomus0 commented Jan 3, 2020

matiasdelellis commented Jan 3, 2020

matiasdelellis commented Jan 3, 2020

matiasdelellis commented Feb 6, 2020

matiasdelellis commented Mar 12, 2020

Manual job start --> "unhandeld exception" #208

Manual job start --> "unhandeld exception" #208

Comments

majordomus0 commented Jan 1, 2020 • edited by matiasdelellis

matiasdelellis commented Jan 1, 2020

majordomus0 commented Jan 1, 2020 • edited

matiasdelellis commented Jan 2, 2020

matiasdelellis commented Jan 2, 2020

majordomus0 commented Jan 2, 2020 • edited

matiasdelellis commented Jan 2, 2020

majordomus0 commented Jan 3, 2020 • edited

matiasdelellis commented Jan 3, 2020

majordomus0 commented Jan 3, 2020

majordomus0 commented Jan 3, 2020

matiasdelellis commented Jan 3, 2020

matiasdelellis commented Jan 3, 2020

matiasdelellis commented Feb 6, 2020

matiasdelellis commented Mar 12, 2020

majordomus0 commented Jan 1, 2020 •

edited by matiasdelellis

majordomus0 commented Jan 1, 2020 •

edited

majordomus0 commented Jan 2, 2020 •

edited

majordomus0 commented Jan 3, 2020 •

edited