Initial version of an external Model

[matiasdelellis]

Issue #210 and #388

Already with something so simple, it should work. 😅 , but it fails, and I want to see scrusizeer. 😅

[matiasdelellis]

[matiasdelellis]

[matias@nube nextcloud]$ sudo -u apache php occ face:setup -m 5
The files of model 5 (ExternalModel) are already installed
The model 5 (ExternalModel) was configured as default
[matias@nube nextcloud]$ sudo -u apache php occ face:setup -m 
You must indicate the ID of the model to install
+----+---------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| Id | Enabled | Name          | Description                                                                                                                                              |
+----+---------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
| 1  |         | Default       | Main model, using dlib defaults: mmod_human_face_detector.dat, shape_predictor_5_face_landmarks.dat and dlib_face_recognition_resnet_model_v1.dat        |
| 2  |         | DlibCnn68     | Alternative default model, using dlib: mmod_human_face_detector.dat, shape_predictor_68_face_landmarks.dat and dlib_face_recognition_resnet_model_v1.dat |
| 3  |         | DlibHog       | DDlib HOG Model which needs lower requirements                                                                                                           |
| 4  |         | DlibCnnHog5   | Extends the main model, doing a face validation with the Hog detector                                                                                    |
| 5  | *       | ExternalModel | External Model (EXPERIMENTAL)                                                                                                                            |
+----+---------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------+
[matias@nube nextcloud]$ sudo -u apache php occ face:background_job -u user
1/10 - Executing task CheckRequirementsTask (Check all requirements)
2/10 - Executing task CheckCronTask (Check that service is started from either cron or from command)
3/10 - Executing task LockTask (Acquire lock so that only one background task can run)
4/10 - Executing task DisabledUserRemovalTask (Purge all the information of a user when disable the analysis.)
5/10 - Executing task StaleImagesRemovalTask (Crawl for stale images (either missing in filesystem or under .nomedia) and remove them from DB)
6/10 - Executing task CreateClustersTask (Create new persons or update existing persons)
	Skipping cluster creation, not enough data (yet) collected. For cluster creation, you need either one of the following:
	* have 1000 faces already processed
	* or you need to have 95% of you images processed
	Use stats command to track progress
7/10 - Executing task AddMissingImagesTask (Crawl for missing images for each user and insert them in DB)
	Finding missing images for user user
8/10 - Executing task EnumerateImagesMissingFacesTask (Find all images which don't have faces generated for them)
9/10 - Executing task ImageProcessingTask (Process all images to extract faces)
	NOTE: Starting face recognition. If you experience random crashes after this point, please look FAQ at https://github.com/matiasdelellis/facerecognition/wiki/FAQ
	Processing image /media/datos/Services/nextcloud/data/user/files/Photos/The Big Bang Theory/1556141424-565364517878206616176057749943542714755750n.jpg
	Faces found: 7
	Processing image /media/datos/Services/nextcloud/data/user/files/Photos/The Big Bang Theory/Big Bang Theory Season 12 Episode 6 "The Imitation Perturbation.jpeg
	Faces found: 4
	Processing image /media/datos/Services/nextcloud/data/user/files/Photos/The Big Bang Theory/The-Big-Bang-Theory-Season-Two-billboard-1548.jpg
	Faces found: 4
	Processing image /media/datos/Services/nextcloud/data/user/files/Photos/The Big Bang Theory/1*cynXSoakjksBCXgsaHUjPA.jpeg

[matiasdelellis]

Better docs:
https://github.com/matiasdelellis/facerecognition-external-model#install

[matiasdelellis]

@stalker314314 , I know you have resistance to this. Me too, but it can really solve a lot of problems, and thanks to all the previous work it was very easy... I guess the benefits outweigh the concerns ..

The problem of data privacy is the same of the demo versions of Libreoffice Online, or the High Performance Talk server. I was thinking of adding some type of "eula", but it would scare more than necessary. So, It will be the responsibility of each administrator use this locally, or if rent an instance on amazon with gpu comment it to users if necessary.

[matiasdelellis]

facerecognition.tar.gz

[matiasdelellis]

First test... 🙈 :

My Nextcloud instance runs on an AMD A8-9600 (4 Cores 3.4Gz) with 8Gb ram... Analyze my 433 photos of TBBT using the Model 1 image size 1920x1440, and it took 1 hour 49 minutes.

So, configure the external model, using the reference model, (Which implements the same model 1), on my laptop with microprocessor Intel i7-7500U (2 Cores, 4 threads 2.70GHz) and 20GB RAM with exactly the same parameters, ..and it only took 47 minutes..

Same parameters, same results, and half the time.. It seems promising. 😄