Making a new pickle file for every image that gets added.

[thakurritesh19]

• face_recognition version: latest
• Python version: 3
• Operating System: Ubuntu 18.04

What I Did

In order to save facial embedding for each face I save them in a .pickle file. However, let us say I have 1000 images, embedding for which are saved in a .pickle file.

When a new image is introduced, I have to run the encoding for 1001 images now. This increases training time a lot.

I tried using appending in .yaml files too, but then .match, that I am using to recognize face, doesnt work as desired.

Any solution ?

[unix0r]

We had similiar problem,
we did not use a pickle file, just plain jpg pictures.
If the software runs the first time, all encodings are generated (takes a lot of time) and saved in an array. If a new face is added while the software is running, only the encoding of the new face is added to the array and saved as a file.
With this we could reduce the time of adding new faces to only encoding one face, but loading all faces on start is still time consuming.

[thakurritesh19]

We had similiar problem,
we did not use a pickle file, just plain jpg pictures.
If the software runs the first time, all encodings are generated (takes a lot of time) and saved in an array. If a new face is added while the software is running, only the encoding of the new face is added to the array and saved as a file.
With this we could reduce the time of adding new faces to only encoding one face, but loading all faces on start is still time consuming.

Where is the array ? Where do you save the array of such embedding for future use ? As of now I store them in pickle. however whenever a new face is added I retrain for all the images then add the new face's embedding together in a pickle file. Couldnt find a way of appending in a pickle file.

[thakurritesh19]

Would it be possible using facial_recognition API to query a database for facial embedding and then compare it to identify someone ?

I am using MongoDB to store Facial Embeddings against the name.

[KaramveerSidhu]

Would it be possible using facial_recognition API to query a database for facial embedding and then compare it to identify someone ?

I am using MongoDB to store Facial Embeddings against the name.

Has It been possible to store the encoding in the database and then use it to compare faces..? And then appending the new encoding if a new face is added!

[jsjaskaran]

We had similiar problem,
we did not use a pickle file, just plain jpg pictures.
If the software runs the first time, all encodings are generated (takes a lot of time) and saved in an array. If a new face is added while the software is running, only the encoding of the new face is added to the array and saved as a file.
With this we could reduce the time of adding new faces to only encoding one face, but loading all faces on start is still time consuming.

Where is the array ? Where do you save the array of such embedding for future use ? As of now I store them in pickle. however whenever a new face is added I retrain for all the images then add the new face's embedding together in a pickle file. Couldnt find a way of appending in a pickle file.

What you can do is, run the encodings and store them as pickle. Then when you get new image, run encodings for that new image only. Read pickle file, append new data to it and then save the pickle again. Let me know if that is clear now.

[farhanrbnn]

We had similiar problem,
we did not use a pickle file, just plain jpg pictures.
If the software runs the first time, all encodings are generated (takes a lot of time) and saved in an array. If a new face is added while the software is running, only the encoding of the new face is added to the array and saved as a file.
With this we could reduce the time of adding new faces to only encoding one face, but loading all faces on start is still time consuming.

Where is the array ? Where do you save the array of such embedding for future use ? As of now I store them in pickle. however whenever a new face is added I retrain for all the images then add the new face's embedding together in a pickle file. Couldnt find a way of appending in a pickle file.

What you can do is, run the encodings and store them as pickle. Then when you get new image, run encodings for that new image only. Read pickle file, append new data to it and then save the pickle again. Let me know if that is clear now.

i think this topic is similar with my problem, i just did what @jsjaskaran said. when i get new face encoding from new image, it append to existing pickle file. but my problem is, how to delete the spesific face encoding ? for example i want to delete one person face from my database. any solution ?

[jsjaskaran]

Okay my first question would why you would wanna delete a face encoding?
But if you have to cause of some legal reason then here's an idea about how you can probably delete face encoding for a particular person.

Lets start from beginning :-

1. You have empty 2 pickles, 1 for known_faces and another for known_names.
2. New person comes in and you store encoding in known_faces and name in known_names.
3. Now this process of new person coming in and getting stored in known_faces and known_names goes on.
4. Now you have to delete a specific person encoding, what you can do is delete from known_name and get index of that person and probably you'd have the same index of that person face encoding in known_faces and you can delete using that.

Let me know if this could work.

[farhanrbnn]

Okay my first question would why you would wanna delete a face encoding?
But if you have to cause of some legal reason then here's an idea about how you can probably delete face encoding for a particular person.

Lets start from beginning :-

1. You have empty 2 pickles, 1 for known_faces and another for known_names.
2. New person comes in and you store encoding in known_faces and name in known_names.
3. Now this process of new person coming in and getting stored in known_faces and known_names goes on.
4. Now you have to delete a specific person encoding, what you can do is delete from known_name and get index of that person and probably you'd have the same index of that person face encoding in known_faces and you can delete using that.

Let me know if this could work.

so i have project for attendance system using face recognition, so if someone resign from the company, all of data from that employee must be delete.

i will try your advice, it does make sense.

thank you

[rathishkumar]

Has It been possible to store the encoding in the database and then use it to compare faces..? And then appending the new encoding if a new face is added!

Yes, possible. I had the same issue mentioned in this thread, I have implemented using PostgreSQL, you can use other databases as well. I have shared the details here for starters.