Multiple faces detected in the provided image for source_face.

[cagrisungur]

I am using the dev branch of the project.

When the input photo contains multiple faces the request fails.

Is this the expected behavior?

_face_embedding.py
if len(detected_faces) > 1: raise ValueError(f"Multiple faces detected in the provided image for {face_name}.")

Question

If there are multiple faces in an image, does the program detect them in a specific order, such as left to right? Can we allow the user to select which face they want to use?

[w4hns1nn]

Hi cagrisungur,

thank you for contacting us.
With the current implementation this is an expected behaviour.

Multi-Face Embeddings

If you add an "Face" to be used later as an embedding, we want to make sure, that only one face is in the image.
The reason for it is, that the saving of "multiple" faces in a embedding caused some unexpected issues, when the multi-face embedding is loaded again.

If you want to switch multiple faces at one, you could use the regular swap function. It should do the trick.

Another option would be, to implement saving a MultiFace embedding correctly so that it can be well stored, loaded and used.

Choose one person

To answer your question, this is indeed simple to implement.
I'll update the function, to accept as face_name a list of Strings. If you then use an image with multiple persons just pass a list with the face names and an embedding for each of them will be created from left to right. If there's less "names" then detected faces it will store as many as there are passed names.

If you give me a quick feedback if this is what you'd expect ill implement it within the next hour

[cagrisungur]

Thanks for the quick reply.

if is_embed and is_embed.lower() == 'true':
            embedding = f2f.add_face(face_name="source_face", image=source_img, save=False)
            result_img = f2f.swap(target_img, embedding[0], enhance_face_model='gpen_bfr_2048')
        else:
            result_img = f2f.swap_img_to_img(source_img, target_img, enhance_face_model='gpen_bfr_2048')

I'm using like this now, but i've edited _face_embedding.py like this,

# TODO: we can do dynamic face selection which one is actual source face
            if len(detected_faces) > 1:
                raise ValueError(f"Multiple faces detected in the provided image for {face_name}.")

            face_name = encode_path_safe(face_name)
            face = detected_faces[0]

Is something wrong my usage? Is the error occurs on f2f.add_face method?

[w4hns1nn]

No, there's nothing wrong with how you use it. In your case you could also just add an try catch around add_face for now and then go to the else block.

[cagrisungur]

Understood.

It will be very good if you implement list of Strings for multiple faces input. Thats what i expected. I think this will be good feature for multiple face detection.

Thank you very much.

[w4hns1nn]

Implemented and pushed for dev.
Please test ;)

def add_face(
self: Face2Face,
face_name: Union[str, List[str]],
image: Union[np.array, str, ImageFile],
save: bool = False
) -> Union[Tuple[str, FileWriteableFace], dict]:
"""
Add one or multiple reference face(s) to the face swapper. This face(s) can be used for swapping in other images.

    :param face_name: The name for the reference face.
        In case you provide a list of face names, an embedding is stored for each face from left to right in the provided image.

    :param image: The image from which to extract the face(s) (can be a numpy array, file path, or ImageFile).
        If there are multiple faces in the image, an embedding will be created for each name from left to right.
        If you only provide one name, only the first face will be stored.
    :param save:
        If True, the reference face will be saved to the _face_embeddings folder for future use.
        If False, the reference face will only be stored in memory.
    :return: A tuple containing the safely encoded face name and the reference face.
        In case of multiple faces returns as dict {face_name: face_embedding}
    """


    if isinstance(face_name, list) and len(face_name) == 0 or face_name is None:
        raise ValueError("Please provide at least one face name.")

    try:
        image = load_image(image)
        detected_faces = self.detect_faces(image)
    except Exception as e:
        raise ValueError(f"Could not load image or face detection failed. Error: {e}")

    # Deal with errors of too much or too few faces in the reference image
    if not detected_faces:
        raise ValueError(f"No faces detected in the provided image for {face_name}.")

    if not isinstance(face_name, list):
        face_name = [face_name]

    if len(face_name) > len(detected_faces):
        print(f"Not enough faces in the image for all provided face names. "
              f"Only {len(detected_faces)} faces found. Skipping the rest")
        face_name = [face_name[i] for i in range(len(detected_faces))]

    face = None
    for i, name in enumerate(face_name):
        face_name = encode_path_safe(face_name)
        face = detected_faces[i]
        # Store the detected faces in memory
        self._face_embeddings[face_name] = face

        # store the detected faces on disc
        face = FileWriteableFace(face)
        # Save face to virtual file
        if save:
            os.makedirs(EMBEDDINGS_DIR, exist_ok=True)
            filename = os.path.join(EMBEDDINGS_DIR, f"{face_name}.npy")
            if os.path.isfile(filename):
                print(f"Reference face {face_name} already exists. Overwriting.")
            face.to_file(filename)
    if len(face_name) > 1:
        # Return a dict with the face names and embeddings
        return {
            name: self._face_embeddings[name]
            for name in face_name
        }
    else:
        return face_name, face

[cagrisungur]

Implemented and pushed for dev. Please test ;)

def add_face( self: Face2Face, face_name: Union[str, List[str]], image: Union[np.array, str, ImageFile], save: bool = False ) -> Union[Tuple[str, FileWriteableFace], dict]: """ Add one or multiple reference face(s) to the face swapper. This face(s) can be used for swapping in other images.

    :param face_name: The name for the reference face.
        In case you provide a list of face names, an embedding is stored for each face from left to right in the provided image.

    :param image: The image from which to extract the face(s) (can be a numpy array, file path, or ImageFile).
        If there are multiple faces in the image, an embedding will be created for each name from left to right.
        If you only provide one name, only the first face will be stored.
    :param save:
        If True, the reference face will be saved to the _face_embeddings folder for future use.
        If False, the reference face will only be stored in memory.
    :return: A tuple containing the safely encoded face name and the reference face.
        In case of multiple faces returns as dict {face_name: face_embedding}
    """


    if isinstance(face_name, list) and len(face_name) == 0 or face_name is None:
        raise ValueError("Please provide at least one face name.")

    try:
        image = load_image(image)
        detected_faces = self.detect_faces(image)
    except Exception as e:
        raise ValueError(f"Could not load image or face detection failed. Error: {e}")

    # Deal with errors of too much or too few faces in the reference image
    if not detected_faces:
        raise ValueError(f"No faces detected in the provided image for {face_name}.")

    if not isinstance(face_name, list):
        face_name = [face_name]

    if len(face_name) > len(detected_faces):
        print(f"Not enough faces in the image for all provided face names. "
              f"Only {len(detected_faces)} faces found. Skipping the rest")
        face_name = [face_name[i] for i in range(len(detected_faces))]

    face = None
    for i, name in enumerate(face_name):
        face_name = encode_path_safe(face_name)
        face = detected_faces[i]
        # Store the detected faces in memory
        self._face_embeddings[face_name] = face

        # store the detected faces on disc
        face = FileWriteableFace(face)
        # Save face to virtual file
        if save:
            os.makedirs(EMBEDDINGS_DIR, exist_ok=True)
            filename = os.path.join(EMBEDDINGS_DIR, f"{face_name}.npy")
            if os.path.isfile(filename):
                print(f"Reference face {face_name} already exists. Overwriting.")
            face.to_file(filename)
    if len(face_name) > 1:
        # Return a dict with the face names and embeddings
        return {
            name: self._face_embeddings[name]
            for name in face_name
        }
    else:
        return face_name, face

Yes, it is what i expected. It's more dynamic now.

Thanks mate.