image_dataset_from_directory: Input 'filename' of 'ReadFile' Op and ValueError: No images found

[jamesbraza]

System information.

• Have I written custom code (as opposed to using a stock example script provided in Keras): yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): macOS Big Sur, version 11.5.1
• TensorFlow installed from (source or binary): binary
• TensorFlow version (use command below): 2.4.4 and 2.9.1
• Python version: 3.8.13
• Bazel version (if compiling from source): n/a
• GPU model and memory: n/a
• Exact command to reproduce: see below

Describe the problem.

Tensorflow 2.4.4's image_dataset_from_directory will output a raw Exception when a dataset is too small for a single image in a given subset (training or validation).

TypeError: Input 'filename' of 'ReadFile' Op has type float32 that does not match expected type of string.

See TypeError: Input 'filename' of 'ReadFile' Op has type float32 that does not match expected type of string where many people have hit this raw Exception message.

Tensorflow 2.9.1's image_dataset_from_directory will output a different and now incorrect Exception under the same circumstances:

ValueError: No images found in directory base_dir. Allowed formats: ('.bmp', '.gif', '.jpeg', '.jpg', '.png')

This is even worse, as the message is misleading that we're not finding the directory. There are actually images in the directory, there's just not enough to make a dataset given the current validation split + subset.

Describe the current behavior.

Stated above.

Describe the expected behavior.

I expect this to raise an Exception saying "not enough images in the directory" or something more precise and related to the actual issue.

Standalone code to reproduce the issue.

The below code block was run with tensorflow~=2.4, Pillow==9.1.1, and numpy~=1.19 to run.

import os

import numpy as np
import tensorflow as tf
from PIL import Image

base_dir = "base_dir"
label = "label"
image_path = os.path.join(os.getcwd(), base_dir, label, "1.png")

# Make a randomly-generated PNG image
image_size = 128, 128
image_data = np.random.randint(
    low=0, high=256, size=image_size[0] * image_size[1] * 3, dtype=np.uint8
)
image_data = image_data.reshape(*image_size, 3)
try:
    dirname = os.path.split(image_path)[0]
    os.makedirs(dirname)
    print(f"Made directory path: {dirname}.")
except FileExistsError:
    pass  # Already created
Image.fromarray(image_data, "RGB").save(image_path)
print(f"Saved image at path {image_path}.")

# Convert that image to a Dataset
dataset = tf.keras.preprocessing.image_dataset_from_directory(
    base_dir, image_size=image_size, seed=42, validation_split=0.2, subset="training"
)

[sushreebarsa]

@gowthamkpr I was able to replicate the issue on colab, please find the gist here for reference. Thank you!

[gowthamkpr]

@jamesbraza Its clearly mentioned in the document that
Directory where the data is located. If labels is "inferred", it should contain subdirectories, each containing images for a class. Otherwise, the directory structure is ignored.

Can you please explain the usecase where one image is used or the users run into this scenario. Thank!!

[google-ml-butler]

This issue has been automatically marked as stale because it has no recent activity. It will be closed if no further activity occurs. Thank you.

[google-ml-butler]

Closing as stale. Please reopen if you'd like to work on this further.

[google-ml-butler]

Are you satisfied with the resolution of your issue?
Yes
No