Gradio digit recogniser app no longer works with sketchpad causing keras input problem

[rishanshah]

I have been trying to follow a tutorial on a gradio digit recognizer app. and I have the following code

import tensorflow as tf
import gradio as gr
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = x_train / 255.0, 
x_test = x_test / 255.0

    model = tf.keras.models.Sequential([
  tf.keras.layers.Flatten(input_shape=(28,28)),
  tf.keras.layers.Dense(128,activation='relu'),
  tf.keras.layers.Dense(10, activation='softmax')
])
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train, validation_data=(x_test, y_test), epochs=6)


def classify(image):
  image = [image]
  prediction = model.predict(image).tolist()[0]
  return {str(i):prediction[i] for i in range(10)}

sketchpad = gr.inputs.Sketchpad()
label = gr.outputs.Label(num_top_classes=3)
interface = gr.Interface(classify, 'sketchpad', label, live=True, capture_session=True,interpretation="default",title = 'Digit Classifier')

Firstly the sketchpad = gr.inputs.Sketchpad() no longer works when the sketchpad variable is put into the interface function - it just appears to be stuck. It works when I put in 'sketchpad' into the interface function. But when I do this I get issues with keras and shape of image/batch with the following error:

WARNING:tensorflow:Model was constructed with shape (None, 28, 28) for input KerasTensor(type_spec=TensorSpec(shape=(None, 28, 28), dtype=tf.float32, name='flatten_6_input'), name='flatten_6_input', description="created by layer 'flatten_6_input'"), but it was called on an input with incompatible shape (None, 28).

I tried to reshape the image that is received from the sketchpad with image = [image] or even image.reshape(0,28,28) but this doesn't work.

I would appreciate any help

Related stackoverflow: https://stackoverflow.com/questions/67089999/gradio-digit-recogniser-app-no-longer-works-with-sketchpad-causing-keras-input-p

[aliabid94]

Hey rishanshah, you have a problem in your code. You create the gr.inputs.Sketchpad object but that's not what you pass as input to the gradio Interface. Instad you pass the string shortcut "sketchpad". The gr.inputs.Sketchpad interface has been deprecated - now the proper way to create a skechpad is by using the gr.inputs.Image component, which the string shortcut "sketchpad" uses anyway. This passes a numpy array of shape (width, height). The model needs an array of shape (1, width, height) for inference. Therefore, use this code:

import numpy as np

def classify(image):
  image = np.expand_dims(image, axis=0)
  prediction = model.predict(image).tolist()[0]
  return {str(i):prediction[i] for i in range(10)}

label = gr.outputs.Label(num_top_classes=3)
interface = gr.Interface(classify, 'sketchpad', label, capture_session=True,interpretation="default",title = 'Digit Classifier')
interface.launch(debug=True)

[abidlabs]

Closing this as this seems to be solved