Image Classification in MATLAB Using TensorFlow

This example shows how to call a TensorFlow™ model from MATLAB® using co-execution with Python®.

There are different options for accessing deep learning models within MATLAB. These include:

1. Using models created in MATLAB using the Deep Learning Toolbox
2. Converting models from other frameworks into MATLAB
3. Co-executing models from other frameworks with MATLAB

This example provides an overview of how to perform 3. Co-execution. For a further example on co-execution see Hyperparameter Tuning in MATLAB using Experiment Manager & TensorFlow.

For reference, the following table provides a comparison for all options.

What is co-execution?

Co-execution between MATLAB and TensorFlow is when both frameworks are used together, in executing a single application. Co-execution can achieve this by passing data to and from each framework. It requires MATLAB and Tensorflow to be installed on the same machine.

Requirements

The example requires the following software:

• MATLAB R2021b or later
• Tensorflow 2.8
• Python 3.8+

For more information about installing Python, see Install Supported Python Implementation.

Why perform co-execution?

There are many benefits to co-execution. These include:

• You can leverage the features and capabilities of both frameworks together in a single application. For example, MATLAB performs the pre-and post-processing of data when training a model in TensorFlow.
• Teams working in multiple frameworks can build applications together. This reduces the need to:
  • Rely on a single language
  • Manually re-code code from one language to another.
  • Force some team members to work in their non-preferred environment.

What are the limitations of co-execution?

• Performance - Due to data passed between frameworks, there is a latency introduced. In most situations, the latency overhead is negligible, e.g., when performing model training. In other situations, e.g., performing inference at high throughputs, the latency impact is higher.
• Automatic code generation to embedded-devices. MATLAB can automatically generate code for the following: C/C++ using MATLAB Coder, CUDA using GPU Coder and, VHDL using Deep Learning HDL Toolbox, including the deep learning model and pre and post-processing steps. This deployment option is not available when co-execution with TensorFlow.
  • Note: For versions R2022a or newer, MATLAB supports integration with TensorFlow Lite (TFLite) pretrained models. This enables the simulation of TensorFlow Lite models from both MATLAB and Simulink. For code generation, MATLAB generates code for pre and/or post-processing and generates a call to the TensorFlow Lite interpreter on a supported target.
• Datatype conversion and data reformatting - Only select data types in both frameworks are supported for co-execution. This is because not all data types from one framework can be mapped and translated into an equivalent type in the other framework. For a list of data types that can be used, see MATLAB to Python Data Type Mapping,

How can co-execution be performed?

The example shows how MATLAB can co-execute with TensorFlow to classify images using a model from tf.Keras.applications. The following steps outline what is covered in the example:

• Configuring python setup:

The script checkPythonSetup contains commands to help set up the python environment. You don't need to run these commands, unless the default Python configuration causes errors.

For more information on setting up or troubleshooting the Python Environment in MATLAB see Calling Python from MATLAB

• Loading an image:

imgOriginal = imread("./Images/banana.png");
imshow(imgOriginal)

• Importing model directly into MATLAB:

model = py.tensorflow.keras.applications.efficientnet_v2.EfficientNetV2L();  

• Calling a TensorFlow pretrained model for image classification:

% converting input from MATLAB array into Python array.
X = py.numpy.asarray(imgforTF);

% call preprocessing function that is required for the image input in Keras.
X = py.tensorflow.keras.applications.efficientnet_v2.preprocess_input(X); 

% classify image 
Y = model.predict(X, batch_size); 

% label of classification output
label = py.tensorflow.keras.applications.efficientnet_v2.decode_predictions(Y); 

Note that many pretrained models are available for use directly in MATLAB without the need for co-execution.

• Gathering and displaying the classification result in MATLAB:

labelStr = string(label); 
imshow(imgOriginal);
title(labelStr,Interpreter="none");

Comparison of Models accessible in MATLAB

Capability	Models created using the Deep Learning Toolbox	Models Converted from other Frameworks	Co-execution
Integrates with pre and post processing with MATLAB
Requires installation of MATLAB products only
Supports debugging from MATLAB
Offers best inference performance in MATLAB and Simulink
Comes with many MATLAB application examples
Requires no datatype conversion and data reformatting
Provides largest coverage for embedded code generation with MATLAB Coder, GPU Coder & Deep Learning HDL Toolbox
Requires no additional libraries for standalone deployment with MATLAB Compiler
Accesses popular models in a single line of code
Access to models from TensorFlow and PyTorch

Key:

Most support and / or low effort Some support and / or some effort Little to no support and / or high effort

Copyright 2022, The MathWorks, Inc.