Simple Inference Test for TensorFlow.js - A benchmark tool for evaluating TensorFlow.js model performance
sit4tfjs is a comprehensive benchmarking tool designed specifically for TensorFlow.js models. It provides detailed performance analysis, supports multiple input models, dynamic tensor shapes, and offers extensive profiling capabilities for optimizing TensorFlow.js model deployment.
- TensorFlow.js Model Support: Native support for TensorFlow.js graph models (model.json + .bin files)
- Performance Benchmarking: Detailed inference time measurements with statistical analysis
- Multi-Input Models: Full support for models with multiple input tensors
- Dynamic Shape Intelligence: Automatic detection and handling of dynamic tensor dimensions
- Memory Usage Estimation: Built-in model size and memory footprint analysis
- External Data Support: Load real input data from numpy files for realistic benchmarking
- Batch Processing: Configurable batch sizes for throughput optimization
- Profiling Mode: Detailed model architecture and performance profiling
- Shape Override: Manual specification of input tensor shapes for dynamic models
- Statistical Analysis: Comprehensive timing statistics (min, max, average, standard deviation)
- Command-Line Interface: Easy-to-use CLI with extensive options
- Python API: Programmatic access for integration into larger workflows
- Cross-Platform: Works on Linux, macOS, and Windows
- Detailed Logging: Comprehensive output for debugging and optimization
pip install sit4tfjsgit clone https://github.com/PINTO0309/sit4tfjs.git
cd sit4tfjs
pip install -e .Core Requirements:
- Python >= 3.10
- TensorFlow.js >= 4.0.0
- NumPy >= 1.19.0
- Rich >= 10.0.0
- psutil >= 5.8.0
Optional Dependencies:
- TensorFlow >= 2.10.0 (for SavedModel conversion only)
- Node.js >= 16.0.0 (for Node.js runtime)
- @tensorflow/tfjs-node (for Node.js runtime)
Note: sit4tfjs supports multiple execution environments:
- Browser CPU (default): Uses headless browser for TensorFlow.js execution
- Browser WebGL/WebGPU/WASM: GPU-accelerated browser-based inference
- Node.js runtime: Server-side TensorFlow.js execution (requires Node.js)
- SavedModel support: Automatic conversion (requires TensorFlow installation)
- Fallback: Mock inference for testing and analysis
# Basic benchmark test (uses browser CPU by default)
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs
# Custom batch size and iterations
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--batch_size 4 \
--test_loop_count 200
# Enable detailed profiling
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--enable_profiling
# Browser benchmarking with GPU acceleration
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--execution_provider webgl
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--execution_provider webgpu
# WebAssembly backend
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--execution_provider wasm
# Node.js runtime (requires Node.js + @tensorflow/tfjs-node)
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--execution_provider nodejs# Specify fixed input shapes (for dynamic models or multiple inputs)
sit4tfjs \
--input_tfjs_file_path ./model_multi_input_fix_tfjs \
--fixed_shapes "feat:1,112,200,64 pc_dep:1,112,200,3"
# Single input model with fixed shape
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--fixed_shapes "input:1,105,90"
# Use external numpy input files (multiple inputs)
sit4tfjs \
--input_tfjs_file_path ./model_multi_input_fix_tfjs \
--input_numpy_file_paths "feat:feat_data.npy pc_dep:depth_data.npy"
# Single input model with external data
sit4tfjs \
--input_tfjs_file_path ./model_optimized_fix_tfjs \
--input_numpy_file_paths "input:input_data.npy"
# Combined advanced options with multiple inputs
sit4tfjs \
--input_tfjs_file_path ./model_multi_input_fix_tfjs \
--batch_size 8 \
--test_loop_count 500 \
--fixed_shapes "feat:8,112,200,64 pc_dep:8,112,200,3" \
--enable_profiling
# Batch processing with external data
sit4tfjs \
--input_tfjs_file_path ./model_multi_input_fix_tfjs \
--batch_size 4 \
--input_numpy_file_paths "feat:batch_feat.npy pc_dep:batch_depth.npy" \
--test_loop_count 200from sit4tfjs import TFJSBenchmark
# Basic usage - multi-input model
benchmark = TFJSBenchmark(
model_path="./model_multi_input_fix_tfjs",
batch_size=1,
test_loop_count=100,
enable_profiling=True
)
# Run benchmark
results = benchmark.benchmark()
# Access results
print(f"Average inference time: {results['avg_time_ms']:.3f} ms")
print(f"Throughput: {1000/results['avg_time_ms']:.1f} FPS")
# Advanced usage - with custom shapes and external data
benchmark_advanced = TFJSBenchmark(
model_path="./model_multi_input_fix_tfjs",
batch_size=4,
fixed_shapes={
"feat": [4, 112, 200, 64],
"pc_dep": [4, 112, 200, 3]
},
input_numpy_file_paths={
"feat": "batch_feat_data.npy",
"pc_dep": "batch_depth_data.npy"
},
test_loop_count=50,
enable_profiling=True
)
results_advanced = benchmark_advanced.benchmark()import numpy as np
# Create sample input data for multi-input model
feat_data = np.random.randn(1, 112, 200, 64).astype(np.float32)
pc_dep_data = np.random.randn(1, 112, 200, 3).astype(np.float32)
# Save as numpy files
np.save('feat_data.npy', feat_data)
np.save('depth_data.npy', pc_dep_data)
# For single input model
input_data = np.random.randn(1, 105, 90).astype(np.float32)
np.save('input_data.npy', input_data)
# For batch processing
batch_feat = np.random.randn(4, 112, 200, 64).astype(np.float32)
batch_depth = np.random.randn(4, 112, 200, 3).astype(np.float32)
np.save('batch_feat.npy', batch_feat)
np.save('batch_depth.npy', batch_depth)For true TensorFlow.js benchmarking, install Node.js dependencies:
# Install Node.js (if not already installed)
# Ubuntu/Debian:
curl -fsSL https://deb.nodesource.com/setup_18.x | sudo -E bash -
sudo apt-get install -y nodejs
# macOS with Homebrew:
brew install node
# Windows: Download from https://nodejs.org/
# Install TensorFlow.js for Node.js
npm install @tensorflow/tfjs-node
# For GPU support (optional):
npm install @tensorflow/tfjs-node-gpuRuntime Detection:
- Node.js Runtime: If Node.js and @tensorflow/tfjs-node are available, uses Node.js TFJS runtime
- Browser Runtime: For browser backends (cpu, webgl, webgpu, wasm), uses headless browser automation
- Fallback: Falls back to mock inference with correct shapes and timing simulation
- TensorFlow.js Graph Models: model.json + .bin weight files
- Multi-Input Models: Models with multiple named input tensors
- Dynamic Shapes: Models with variable input dimensions (-1 dimensions)
- Mixed Precision: Support for various tensor data types (float32, int32, etc.)
- Automatic Shape Inference: Reads input specifications from model.json
- Multi-Input Support: Full support for models with multiple named inputs (e.g., feat + pc_dep)
- Dynamic Tensor Support: Handles models with variable batch sizes or sequence lengths
- Custom Shape Override: Manual specification for dynamic dimensions using
--fixed_shapes - External Data Loading: Support for numpy (.npy) input files with
--input_numpy_file_paths - Random Data Generation: Automatic generation of realistic test inputs
- Flexible Input Specification: Space-separated format for multiple inputs:
--fixed_shapes "input1:1,224,224,3 input2:1,100"--input_numpy_file_paths "input1:data1.npy input2:data2.npy"
sit4tfjs supports multiple execution providers for comprehensive benchmarking:
| Provider | Environment | Description | Use Case |
|---|---|---|---|
cpu (default) |
Browser | Browser CPU backend | Web deployment, CPU-only environments, default choice |
webgl |
Browser | WebGL-accelerated execution | Web deployment with GPU acceleration |
webgpu |
Browser | WebGPU backend (experimental) | Latest web GPU acceleration |
wasm |
Browser | WebAssembly backend | Cross-platform web deployment |
nodejs |
Node.js | Native TensorFlow.js runtime | Server-side deployment, highest accuracy |
Performance Characteristics:
- webgpu > webgl > wasm > cpu (typical performance order)
- nodejs provides the most accurate server-side performance
- cpu (default) provides good baseline performance for web deployment
- Browser backends simulate real web deployment scenarios
- Computer Vision Models (CNN, ResNet, EfficientNet)
- Natural Language Processing (BERT, Transformer)
- Time Series Analysis
- Multi-Modal Models (Vision + Text)
- Custom Neural Networks
- Inference Time: Per-inference timing in milliseconds
- Statistical Analysis: Min, max, average, and standard deviation
- Throughput: Inferences per second (FPS)
- Memory Usage: Estimated model memory footprint
- Parameter Count: Total model parameters
================================================================================
sit4tfjs - Simple Inference Test for TensorFlow.js
Benchmark tool for TensorFlow.js models
================================================================================
Model: ./model_multi_input_fix_tfjs
Test loops: 100
Batch size: 1
Input tensors:
feat: [1, 112, 200, 64] (DT_FLOAT)
pc_dep: [1, 112, 200, 3] (DT_FLOAT)
Output tensors:
output_0: [1, 112, 200, 3] (DT_FLOAT)
output_2: [1, 112, 200, 1] (DT_FLOAT)
Identity_3:0: [1, 112, 200, 8] (DT_FLOAT)
Identity_1:0: [1, 112, 200, 8] (DT_FLOAT)
Running 100 inference iterations...
Completed 100/100 iterations
============================================================
BENCHMARK RESULTS
============================================================
Average inference time: 45.234 ms
Minimum inference time: 42.156 ms
Maximum inference time: 52.789 ms
Standard deviation: 2.145 ms
============================================================
| Option | Type | Default | Description |
|---|---|---|---|
--input_tfjs_file_path |
string | Required | Path to TensorFlow.js model directory |
--batch_size |
int | 1 | Batch size for inference |
--test_loop_count |
int | 100 | Number of inference iterations |
--fixed_shapes |
string | "" | Fixed input shapes (format: "input1:1,224,224,3") |
--input_numpy_file_paths |
string | "" | External numpy input files (format: "input1:data1.npy") |
--enable_profiling |
flag | False | Enable detailed profiling output |
--info_only |
flag | False | Display model info without benchmarking |
--debug |
flag | False | Enable debug mode with detailed errors |
Model Loading Errors
# Verify model structure
sit4tfjs --input_tfjs_file_path ./model --info_only
# Check file permissions
ls -la ./model/Shape Mismatch Errors
# Override dynamic shapes
sit4tfjs \
--input_tfjs_file_path ./model \
--fixed_shapes "input:1,224,224,3"Memory Issues
# Reduce batch size
sit4tfjs \
--input_tfjs_file_path ./model \
--batch_size 1# Upgrade pip and dependencies
pip install --upgrade pip setuptools wheel
pip install --upgrade tensorflowjs
# Add TensorFlow if needed for SavedModel conversion
pip install sit4tfjs[tensorflow]
# For M1 Macs (if using TensorFlow)
pip install tensorflow-macos- Use smaller batch sizes for memory-constrained environments
- Enable CPU optimizations in TensorFlow
- Consider model quantization for faster inference
- Use profiling mode to identify bottlenecks
# Enable detailed error output
sit4tfjs --input_tfjs_file_path ./model --debug- Operating System: Linux, macOS, Windows
- Python: 3.10+
- Memory: Minimum 4GB RAM (8GB+ recommended for large models)
- Storage: Model size + 1GB for temporary files
# Core dependencies (always required)
tensorflowjs>=4.0.0
numpy>=1.19.0
rich>=10.0.0
psutil>=5.8.0
# Optional dependencies
tensorflow>=2.10.0 # Only for SavedModel conversion
# Basic installation (browser backends only)
pip install sit4tfjs
# With TensorFlow for SavedModel support
pip install sit4tfjs[tensorflow]
# For development
pip install sit4tfjs[dev]# Install development dependencies
pip install -e .[dev]
# Run tests
pytest tests/
# Run with coverage
pytest --cov=sit4tfjs tests/- Fork the repository
- Create a feature branch
- Make your changes
- Add tests for new functionality
- Submit a pull request
MIT License - see LICENSE file for details.
- sit4onnxw: ONNX model benchmarking tool
- TensorFlow.js: Machine learning for JavaScript
- PINTO Model Zoo: Pre-trained model collection