Open
Description
System information
- OS Platform and Distribution (e.g., Linux Ubuntu 24.04):
- Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: Samsung Galaxy S23
- TensorFlow.js installed from (npm or script link):
yarn add @tensorflow/tfjs @tensorflow/tfjs-react-native - TensorFlow.js version:
^4.22.0 - React-native version:
^1.0.0
Current behavior
This is the current source-code of the BundleResourceHandler class:
class BundleResourceHandler implements io.IOHandler {
constructor(
protected readonly modelJson: io.ModelJSON,
protected readonly modelWeightsId: Array<string|number>) {
if (modelJson == null || modelWeightsId == null) {
throw new Error(
'Must pass the model json object and the model weights path.');
}
}
/**
* Save model artifacts. This IO handler cannot support writing to the
* packaged bundle at runtime and is exclusively for loading a model
* that is already packages with the app.
*/
async save(): Promise<io.SaveResult> {
throw new Error(
'Bundle resource IO handler does not support saving. ' +
'Consider using asyncStorageIO instead');
}
/**
* Load a model from local storage.
*
* See the documentation to `browserLocalStorage` for details on the saved
* artifacts.
*
* @returns The loaded model (if loading succeeds).
*/
async load(): Promise<io.ModelArtifacts> {
const weightsAssets = this.modelWeightsId.map(id => Asset.fromModule(id));
if (weightsAssets[0].uri.match('^http')) {
// In debug/dev mode RN will serve these assets over HTTP
return this.loadViaHttp(weightsAssets);
} else {
// In release mode the assets will be on the file system.
return this.loadLocalAsset(weightsAssets);
}
}
async loadViaHttp(weightsAssets: Asset[]): Promise<io.ModelArtifacts> {
const modelJson = this.modelJson;
const modelArtifacts: io.ModelArtifacts = Object.assign({}, modelJson);
modelArtifacts.weightSpecs = modelJson.weightsManifest[0].weights;
//@ts-ignore
delete modelArtifacts.weightManifest;
// Load the weights
const weightsDataArray =
await Promise.all(weightsAssets.map(async (weightAsset) => {
const url = weightAsset.uri;
const requestInit: undefined = undefined;
const response = await fetch(url, requestInit, {isBinary: true});
const weightData = await response.arrayBuffer();
return weightData;
}));
modelArtifacts.weightData = io.concatenateArrayBuffers(weightsDataArray);
return modelArtifacts;
}
async loadLocalAsset(weightsAssets: Asset[]): Promise<io.ModelArtifacts> {
// Use a dynamic import here because react-native-fs is not compatible
// with managed expo workflow. However the managed expo workflow should
// never hit this code path.
// tslint:disable-next-line: no-require-imports
const RNFS = require('react-native-fs');
const modelJson = this.modelJson;
const modelArtifacts: io.ModelArtifacts = Object.assign({}, modelJson);
modelArtifacts.weightSpecs = modelJson.weightsManifest[0].weights;
//@ts-ignore
delete modelArtifacts.weightManifest;
// Load the weights
const weightsDataArray =
await Promise.all(weightsAssets.map(async (weightsAsset) => {
let base64Weights: string;
if (Platform.OS === 'android') {
// On android we get a resource id instead of a regular path. We
// need to load the weights from the res/raw folder using this id.
const fileName = `${weightsAsset.uri}.${weightsAsset.type}`;
try {
base64Weights = await RNFS.readFileRes(fileName, 'base64');
} catch (e) {
throw new Error(
`Error reading resource ${fileName}. Make sure the file is
in located in the res/raw folder of the bundle`,
);
}
} else {
try {
base64Weights = await RNFS.readFile(weightsAsset.uri, 'base64');
} catch (e) {
throw new Error(
`Error reading resource ${weightsAsset.uri}.`,
);
}
}
const weightData = util.encodeString(base64Weights, 'base64').buffer;
return weightData;
}));
modelArtifacts.weightData = io.concatenateArrayBuffers(weightsDataArray);
return modelArtifacts;
}
}And my metro config:
const { getDefaultConfig } = require("expo/metro-config");
const config = getDefaultConfig(__dirname);
config.resolver.assetExts = [...config.resolver.assetExts, "bin"];
module.exports = config;In release mode, the loadLocalAsset method loads weights to the model. But whenever expo-updates is installed, it bundles assets differently in the release build. In turn, the weightsAsset.uri is undefined and an error is thrown saying
Error reading resource .bin. Make sure the file is
in located in the res/raw folder of the bundle
This is crashing the app while loading models.
Expected behavior
The app should be able to load the weights in every situation.
Possible Solution
Fallback to the localUri property of the asset when uri is not available
// On android we get a resource id instead of a regular path. We
// need to load the weights from the res/raw folder using this id.
- const fileName = `${weightsAsset.uri}.${weightsAsset.type}`;
+ const fileName = weightAssets.uri ? `${weightAssets.uri}.${weightAssets.type}` : weightAssets.localUri;This helps ensure that the model weights from either the uri or the localUri.
Standalone code to reproduce the issue
Basically any expo app in release mode with expo-updates install.
N.B.: Run apps as a development build and not inside Expo Go.