Skip to content

Trying to load a model with use_external_data_format in v3 #963

@benc-uk

Description

@benc-uk

System Info

Version: transformers@3.0.1
Runtime: Vanilla JS (ESM) in browser, no React or Next.js

Environment/Platform

  • Website/web-app
  • Browser extension
  • Server-side (e.g., Node.js, Deno, Bun)
  • Desktop app (e.g., Electron)
  • Other (e.g., VSCode extension)

Description

Trying to create a pipeline with the Phi-3 models
microsoft/Phi-3-mini-4k-instruct-onnx-web or Xenova/Phi-3-mini-4k-instruct which both have external data files, results in a error message that the external data file is not available (which is incorrect, the file exists!),
I am setting use_external_data_format: true and dtype: "q4f16" (or q4 in the case of the Xenova version of the model)

I don't believe this is a dupe of this issue, as I'm not using Node.js

image

Error: Can't create a session. ERROR_CODE: 1, 
ERROR_MESSAGE: Deserialize tensor model.layers.27.mlp.gate_proj.MatMul.weight_Q4 failed.Failed to load external data file ""model_q4f16.onnx_data"", error: Module.MountedFiles is not available.

Reproduction

Try to create a pipeline with use_external_data_format: true e.g.

import { env, pipeline } from 'https://cdn.jsdelivr.net/npm/@huggingface/transformers@3.0.1'

pipeline = await pipeline('text-generation', 'microsoft/Phi-3-mini-4k-instruct-onnx-web', {
  device: 'webgpu',
  use_external_data_format: true,
  dtype: 'q4f16',
})

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions