Error while converting the model: attempted relative import beyond top-level package

[mram0509]

Trying to quantize distilbert-base-uncased-distilled-squad model using the conversion tool in Transformers library from HuggingFace.

python convert_graph_to_onnx.py --framework pt --opset 13 --pipeline question-answering --quantize --model distilbert-base-uncased-distilled-squad distilbert-base-uncased-distilled-squad_quant.onnx

The script imports the following:
import onnx
from onnxruntime.quantization import QuantizationMode, quantize

and then loads the model.

Getting this error at this point: Error while converting the model: attempted relative import beyond top-level package

Urgency
If there are particular important use cases blocked by this or strict project-related timelines, please share more information and dates. If there are no hard deadlines, please specify none.

System information

• Windows 10
• ONNX Runtime installed: binary
• ONNX Runtime version:v1.8.1
• Python version:Python 3.9.6
• Visual Studio version (if applicable): NA
• GCC/Compiler version (if compiling from source): NA
• CUDA/cuDNN version: NA
• GPU model and memory: NA

Expected behavior
Produce a quantized model

[yufenglee]

@CACUser , I tried running following code with ORT 1.8.1 and it succeeds. Is this error caused by OnnxRuntime or transformer?
from onnxruntime.quantization import QuantizationMode, quantize

[mram0509]

It is from Transformer. I just tried with ORT 1.8.1 runtime as well and it succeeds. So I have a quantized model now. It succeeded with opset 11 and 12, but fails with opset 13. I am running distilbert-base-uncased-distilled-squad model for inference using ortruntime-web (WASM) and in general the quantized model in 2x slower than the original model. Could you help me understand why? thanks.

edit: originally converted form a pt model

[yufenglee]

@CACUser , we don't have quantization optimization for WASM yet. For the failure with opset 13, I want to confirm if it fails with ORT 1.8.1.

[mram0509]

Yes - I believe it is 1.8.1
onnx 1.10.1
onnxruntime 1.8.1
onnxruntime-tools 1.7.0
tf2onnx 1.10.0

When converting a pt model to onnx using opset 13, I get the following error:
Error while converting the model: 'NoneType' object is not subscriptable
(Occurred when translating split).

Works with opset 12

[stale]

This issue has been automatically marked as stale due to inactivity and will be closed in 7 days if no further activity occurs. If further support is needed, please provide an update and/or more details.