Skip to content

feat: add isolated GPTQ-to-OpenVINO export environment under envs/openvino - #65

Merged
aki916f merged 2 commits into
FujitsuResearch:develop/v1-4-0from
computermind-corp:feature/openvino-2026.03
Sep 10, 2026
Merged

feat: add isolated GPTQ-to-OpenVINO export environment under envs/openvino#65
aki916f merged 2 commits into
FujitsuResearch:develop/v1-4-0from
computermind-corp:feature/openvino-2026.03

Conversation

@k-arima-3150

Copy link
Copy Markdown
Contributor

Summary

This PR adds an isolated OpenVINO 2026.3.1 export environment under
envs/openvino/. It provides a repeatable workflow for converting OneComp
GPTQ 4-bit checkpoints to OpenVINO IR and running inference with OpenVINO
GenAI on NPU.

  • Isolated environment (envs/openvino/pyproject.toml and
    envs/openvino/uv.lock): provides pinned conversion dependencies in a
    dedicated .venv, independently of the main OneComp environment. It uses
    OpenVINO 2026.3.1 with the matching OpenVINO GenAI and OpenVINO Tokenizers
    2026.3.1.0 releases.
  • Export example (envs/openvino/example_export_openvino.py): exports
    supported text-generation checkpoints while preserving their existing GPTQ
    4-bit weights. It also saves the Hugging Face tokenizer files and the
    OpenVINO tokenizer and detokenizer IR required by OpenVINO GenAI.
  • Compatibility utilities (envs/openvino/openvino_export_utils.py):
    when modules_in_block_to_quantize uses the legacy flat List[str] shape,
    the exporter copies the checkpoint to a temporary directory and normalizes
    the copied configuration to List[List[str]] before Transformers and
    Optimum Intel load it. The source checkpoint is not modified.
  • Documentation (envs/openvino/README.md): describes environment setup,
    checkpoint preparation, configuration of MODEL_PATH and OUT_DIR, export
    execution, NPU inference, and the changes required when adapting the example
    to a VLM.

Notes

  • This environment is intentionally isolated and does not modify the
    repository-root pyproject.toml or uv.lock.
  • Legacy flat checkpoint metadata requires a temporary copy of the checkpoint.
    TMPDIR can be set to an existing directory on a filesystem with sufficient
    capacity when the default temporary directory is too small.

Validation

  • uv lock --check --project envs/openvino
  • uv pip check --python envs/openvino/.venv/bin/python
  • Import of openvino_genai
  • OpenVINO tokenizer and detokenizer conversion
  • Full export and IR parsing with an actual OneComp GPTQ 4-bit checkpoint
  • OpenVINO GenAI inference on NPU using the exported IR

Changes

Environment

  • Add an isolated OpenVINO 2026.3.1 environment with locked conversion dependencies and a
    OneComp GPTQ 4-bit export example for text-generation models. Flat checkpoint
    metadata is normalized in a temporary copy without modifying the source checkpoint.

@aki916f aki916f left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@aki916f
aki916f merged commit a8945b3 into FujitsuResearch:develop/v1-4-0 Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants