Running FastPitch inference on Intel® Data Center GPU Flex Series using Intel® Extension for PyTorch*
This document has instructions for running FastPitch inference using Intel® Extension for PyTorch on Intel® Flex Series GPU.
Item | Detail |
---|---|
Host machine | Intel® Data Center GPU Flex Series 170 |
Drivers | GPU-compatible drivers need to be installed: Download Driver |
Software | Docker* |
Script name | Description |
---|---|
run_model.sh |
Inference with FP16 for specified batch size on Flex series 170 |
Note
The script is validated for FP16
precision on Flex Series GPU.
Refer to instructions provided here to download datasets and models. Set the DATASET_DIR
and CKPT_DIR
environment variables to point to the directories of the dataset and model respectively.
docker pull intel/speech-generation:pytorch-flex-gpu-fast-pitch-inference
The FastPitch inference container includes scripts, model and libraries needed to run FP16 inference. To run the quickstart script using this container, you will need to provide the paths for dataset and pre-trained model. You will also need to provide an output directory where log files will be written.
#Optional
export PRECISION=FP16
export BATCH_SIZE=<provide batch size, otherwise (default:8)>
export NUM_ITERATIONS=<provide num_iterations, otherwise (default:100)>
#Required
export DATASET_DIR=<provide path to pre-processed dataset>
export CKPT_DIR=<provide path to pre-trained model>
export PLATFORM=Flex
export MULTI_TILE=False
export OUTPUT_DIR=<provide path to output logs directory>
IMAGE_NAME=intel/speech-generation:pytorch-flex-gpu-fast-pitch-inference
SCRIPT=run_model.sh
DOCKER_ARGS="--rm -it"
docker run \
--device=/dev/dri \
--env BATCH_SIZE=${BATCH_SIZE} \
--env NUM_ITERATIONS=${NUM_ITERATIONS} \
--env PLATFORM=${PLATFORM} \
--env MULTI_TILE=${MULTI_TILE} \
--env CKPT_DIR=${CKPT_DIR} \
--env DATASET_DIR=${DATASET_DIR} \
--env OUTPUT_DIR=${OUTPUT_DIR} \
--env PRECISION=${PRECISION} \
--env http_proxy=${http_proxy} \
--env https_proxy=${https_proxy} \
--env no_proxy=${no_proxy} \
--volume ${CKPT_DIR}:${CKPT_DIR} \
--volume ${DATASET_DIR}:${DATASET_DIR} \
--volume ${OUTPUT_DIR}:${OUTPUT_DIR} \
${DOCKER_ARGS} \
${IMAGE_NAME} \
/bin/bash $SCRIPT
Support for Intel® Extension for PyTorch* is found via the Intel® AI Analytics Toolkit. Additionally, the Intel® Extension for PyTorch* team tracks both bugs and enhancement requests using GitHub issues. Before submitting a suggestion or bug report, please search the GitHub issues to see if your issue has already been reported.
LEGAL NOTICE: By accessing, downloading or using this software and any required dependent software (the “Software Package”), you agree to the terms and conditions of the software license agreements for the Software Package, which may also include notices, disclaimers, or license terms for third party software included with the Software Package. Please refer to the license file for additional details.