Add xpu device support. by feng-intel · Pull Request #1 · oneapi-src/text-data-generation

feng-intel · 2023-03-22T05:24:25Z

Fix ipex inference bug: choose first tensor of model() output as well as onnx inferenceSession
Add xpu device support: model and data tensor to xpu
Add --device argument as the below. The default is cpu $ python src/generate_text.py --model_config configs/config_finetuned_intel.yml --device cpu/xpu

yinghu5

extend the case to xpu support.

devpramod-intel · 2023-03-23T21:13:56Z

Hi @feng-intel

env/intel/text-intel-torch.yml needs to be updated to have ipex xpu package
The readme must have '--device cpu' with a note that mentions --device xpu runs the code on intel's GPUs

devpramod-intel · 2023-03-23T21:14:08Z

Hi @feng-intel

env/intel/text-intel-torch.yml needs to be updated to have ipex xpu package
The readme must have '--device cpu' with a note that mentions --device xpu runs the code on intel's GPUs

feng-intel · 2023-03-24T08:48:34Z

env/intel/text-intel-torch.yml needs to be updated to have ipex xpu package
Hi @devpramod-intel
"intel_extension_for_pytorch==1.13.100" has ipex xpu yet I think.

devpramod-intel · 2023-03-24T18:29:38Z

@feng-intel Could you please add a new config file for xpu as we don't want to change the previously working config file.

devpramod-intel · 2023-03-24T20:49:10Z

env/intel/text-intel-torch.yml needs to be updated to have ipex xpu package
Hi @devpramod-intel
"intel_extension_for_pytorch==1.13.100" has ipex xpu yet I think.
@feng-intel I believe we need to install intel_extension_for_pytorch==1.13.10+xpu

devpramod-intel · 2023-03-31T15:49:03Z

@feng-intel the new xpu env creation file has an issue
`
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:

pytorch::pytorch=1.13.0a0
`

devpramod-intel · 2023-04-03T20:45:57Z

                model(
                    input_ids=all_token_ids,
-                    attention_mask=all_attention_masks)[:, -1, :], dim=1)
+                    attention_mask=all_attention_masks)[0][:, -1, :], dim=1)


@feng-intel please make the 'if-else' change as we discussed

devpramod-intel · 2023-04-03T21:04:32Z

The script "training_args.py" is missing "import intel_extension_for_pytorch as ipex",
This leads to an error as follows:
AttributeError: module 'torch' has no attribute 'xpu'
Please also include this change in the patch file

devpramod-intel · 2023-04-24T21:58:12Z


 ### Optimized Solution Setup

 Follow the below conda installation commands to setup the Intel® oneAPI optimized PyTorch environment for model training and text generation. 


Suggested change

Follow the below conda installation commands to setup the Intel® oneAPI optimized PyTorch environment for model training and text generation.

Follow the below conda installation commands to setup the Intel® oneAPI optimized PyTorch environment for model training and text generation. Please choose `env/intel/text-intel-torch-xpu.yml` if you have Intel GPU available.

devpramod-intel · 2023-04-24T21:59:35Z

+conda env create -f env/intel/text-intel-torch-cpu.yml # For CPU
+conda env create -f env/intel/text-intel-torch-xpu.yml # For XPU


Suggested change

conda env create -f env/intel/text-intel-torch-cpu.yml # For CPU

conda env create -f env/intel/text-intel-torch-xpu.yml # For XPU

conda env create -f env/intel/text-intel-torch-cpu.yml # For CPU

or

conda env create -f env/intel/text-intel-torch-xpu.yml # For XPU

devpramod-intel · 2023-04-24T22:01:32Z

 ```sh
 conda activate text-intel-torch
 ```
+Especially for XPU platform:


Suggested change

Especially for XPU platform:

Please perform this additional installation step only if you are using an Intel GPU, CPU users can skip this step:

devpramod-intel · 2023-04-24T22:31:58Z

+**For intel xpu training**, 'xpu' device is needed to add to 'python/site-packages/transformers'.    
+It can be changed manually as the following:


Suggested change

**For intel xpu training**, 'xpu' device is needed to add to 'python/site-packages/transformers'.

It can be changed manually as the following:

**For Intel GPU training**, device 'xpu' must be added to 'python/site-packages/transformers/training_args.py as shown in the below diff at lines'.

It can be changed manually follows:

devpramod-intel · 2023-04-24T22:32:33Z

+             device = xm.xla_device()
+             self._n_gpu = 0
+```
+Or run this python code only once:


Suggested change

Or run this python code only once:

Or to apply the patch automatically, please run the following Python script only once

devpramod-intel · 2023-04-24T22:33:22Z


+**For intel xpu training**, 'xpu' device is needed to add to 'python/site-packages/transformers'.    
+It can be changed manually as the following:
+```python


Please update the diff with the latest one (one that also has "import intel_extension_for_pytorch as ipex")

devpramod-intel · 2023-04-24T22:38:18Z

                        type=int,
                        default=10
                        )
+    parser.add_argument('--device',


please add a choice here between cpu and xpu

don't get it.

devpramod-intel · 2023-04-24T22:45:08Z

+$ cd src
+$ python ./apply_xpu_patch.py


Suggested change

$ cd src

$ python ./apply_xpu_patch.py

$ cd src

$ python ./apply_xpu_patch.py

and then run the fine tuning step as follows:

@feng-intel please add finetuning step here same as above

devpramod-intel · 2023-04-25T17:32:49Z

hi @feng-intel here's the final change
Could you please also add the following in two places lines 342 and 382

To run inference on an Intel GPU, please run the following
python src/generate_text.py --model_config configs/config_finetuned_inc.yml --device xpu

1. Fix ipex inference bug: choose first tensor of model() output as well as onnx inferenceSession 2. Add xpu device support: model and data tensor to xpu 3. Add --device argument as the below. The default is cpu $ python src/generate_text.py --model_config configs/config_finetuned_intel.yml --device cpu/xpu

* Remove stock env. * Remove text-intel-torch.yml * Add intel_env.yml * Update quantize_inc_gpt2.py -Fix broken instructions. * Update generate_text.py -Add xpu compatibility. * Add .gitignore * Remove inference-transformers.png * Remove E2E_stock-transformers.png * Remove intel flag from finetune_model.py * Add kaggle to intel_env.yml * Update README.md - Remove stock references - Correct style - Redistribute information - Add Introduction - Add Solution Technical Overview - Add Solution Technical Details - Add Validated Hardware Details - Add How it Works - Add Get Started - Add Supported Runtime Environment - Add Summary and Next Steps - Add Appendix * Remove README.md from data directory * Update SECURITY.md file * Update intel_env.yml file - Add gperftools to dependencies. * Remove config_finetuned.yml * Move prompt.csv to config dir * Add gpt_generate_text.py file * Add files to patch transformers to use xpu. * Added logger to gptj_generate_text.py * Update README.md - Correct styles - Fix typos - Add sections - Format commands * Fix bfloat16 RuntimeError * Update intel_env.yml dependencies * Update transformers_xpu.patch * Updated license year to 2024 * Remove xpu dependency from intel_env.yml * Create intel_env_xpu.yml - Differs with intel_env.yml by including a intel-extension-for-pytorch version capable of using XPU. * Add instructions to use XPU to README.md * Make typo and corrections to instructions for README.md * Add blank space line at EOF * Remove inconsistent sentence from README.md

yinghu5 approved these changes Mar 23, 2023

View reviewed changes

feng-intel force-pushed the main branch 3 times, most recently from b1621f8 to bd20c28 Compare March 27, 2023 02:24

devpramod-intel reviewed Apr 3, 2023

View reviewed changes

feng-intel force-pushed the main branch from 53f2a31 to c9870ce Compare April 10, 2023 01:26

feng-intel force-pushed the main branch from c9870ce to f811ecb Compare April 23, 2023 03:32

devpramod-intel reviewed Apr 24, 2023

View reviewed changes

feng-intel force-pushed the main branch from f811ecb to fd65ef7 Compare June 6, 2023 04:05

aagalleg deleted the branch oneapi-src:main February 8, 2024 21:49

aagalleg closed this Feb 8, 2024


		### Optimized Solution Setup

		Follow the below conda installation commands to setup the Intel® oneAPI optimized PyTorch environment for model training and text generation.

		conda env create -f env/intel/text-intel-torch-cpu.yml # For CPU
		conda env create -f env/intel/text-intel-torch-xpu.yml # For XPU

	Especially for XPU platform:
	Please perform this additional installation step only if you are using an Intel GPU, CPU users can skip this step:

		For intel xpu training, 'xpu' device is needed to add to 'python/site-packages/transformers'.
		It can be changed manually as the following:

	Or run this python code only once:
	Or to apply the patch automatically, please run the following Python script only once

Conversation

feng-intel commented Mar 22, 2023

Uh oh!

yinghu5 left a comment

Choose a reason for hiding this comment

Uh oh!

devpramod-intel commented Mar 23, 2023

Uh oh!

devpramod-intel commented Mar 23, 2023

Uh oh!

feng-intel commented Mar 24, 2023

Uh oh!

devpramod-intel commented Mar 24, 2023

Uh oh!

devpramod-intel commented Mar 24, 2023

Uh oh!

devpramod-intel commented Mar 31, 2023

Uh oh!

devpramod-intel Apr 3, 2023

Choose a reason for hiding this comment

Uh oh!

devpramod-intel commented Apr 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

devpramod-intel Apr 24, 2023

Choose a reason for hiding this comment

Uh oh!

devpramod-intel Apr 24, 2023

Choose a reason for hiding this comment

Uh oh!

devpramod-intel Apr 24, 2023

Choose a reason for hiding this comment

Uh oh!

devpramod-intel Apr 24, 2023

Choose a reason for hiding this comment

Uh oh!

devpramod-intel Apr 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devpramod-intel Apr 24, 2023

Choose a reason for hiding this comment

Uh oh!

devpramod-intel Apr 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

feng-intel Jun 6, 2023

Choose a reason for hiding this comment

Uh oh!

devpramod-intel Apr 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devpramod-intel commented Apr 25, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

devpramod-intel commented Apr 3, 2023 •

edited

Loading

devpramod-intel Apr 24, 2023 •

edited

Loading

devpramod-intel Apr 24, 2023 •

edited

Loading

devpramod-intel Apr 24, 2023 •

edited

Loading