-
Notifications
You must be signed in to change notification settings - Fork 268
/
Copy pathindex.html
131 lines (127 loc) · 12.8 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
<meta charset="utf-8" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />
<meta content="This website introduces Intel® Extension for PyTorch*" name="description" />
<meta content="Intel optimization, PyTorch, Intel® Extension for PyTorch*, GPU, discrete GPU, Intel discrete GPU" name="keywords" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Welcome to Intel® Extension for PyTorch* Documentation!</title>
<link rel="stylesheet" href="_css/theme.css" type="text/css" />
<link rel="stylesheet" href="_css/custom.css" type="text/css" />
<script src="_scripts/jquery.js"></script>
<script src="_scripts/theme.js"></script>
<script src="_scripts/actions.js"></script>
<script src="_scripts/load_navi.js"></script>
<script type="text/javascript">
// Configure TMS settings
window.wapProfile = 'profile-microsite'; // This is mapped by WAP authorize value
window.wapLocalCode = 'us-en'; // Dynamically set per localized site, see mapping table for values
window.wapSection = "intel-extension-for-pytorch"; // WAP team will give you a unique section for your site
window.wapEnv = 'prod'; // environment to be use in Adobe Tags.
var wapSinglePage = false; // Include this variable only if your site is a single page application, such as one developed with the React framework
// Load TMS
(() => {
let url = 'https://www.intel.com/content/dam/www/global/wap/main/wap-microsite.js';
let po = document.createElement('script'); po.type = 'text/javascript'; po.async = true; po.src = url;
let s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
}) ();
</script>
</head>
<body class="wy-body-for-nav">
<div class="wy-grid-for-nav">
<nav data-toggle="wy-nav-shift" class="wy-nav-side">
<div class="wy-side-scroll" id="nav-side">
</div>
</nav>
<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
<nav class="wy-nav-top" aria-label="Mobile navigation menu" >
<i data-toggle="wy-nav-top" class="fa fa-bars"></i>
<a href="#">intel_extension_for_pytorch</a>
</nav>
<div class="wy-nav-content">
<div class="rst-content">
<div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
<div itemprop="articleBody">
<section id="welcome-to-intel-extension-for-pytorch-documentation">
<div class="menu">
<div class="menu-element" id="menu-introduction">Introduction</div>
<div class="menu-element" id="menu-installation">Installation</div>
</div>
<h1>Welcome to Intel® Extension for PyTorch* Documentation!<a class="headerlink" href="#welcome-to-intel-extension-for-pytorch-documentation" title="Permalink to this heading"></a></h1>
<div id="div-introduction">
<p>Intel® Extension for PyTorch* extends PyTorch* with the latest performance optimizations for Intel hardware.
Optimizations take advantage of Intel® Advanced Vector Extensions 512 (Intel® AVX-512) Vector Neural Network Instructions (VNNI) and Intel® Advanced Matrix Extensions (Intel® AMX) on Intel CPUs as well as Intel X<sup>e</sup>Matrix Extensions (XMX) AI engines on Intel discrete GPUs.
Moreover, Intel® Extension for PyTorch* provides easy GPU acceleration for Intel discrete GPUs through the PyTorch* <code class="docutils literal notranslate"><span class="pre">xpu</span></code> device.</p>
<p>The extension can be loaded as a Python module for Python programs or linked as a C++ library for C++ programs. In Python scripts, users can enable it dynamically by importing <code class="docutils literal notranslate"><span class="pre">intel_extension_for_pytorch</span></code>.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<ul class="simple">
<li><p>GPU features are not included in CPU-only packages.</p></li>
<li><p>Optimizations for CPU-only may have a newer code base due to different development schedules.</p></li>
</ul>
</div>
<p>In the current technological landscape, Generative AI (GenAI) workloads and models have gained widespread attention and popularity. Large Language Models (LLMs) have emerged as the dominant models driving these GenAI applications. Starting from 2.1.0, specific optimizations for certain LLM models are introduced in the Intel® Extension for PyTorch*.</p>
<p>Intel® Extension for PyTorch* has been released as an open–source project at <a class="reference external" href="https://github.com/intel/intel-extension-for-pytorch">Github</a>. You can find the source code and instructions on how to get started at:</p>
<ul class="simple">
<li><p><strong>CPU</strong>: <a class="reference external" href="https://github.com/intel/intel-extension-for-pytorch/tree/main">CPU main branch</a> | <a class="reference external" href="./cpu/latest/tutorials/getting_started.html">Get Started</a></p></li>
<li><p><strong>XPU</strong>: <a class="reference external" href="https://github.com/intel/intel-extension-for-pytorch/tree/xpu-main">XPU main branch</a> | <a class="reference external" href="./xpu/latest/tutorials/getting_started.html">Get Started</a></p></li>
</ul>
<section id="architecture">
<h2>Architecture<a class="headerlink" href="#architecture" title="Permalink to this heading"></a></h2>
<p>Intel® Extension for PyTorch* is structured as shown in the following figure:</p>
<figure class="align-center">
<a class="reference internal image-reference" href="_images/intel_extension_for_pytorch_structure.png"><img alt="Architecture of Intel® Extension for PyTorch*" src="_images/intel_extension_for_pytorch_structure.png" style="width: 800px;" /></a>
</figure>
<ul class="simple">
<li><p><strong>Eager Mode</strong>: In the eager mode, the PyTorch frontend is extended with custom Python modules (such as fusion modules), optimal optimizers, and INT8 quantization APIs. Further performance improvement is achieved by converting eager-mode models into graph mode using extended graph fusion passes.</p></li>
<li><p><strong>Graph Mode</strong>: In the graph mode, fusions reduce operator/kernel invocation overhead, resulting in improved performance. Compared to the eager mode, the graph mode in PyTorch* normally yields better performance from the optimization techniques like operation fusion. Intel® Entension for PyTorch* amplifies them with more comprehensive graph optimizations. Both PyTorch <code class="docutils literal notranslate"><span class="pre">Torchscript</span></code> and <code class="docutils literal notranslate"><span class="pre">TorchDynamo</span></code> graph modes are supported. With <code class="docutils literal notranslate"><span class="pre">Torchscript</span></code>, we recommend using <code class="docutils literal notranslate"><span class="pre">torch.jit.trace()</span></code> as your preferred option, as it generally supports a wider range of workloads compared to <code class="docutils literal notranslate"><span class="pre">torch.jit.script()</span></code>. With <code class="docutils literal notranslate"><span class="pre">TorchDynamo</span></code>, ipex backend is available to provide good performances.</p></li>
<li><p><strong>CPU Optimization</strong>: On CPU, Intel® Extension for PyTorch* automatically dispatches operators to underlying kernels based on detected ISA. The extension leverages vectorization and matrix acceleration units available on Intel hardware. The runtime extension offers finer-grained thread runtime control and weight sharing for increased efficiency.</p></li>
<li><p><strong>GPU Optimization</strong>: On GPU, optimized operators and kernels are implemented and registered through PyTorch dispatching mechanism. These operators and kernels are accelerated from native vectorization feature and matrix calculation feature of Intel GPU hardware. Intel® Extension for PyTorch* for GPU utilizes the <a class="reference external" href="https://github.com/intel/llvm#oneapi-dpc-compiler">DPC++</a> compiler that supports the latest <a class="reference external" href="https://registry.khronos.org/SYCL/specs/sycl-2020/html/sycl-2020.html">SYCL*</a> standard and also a number of extensions to the SYCL* standard, which can be found in the <a class="reference external" href="https://github.com/intel/llvm/tree/sycl/sycl/doc/extensions">sycl/doc/extensions</a> directory.</p></li>
</ul>
</section>
<section id="support">
<h2>Support<a class="headerlink" href="#support" title="Permalink to this heading"></a></h2>
<p>The team tracks bugs and enhancement requests using <a class="reference external" href="https://github.com/intel/intel-extension-for-pytorch/issues/">GitHub issues</a>. Before submitting a suggestion or bug report, search the existing GitHub issues to see if your issue has already been reported.</p>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
<div class="toctree-wrapper compound">
</div>
</section>
</div>
<div id="div-installation">
<div class="row">
<div class="row-element-1" id="col-headings"></div>
<div class="row-element-2" id="col-values"></div>
</div>
<div id="install-instructions"></div>
</div>
</section>
</div>
</div>
<footer>
<hr/>
<div role="contentinfo">
<p>© Copyright Intel(R).</p>
</div>
Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
<a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
provided by <a href="https://readthedocs.org">Read the Docs</a>.
<p></p><div><a href='https://www.intel.com/content/www/us/en/privacy/intel-cookie-notice.html' data-cookie-notice='true'>Cookies</a><a href='https://www.intel.com/content/www/us/en/privacy/intel-privacy-notice.html'>| Privacy</a><a href='/#' data-wap_ref='dns' id='wap_dns'><small>| Your Privacy Choices</small><span style='height:10px;width:28px;display:inline-block;position:relative;'><svg style='position:absolute;width:28px;bottom:-2px;' version='1.1' id='Layer_1' xmlns='http://www.w3.org/2000/svg' xmlns:xlink='http://www.w3.org/1999/xlink' x='0px' y='0px' viewBox='0 0 30 14' xml:space='preserve'><title>California Consumer Privacy Act (CCPA) Opt-Out Icon</title><style type='text/css'> .st0 { fill-rule: evenodd; clip-rule: evenodd; fill: #FFFFFF; } .st1 { fill-rule: evenodd; clip-rule: evenodd; fill: #0066FF; } .st2 { fill: #FFFFFF; } .st3 { fill: #0066FF; } </style><g><g id='final---dec.11-2020_1_'><g id='_x30_208-our-toggle_2_' transform='translate(-1275.000000, -200.000000)'><g id='Final-Copy-2_2_' transform='translate(1275.000000, 200.000000)'><path class='st0' d='M7.4,12.8h6.8l3.1-11.6H7.4C4.2,1.2,1.6,3.8,1.6,7S4.2,12.8,7.4,12.8z'></path></g></g></g><g id='final---dec.11-2020'><g id='_x30_208-our-toggle' transform='translate(-1275.000000, -200.000000)'><g id='Final-Copy-2' transform='translate(1275.000000, 200.000000)'><path class='st1' d='M22.6,0H7.4c-3.9,0-7,3.1-7,7s3.1,7,7,7h15.2c3.9,0,7-3.1,7-7S26.4,0,22.6,0z M1.6,7c0-3.2,2.6-5.8,5.8-5.8 h9.9l-3.1,11.6H7.4C4.2,12.8,1.6,10.2,1.6,7z'></path><path id='x' class='st2' d='M24.6,4c0.2,0.2,0.2,0.6,0,0.8l0,0L22.5,7l2.2,2.2c0.2,0.2,0.2,0.6,0,0.8c-0.2,0.2-0.6,0.2-0.8,0 l0,0l-2.2-2.2L19.5,10c-0.2,0.2-0.6,0.2-0.8,0c-0.2-0.2-0.2-0.6,0-0.8l0,0L20.8,7l-2.2-2.2c-0.2-0.2-0.2-0.6,0-0.8 c0.2-0.2,0.6-0.2,0.8,0l0,0l2.2,2.2L23.8,4C24,3.8,24.4,3.8,24.6,4z'></path><path id='y' class='st3' d='M12.7,4.1c0.2,0.2,0.3,0.6,0.1,0.8l0,0L8.6,9.8C8.5,9.9,8.4,10,8.3,10c-0.2,0.1-0.5,0.1-0.7-0.1l0,0 L5.4,7.7c-0.2-0.2-0.2-0.6,0-0.8c0.2-0.2,0.6-0.2,0.8,0l0,0L8,8.6l3.8-4.5C12,3.9,12.4,3.9,12.7,4.1z'></path></g></g></g></g></svg></span></a><a href=https://www.intel.com/content/www/us/en/privacy/privacy-residents-certain-states.html data-wap_ref='nac' id='wap_nac'><small>| Notice at Collection</small></a></div><p></p><div>© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others. No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document, with the sole exception that code included in this document is licensed subject to the Zero-Clause BSD open source license (OBSD), <a href='http://opensource.org/licenses/0BSD'>http://opensource.org/licenses/0BSD</a>.</div>
</footer>
</div>
</div>
</section>
</div>
<script>
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
</script>
</body>
</html>