Skip to content

Commit

Permalink
Added cuda-aware-mpi-examples.
Browse files Browse the repository at this point in the history
  • Loading branch information
@jirikraus
jirikraus committed Mar 7, 2013
1 parent 3069da6 commit aeb0434
Show file tree
Hide file tree
Showing 200 changed files with 11,685 additions and 0 deletions.
228 changes: 228 additions & 0 deletions posts/cuda-aware-mpi-example/doc/html/CUDA__Aware__MPI_8c.html
View file
@@ -0,0 +1,228 @@
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<title>Jacobi: CUDA_Aware_MPI.c File Reference</title>

<link href="tabs.css" rel="stylesheet" type="text/css"/>
<link href="doxygen.css" rel="stylesheet" type="text/css" />

<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
$(document).ready(function() { searchBox.OnSelectItem(0); });
</script>

</head>
<body>
<div id="top"><!-- do not remove this div! -->


<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">


<td style="padding-left: 0.5em;">
<div id="projectname">Jacobi

</div>

</td>



</tr>
</tbody>
</table>
</div>

<!-- Generated by Doxygen 1.7.6.1 -->
<script type="text/javascript">
var searchBox = new SearchBox("searchBox", "search",false,'Search');
</script>
<div id="navrow1" class="tabs">
<ul class="tablist">
<li><a href="index.html"><span>Main&#160;Page</span></a></li>
<li class="current"><a href="files.html"><span>Files</span></a></li>
<li>
<div id="MSearchBox" class="MSearchBoxInactive">
<span class="left">
<img id="MSearchSelect" src="search/mag_sel.png"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
alt=""/>
<input type="text" id="MSearchField" value="Search" accesskey="S"
onfocus="searchBox.OnSearchFieldFocus(true)"
onblur="searchBox.OnSearchFieldFocus(false)"
onkeyup="searchBox.OnSearchFieldChange(event)"/>
</span><span class="right">
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.png" alt=""/></a>
</span>
</div>
</li>
</ul>
</div>
<div id="navrow2" class="tabs2">
<ul class="tablist">
<li><a href="files.html"><span>File&#160;List</span></a></li>
<li><a href="globals.html"><span>File&#160;Members</span></a></li>
</ul>
</div>
</div>
<div class="header">
<div class="summary">
<a href="#func-members">Functions</a> </div>
<div class="headertitle">
<div class="title">CUDA_Aware_MPI.c File Reference</div> </div>
</div><!--header-->
<div class="contents">

<p>The implementation details for the CUDA-aware MPI version.
<a href="#details">More...</a></p>
<div class="textblock"><code>#include &quot;<a class="el" href="Jacobi_8h.html">Jacobi.h</a>&quot;</code><br/>
</div><div class="textblock"><div class="dynheader">
Include dependency graph for CUDA_Aware_MPI.c:</div>
<div class="dyncontent">
<div class="center"><img src="CUDA__Aware__MPI_8c__incl.png" border="0" usemap="#CUDA__Aware__MPI_8c" alt=""/></div>
<map name="CUDA__Aware__MPI_8c" id="CUDA__Aware__MPI_8c">
<area shape="rect" id="node3" href="Jacobi_8h.html" title="The header containing the most relevant functions for the Jacobi solver." alt="" coords="121,83,191,112"/></map>
</div>
</div><table class="memberdecls">
<tr><td colspan="2"><h2><a name="func-members"></a>
Functions</h2></td></tr>
<tr><td class="memItemLeft" align="right" valign="top"><a class="anchor" id="ac7925a5469b04c01679b20577671c975"></a><!-- doxytag: member="CUDA_Aware_MPI.c::SetDeviceBeforeInit" ref="ac7925a5469b04c01679b20577671c975" args="()" -->
void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="CUDA__Aware__MPI_8c.html#ac7925a5469b04c01679b20577671c975">SetDeviceBeforeInit</a> ()</td></tr>
<tr><td class="mdescLeft">&#160;</td><td class="mdescRight">This allows the MPI process to set the CUDA device before the MPI environment is initialized For the CUDA-aware MPI version, the is the only place where the device gets set. In order to do this, we rely on the node's local rank, as the MPI environment has not been initialized yet. <br/></td></tr>
<tr><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="CUDA__Aware__MPI_8c.html#aa8cb031e8cba7840a96582f530283fd5">SetDeviceAfterInit</a> (int rank)</td></tr>
<tr><td class="mdescLeft">&#160;</td><td class="mdescRight">This allows the MPI process to set the CUDA device after the MPI environment is initialized For the CUDA-aware MPI version, there is nothing to be done here. <a href="#aa8cb031e8cba7840a96582f530283fd5"></a><br/></td></tr>
<tr><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="CUDA__Aware__MPI_8c.html#a1dc0674ebcbca5a781885f2204d78251">ExchangeHalos</a> (MPI_Comm cartComm, real *devSend, real *hostSend, real *hostRecv, real *devRecv, int neighbor, int elemCount)</td></tr>
<tr><td class="mdescLeft">&#160;</td><td class="mdescRight">Exchange halo values between 2 direct neighbors This is the main difference between the normal CUDA &amp; MPI version and the CUDA-aware MPI version. In the former, the exchange first requires a copy from device to host memory, an MPI call using the host buffer and lastly, a copy of the received host buffer back to the device memory. In the latter, the host buffers are completely skipped, as the MPI environment uses the device buffers directly. <a href="#a1dc0674ebcbca5a781885f2204d78251"></a><br/></td></tr>
</table>
<hr/><a name="details" id="details"></a><h2>Detailed Description</h2>
<div class="textblock"><p>The implementation details for the CUDA-aware MPI version. </p>
</div><hr/><h2>Function Documentation</h2>
<a class="anchor" id="a1dc0674ebcbca5a781885f2204d78251"></a><!-- doxytag: member="CUDA_Aware_MPI.c::ExchangeHalos" ref="a1dc0674ebcbca5a781885f2204d78251" args="(MPI_Comm cartComm, real *devSend, real *hostSend, real *hostRecv, real *devRecv, int neighbor, int elemCount)" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname">void <a class="el" href="Jacobi_8h.html#a1dc0674ebcbca5a781885f2204d78251">ExchangeHalos</a> </td>
<td>(</td>
<td class="paramtype">MPI_Comm&#160;</td>
<td class="paramname"><em>cartComm</em>, </td>
</tr>
<tr>
<td class="paramkey"></td>
<td></td>
<td class="paramtype">real *&#160;</td>
<td class="paramname"><em>devSend</em>, </td>
</tr>
<tr>
<td class="paramkey"></td>
<td></td>
<td class="paramtype">real *&#160;</td>
<td class="paramname"><em>hostSend</em>, </td>
</tr>
<tr>
<td class="paramkey"></td>
<td></td>
<td class="paramtype">real *&#160;</td>
<td class="paramname"><em>hostRecv</em>, </td>
</tr>
<tr>
<td class="paramkey"></td>
<td></td>
<td class="paramtype">real *&#160;</td>
<td class="paramname"><em>devRecv</em>, </td>
</tr>
<tr>
<td class="paramkey"></td>
<td></td>
<td class="paramtype">int&#160;</td>
<td class="paramname"><em>neighbor</em>, </td>
</tr>
<tr>
<td class="paramkey"></td>
<td></td>
<td class="paramtype">int&#160;</td>
<td class="paramname"><em>elemCount</em>&#160;</td>
</tr>
<tr>
<td></td>
<td>)</td>
<td></td><td></td>
</tr>
</table>
</div>
<div class="memdoc">

<p>Exchange halo values between 2 direct neighbors This is the main difference between the normal CUDA &amp; MPI version and the CUDA-aware MPI version. In the former, the exchange first requires a copy from device to host memory, an MPI call using the host buffer and lastly, a copy of the received host buffer back to the device memory. In the latter, the host buffers are completely skipped, as the MPI environment uses the device buffers directly. </p>
<dl class="params"><dt><b>Parameters:</b></dt><dd>
<table class="params">
<tr><td class="paramdir">[in]</td><td class="paramname">cartComm</td><td>The carthesian MPI communicator </td></tr>
<tr><td class="paramdir">[in]</td><td class="paramname">devSend</td><td>The device buffer that needs to be sent </td></tr>
<tr><td class="paramdir">[in]</td><td class="paramname">hostSend</td><td>The host buffer where the device buffer is first copied to (not needed here) </td></tr>
<tr><td class="paramdir">[in]</td><td class="paramname">hostRecv</td><td>The host buffer that receives the halo values (not needed here) </td></tr>
<tr><td class="paramdir">[in]</td><td class="paramname">devRecv</td><td>The device buffer that receives the halo buffers directly </td></tr>
<tr><td class="paramdir">[in]</td><td class="paramname">neighbor</td><td>The rank of the neighbor MPI process in the carthesian communicator </td></tr>
<tr><td class="paramdir">[in]</td><td class="paramname">elemCount</td><td>The number of elements to transfer </td></tr>
</table>
</dd>
</dl>

</div>
</div>
<a class="anchor" id="aa8cb031e8cba7840a96582f530283fd5"></a><!-- doxytag: member="CUDA_Aware_MPI.c::SetDeviceAfterInit" ref="aa8cb031e8cba7840a96582f530283fd5" args="(int rank)" -->
<div class="memitem">
<div class="memproto">
<table class="memname">
<tr>
<td class="memname">void <a class="el" href="Jacobi_8h.html#aa8cb031e8cba7840a96582f530283fd5">SetDeviceAfterInit</a> </td>
<td>(</td>
<td class="paramtype">int&#160;</td>
<td class="paramname"><em>rank</em></td><td>)</td>
<td></td>
</tr>
</table>
</div>
<div class="memdoc">

<p>This allows the MPI process to set the CUDA device after the MPI environment is initialized For the CUDA-aware MPI version, there is nothing to be done here. </p>
<dl class="params"><dt><b>Parameters:</b></dt><dd>
<table class="params">
<tr><td class="paramdir">[in]</td><td class="paramname">rank</td><td>The global rank of the calling MPI process </td></tr>
</table>
</dd>
</dl>

</div>
</div>
</div><!-- contents -->
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
<a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(0)"><span class="SelectionMark">&#160;</span>All</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(1)"><span class="SelectionMark">&#160;</span>Files</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(2)"><span class="SelectionMark">&#160;</span>Functions</a><a class="SelectItem" href="javascript:void(0)" onclick="searchBox.OnSelectItem(3)"><span class="SelectionMark">&#160;</span>Defines</a></div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>



<hr class="footer"/><address class="footer"><small>
Generated on Fri Mar 1 2013 15:52:54 for Jacobi by &#160;<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/>
</a> 1.7.6.1
</small></address>

</body>
</html>
3 changes: 3 additions & 0 deletions posts/cuda-aware-mpi-example/doc/html/CUDA__Aware__MPI_8c__incl.map
View file
@@ -0,0 +1,3 @@
<map id="G" name="G">
<area shape="rect" id="node3" href="$Jacobi_8h.html" title="The header containing the most relevant functions for the Jacobi solver." alt="" coords="121,83,191,112"/>
</map>
1 change: 1 addition & 0 deletions posts/cuda-aware-mpi-example/doc/html/CUDA__Aware__MPI_8c__incl.md5
View file
@@ -0,0 +1 @@
dfc471aadc908d746450f4de33fa3187
Binary file added posts/cuda-aware-mpi-example/doc/html/CUDA__Aware__MPI_8c__incl.png
View file
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit aeb0434

Please sign in to comment.