Make HPX usable for Xeon Phi

[hkaiser]

In order to be able utilize a Xeon Phi with HPX several things have to be implemented:

• extend mask_type to hold more than 64 bits, which is needed for proper
  affinity control on the 60 Xeon Phi cores (each has 4 hyper threads)
• reliably fix Intel compiler support, this includes pushing Boost changes upstream
• create an Intel compiler build bot
• figure out how to launch an HPX application on the Xeon Phi
• figure out how to communicate with an HPX application running on the Xeon Phi
• HCA capabilities (ibverbs) on Xeon Phi
• Enabling support for more than 64 threads in HPX by default (for Xeon Phi builds)

[sithhell]

For the last item, an ib verbs parcel layer is crucial. Hpx should compile with ICC 13 out of the box. I regularly test this.

[sithhell]

Launching an application on the Phi is as simple as launching a program on any other linux system.

[sithhell]

I'd suggest to change mask_type to boost::dynamic_bitset once and for all to ensure compatibility with future systems.

[hkaiser]

Launching an application on the Phi is as simple as launching a program on any other linux system

Yes, but only if you use the Xeon Phi in a certain way. I should rephrase my question: in what mode should we use the Xeon Phi? Possible modes are (as taken from here):

• Using pragmas to augment existing codes so they offload work from the host processor to the Intel Xeon Phi coprocessors(s)
• Recompiling source code to run directly on coprocessor as a separate many-core Linux SMP compute node
• Accessing the coprocessor as an accelerator through optimized libraries such as the Intel MKL (Math Kernel Library), or by creating special HPX components similar to what we plan to do for GPUs
• Using each coprocessor as a node in an MPI cluster or, alternatively, as a device containing a cluster of MPI nodes (in our case this is probably not what we want :-P )

[hkaiser]

I'd suggest to change mask_type to boost::dynamic_bitset once and for all to ensure compatibility with future systems.

Yes, that's what I had in mind. However we shouldn't do that always, just for systems where we might need more than 64 threads.

[sithhell]

On Sunday, February 24, 2013 08:02:20 Hartmut Kaiser wrote:

Launching an application on the Phi is as simple as launching a program on any other linux system
Yes, but only if you use the Xeon Phi in a certain way. I should rephrase my question: in what mode should we use the Xeon Phi? Possible modes are (as taken from here):

• Using pragmas to augment existing codes so they offload work from the host processor to the Intel Xeon Phi coprocessors(s)
  That is probably the worst way ... And not at all in the spirit of HPX
  • Recompiling source code to run directly on coprocessor as a separate many-core Linux SMP compute node
    That is my favorite option!
• Accessing the coprocessor as an accelerator through optimized libraries such as the Intel MKL (Math Kernel Library), or by creating special HPX components similar to what we plan to do for GPUs
  I wouldn't like to do that.
  • Using each coprocessor as a node in an MPI cluster or, alternatively, as a device containing a cluster of MPI nodes (in our case this is probably not what we want :-P )
    This is similar to option 2.

[sithhell]

Looks like we are almost ready. The only thing missing is how to reliably support the intel builds. We would need to fix upstream boost for that to happen.
Running an HPX application on the Phi is simple. We just compile it as native MIC code and then run it. Communication with other HPX localities is also easy as the MIC supports ethernet bridges (The HCA capabilities need to still be worked on though).

[sithhell]

So, looks like we are maturing here.
The last things to do is to push our changes to boost upstream and fix the communication between the host and mic. It is currently failing with some assertions in the serialization code. The communication item will be fixed once we have a working MPI backend for the parcelport.

[sithhell]

#797 Takes care of the fast communication between MICs and between MIC and host.

[sithhell]

#797 has been closed as such, high speed communication between host and mic is available. Additionally, the serialization problem has been fixed.

[hkaiser]

What's still missing to close this ticket?

[sithhell]

Nothing. everything is working out of the box now.