Difference between revisions of "APEnet+ project"

From APEWiki
Jump to: navigation, search
Line 45: Line 45:
 
<br>
 
<br>
 
<!-- widths=400px heights=300px  -->
 
<!-- widths=400px heights=300px  -->
 
 
''' PERFORMANCE '''
 
''' PERFORMANCE '''
  
  
<gallery widths=400px heights=300px caption="HSG">
+
<gallery widths=640px heights=480px caption="HSG">
 
File:apenet_6link_hsg_mainloop_cudaK_A0F.jpg  | mainloop
 
File:apenet_6link_hsg_mainloop_cudaK_A0F.jpg  | mainloop
 
File:apenet_6link_hsg_mainloop_A0F.jpg‎        | mainloop. Cuda Kernel disabled, memcpy and commmunication only  
 
File:apenet_6link_hsg_mainloop_A0F.jpg‎        | mainloop. Cuda Kernel disabled, memcpy and commmunication only  
Line 55: Line 54:
  
  
<gallery widths=400px heights=300px caption="HSG">
+
<gallery widths=640px heights=480px caption="HSG">
 
File:apenet_6link_hsg_iterations_cudaK_A0F.jpg  | iteration of overrelaxations
 
File:apenet_6link_hsg_iterations_cudaK_A0F.jpg  | iteration of overrelaxations
 
File:apenet_6link_hsg_iterations_A0F.jpg‎        | iteration of overrelaxations. Cuda Kernel disabled, memcpy and commmunication only
 
File:apenet_6link_hsg_iterations_A0F.jpg‎        | iteration of overrelaxations. Cuda Kernel disabled, memcpy and commmunication only
Line 61: Line 60:
  
  
<gallery widths=400px heights=300px caption="HSG">
+
<gallery widths=640px heights=480px caption="HSG">
 
File:apenet_6link_hsg_spin_cudaK_A0F.jpg  | spin
 
File:apenet_6link_hsg_spin_cudaK_A0F.jpg  | spin
 
File:apenet_6link_hsg_spin_A0F.jpg‎        | spin. Cuda Kernel disabled, memcpy and commmunication only
 
File:apenet_6link_hsg_spin_A0F.jpg‎        | spin. Cuda Kernel disabled, memcpy and commmunication only
Line 67: Line 66:
  
  
<gallery widths=400px heights=300px caption="HSG">
+
<gallery widths=640px heights=480px caption="HSG">
 
File:apenet_6link_hsg_energy_cudaK_A0F.jpg  | energy
 
File:apenet_6link_hsg_energy_cudaK_A0F.jpg  | energy
 
File:apenet_6link_hsg_energy_A0F.jpg‎ | energy. Cuda Kernel disabled, memcpy and commmunication only
 
File:apenet_6link_hsg_energy_A0F.jpg‎ | energy. Cuda Kernel disabled, memcpy and commmunication only

Revision as of 16:00, 29 August 2012

APEnet+ is the new generation of our 3D network adapters for PC clusters.

Project Background

Many scientific computations need multi-node parallelism for matching up both space (memory) and time (speed) ever-increasing requirements. The use of GPUs as accelerators introduces yet another level of complexity for the programmer and may potentially result in large overheads due to bookkeeping of memory buffers. Additionally, top-notch problems may easily employ more than a PetaFlops of sustained computing power, requiring thousands of GPUs orchestrated via some parallel programming model, mainly Message Passing Interface (MPI).


APEnet+ aim and features

The project target is the development of a low latency, high bandwidth direct network, supporting state-of-the-art wire speeds and PCIe X8 gen2 while improving the price/performance ratio on scaling the cluster size. The network interface provides hardware support for the RDMA programming model. A Linux kernel driver, a set of low-level RDMA APIs and an OpenMPI library driver are available; this allows for painless porting of standard applications.

Highlights

  • APEnet+ is a packet-based direct network of point-to-point links with 2D/3D toroidal topology.
  • Packets have a fixed size envelope (header+footer) and are auto-routed to their final destinations according to wormhole dimension-ordered static routing, with dead-lock avoidance.
  • Error detection is implemented via CRC at packet level.
  • Basic RDMA capabilities, e.g. RDMA PUT & SEND as well as address translation of memory registration, are implemented at the firmware level. RDMA GET is under development.
  • Fault-tolerance features (will be added starting from 2012).
  • Direct access to GPU memory using PCI express peer-to-peer (NVidia Fermi GPUs only).


PERFORMANCE






GPU Cluster installation


APEnet+ Public Documentation


Internal links (require login):
APEnet+ HW, APEnet+ SW, APEnet+ specification, Next Deadlines For Pubblication