{"id":652,"date":"2019-02-22T10:15:36","date_gmt":"2019-02-22T10:15:36","guid":{"rendered":"https:\/\/wordpress.ape\/?page_id=652"},"modified":"2019-02-25T10:07:27","modified_gmt":"2019-02-25T10:07:27","slug":"apenet","status":"publish","type":"page","link":"https:\/\/apegate.roma1.infn.it\/?page_id=652","title":{"rendered":"APEnet"},"content":{"rendered":"\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t

Leveraging the acquired know-how in\u00a0<\/span>networking and re-employing the gained insights, a spin-off project called\u00a0<\/span>APEnet\u00a0<\/span>developed an interconnect board based on FPGA that allows to assemble\u00a0<\/span>a PC cluster a la APE with off-the-shelf components.
<\/span><\/p>

The design of APEnet interconnect is easily portable and can be configured\u00a0<\/span>for different environments: (i) the APEnet was the first point-to-point,\u00a0<\/span>low-latency, high-throughput network interface card for LQCD dedicated\u00a0<\/span>clusters; (ii) the Distributed Network Processor (DNP) was one of the\u00a0<\/span>key elements of RDT (Risc+DSP+DNP) chip for the implementation of a\u00a0<\/span>tiled architecture in the framework of the EU FP6 SHAPES project;\u00a0<\/span>(iii) the APEnet Network Interface Card, based on an Altera Stratix IV FPGA,\u00a0<\/span>was used in a hybrid, GPU-accelerated x86 64 cluster QUonG with a\u00a0<\/span>3D toroidal mesh topology, able to scale up to 10^4 – 10^5 nodes in the framework\u00a0<\/span>of the EU FP7 EURETILE project. APEnet+ was the first device to directly\u00a0<\/span>access the memory of the NVIDIA GPU providing GPUDirect RDMA\u00a0<\/span>capabilities and experiencing a boost in GPU to GPU latency test; (iv) the\u00a0<\/span>APEnet network IP \u2014 i.e. routing logic and link controller \u2014 is responsible\u00a0<\/span>for data transmision at Tier 0\/1\/2 in the framework of H2020 ExaNeSt\u00a0<\/span>project<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t

\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t
<\/div>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\n\n\n\n\t\n\n\t\n\t\n\t\n\t\n\t\n\t
<\/td>
APEnet<\/center><\/th>
DNP<\/center><\/th>
APEnet+<\/center><\/th>
APEnet+ v5<\/center><\/th>
ExaNet<\/center><\/th>\n<\/tr>\n<\/thead>\n
Year<\/strong><\/td>
2003<\/td>
2007<\/center><\/td>
2012<\/center><\/td>
2014<\/center><\/td>
2017<\/center><\/td>\n<\/tr>\n
FPGA<\/strong><\/td>
Altera Stratix III<\/td>
ASIC<\/center><\/td>
Altera Stratix IV<\/center><\/td>
Altera Stratix V<\/center><\/td>
Xilinx Ultrascale+<\/center><\/td>\n<\/tr>\n
BUS<\/strong><\/td>
PCI-X<\/td>
AMBA-AHB<\/center><\/td>
PCIe Gen2<\/center><\/td>
PCIe Gen3<\/center><\/td>
AXI<\/center><\/td>\n<\/tr>\n
Computing<\/strong><\/td>
Intel CPU<\/td>
RISC+DNP<\/center><\/td>
NVIDIA GPU<\/center><\/td>
NVIDIA GPU<\/center><\/td>
ARM+FPGA<\/center><\/td>\n<\/tr>\n
Bandwidth<\/strong><\/td>
6.4 Gbps<\/td>
<\/td>
34 Gbps<\/center><\/td>
45 Gbps<\/center><\/td>
32 Gbps<\/center><\/td>\n<\/tr>\n
Latency<\/strong><\/td>
6.5 us<\/td>
<\/td>
6.5 us<\/center><\/td>
5 us<\/center><\/td>
1.1 us<\/center><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t

APEnet Interconnect Architecture based on a layer models<\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\"\"\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t
\n\t\t\t\t\t\t\t
\n\t\t\t
\n\t\t\t\t\t\t\t\t\t
APEphy<\/div>\n\t\t\t\t\t\t\t\t\t
APElink<\/div>\n\t\t\t\t\t\t\t\t\t
APErouter<\/div>\n\t\t\t\t\t\t\t\t\t
APEni<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t\t\t
\n\t\t\t\t\t\t\t\t\t
APEphy<\/div>\n\t\t\t\t\t