Difference between revisions of "GPUcluster"

From APEWiki
Jump to: navigation, search
(Created page with '{| style="border-color: #999999; border-style: solid; border-width: 1px 1px 1px 1px; border-spacing: 0; border-collapse: collapse; margin: 0; padding: 4px; width:100%" |+ style=...')
 
m (→‎Node list)
 
(143 intermediate revisions by 4 users not shown)
Line 1: Line 1:
−
{| style="border-color: #999999; border-style: solid; border-width: 1px 1px 1px 1px; border-spacing: 0; border-collapse: collapse; margin: 0; padding: 4px; width:100%"
+
== Node type ==
  
−
|+ style="border-color: #999999; border-width: 1px 1px 0px 1px; border-style: solid; width:100%; background-color: #D8D8D8"|Monday 17 January 2011: Day 1
+
Most nodes run CentOS 5.x, with openmpi-1.4.
  
−
!style="border-width: 1px 1px 1px 1px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|Host name
+
On COS 5.x, to properly get a functional MPI environment, it is sufficient to create a file named ~/.mpi-selector with the content:<br>
−
!style="border-width: 1px 1px 1px 1px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|Motherboard
+
openmpi-1.4-gcc-x86_64
−
!style="border-width: 0px 1px 1px 0px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|Processor
+
 
−
!style="border-width: 0px 1px 1px 0px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|#cores
+
On COS 6.X, simply add the line below to your .bashrc: <br>
−
!style="border-width: 0px 1px 1px 0px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|Freq
+
module load openmpi-x86_64
−
!style="border-width: 0px 1px 1px 0px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|RAM
+
 
−
!style="border-width: 0px 1px 1px 0px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|GPU
+
== Node list ==
−
!style="border-width: 0px 1px 1px 0px; border-style: solid; border-color: #999999; width:5%; background-color: #FFFFFF"|CUDA vers.
+
{| class="wikitable"
 +
|+ The current node list with their characteristics
 +
!Host name
 +
!MTHBRD
 +
!CPU
 +
!#cores
 +
!Freq
 +
!RAM
 +
!#GPU
 +
!GPU model
 +
!CUDA vers.
 +
!NVIDIA driver vers.
 +
!APEnet card
 +
!Op. Sys.
 +
!IPMI address
 +
!Quartus II versions installed:
 +
!BIOS Version
 +
!IB board
 +
!North Bridge Rev.
 
|-
 
|-
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|agape0
+
! scope="col" |agape0
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X8DTG-D
+
|X8DTG-D
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X5570
+
|X5570
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|16
+
|4
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|2.93Mhz
+
|2.93 GHz
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|12280MB
+
|24576 MB
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|C1060
+
|1
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|3.2
+
|M2070
 +
|4.2
 +
|319.21
 +
|3links
 +
|CentOS 5.8
 +
|10.0.0.200
 +
|11.0 (no SP), 11.1 (SP1), 12.0 (SP2), 12.1 (no SP)
 +
|2.1a (28/10/2011)
 +
! scope="col" |YES
 
|-
 
|-
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|agape1
+
! scope="col" |agape1
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X8DTG-D
+
|X8DTG-D
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X5570
+
|X5570
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|16
+
|4
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|2.93Mhz
+
|2.93 GHz
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|12280MB
+
|49152 MB
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|C1060
+
|1
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|3.2
+
|C1060
 +
|5.0
 +
|310.19
 +
|
 +
|CentOS 6.3
 +
|10.0.0.201
 +
|11.1 (SP2), 12.0 (SP2), 12.1 (no SP), 13.0 (SP1)
 +
|2.1 (15/09/2011)
 +
! scope="col" |NO
 
|-
 
|-
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|agape2
+
! scope="col" |agape2
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X8DTG-D
+
|X8DTG-D
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X5650
+
|X5650
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|24
+
|6
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|2.67Mhz
+
|2.67 GHz
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|32768MB
+
|32768 MB
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|M2050
+
|1
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|3.2
+
|M2050
 +
|5.5
 +
|319.37
 +
|6links (nuova - 001)
 +
|CentOS 5.9
 +
|10.0.0.202
 +
|11.0 (no SP), 11.1 (SP1), 12.0 (no SP), 12.1 (no SP)
 +
|2.1a (28/10/2011)
 +
! scope="col" |YES
 +
!C2
 
|-
 
|-
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|agape3
+
! scope="col" |agape3
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X8DTG-D
+
|X8DTG-D
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|E5620
+
|E5620
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|16
+
|4
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|2.40Mhz
+
|2.40 GHz
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|12280MB
+
|12280 MB
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|M2050
+
|1
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|3.2
+
|M2050
 +
|4.2
 +
|304.60
 +
|6links (nuova - 00?)
 +
|CentOS 5.9
 +
|10.0.0.203
 +
|11.0 (no SP), 11.1 (SP2), 12.0 (no SP), 12.1 (no SP)
 +
|2.0 (03/09/2010)
 +
! scope="col" |YES
 
|-
 
|-
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|agape4
+
! scope="col" |agape4
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X8DTG-D
+
|X8DTG-D
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|E5620
+
|E5620
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|16
+
|4
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|2.40Mhz
+
|2.40 GHz
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|12280MB
+
|12280 MB
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|M2050
+
|1
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|3.2
+
|M2050
 +
|4.2
 +
|304.60
 +
|6links
 +
|CentOS 5.9
 +
|10.0.0.204
 +
|11.0 (no SP), 11.1 (SP2), 12.0 (SP2), 12.1 (no SP)
 +
|2.0 (03/09/2010)
 +
! scope="col" |YES
 
|-
 
|-
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|agape5
+
! scope="col" |agape5
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X8DTG-D
+
|X8DTG-D
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|E5620
+
|E5620
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|16
+
|4
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|2.40Mhz
+
|2.40 GHz
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|12280MB
+
|12280 MB
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|S2050
+
|2
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|3.2
+
|S2050
 +
|4.2
 +
|304.60
 +
|6links
 +
|CentOS 5.9
 +
|10.0.0.205
 +
|11.0 (no SP), 11.1 (SP2), 12.1 (no SP)
 +
|2.0 (03/09/2010)
 +
! scope="col" |YES
 +
|-
 +
! scope="col" |agape6
 +
|X8DTG-D
 +
|E5620
 +
|4
 +
|2.40 GHz
 +
|12280 MB
 +
|2
 +
|S2050
 +
|4.2
 +
|304.60
 +
|6links
 +
|CentOS 5.9
 +
|10.0.0.206
 +
|10.0 (SP1), 11.0 (no SP), 11.1 (no SP), 12.1 (no SP)
 +
|2.0 (03/09/2010)
 +
! scope="col" |YES
 +
|-
 +
! scope="col" |agape7
 +
|X8DTG-D
 +
|X5660
 +
|6
 +
|2.80 GHz
 +
|10240 MB
 +
|1
 +
|M2070
 +
|4.2
 +
|304.60
 +
|6links
 +
|CentOS 5.9
 +
|10.0.0.207
 +
|11.0 (no SP), 11.1 (SP2), 12.1 (no SP)
 +
|2.1a (28/10/2011)
 +
! scope="col" |YES
 +
|-
 +
! scope="col" |agape8
 +
|X8DTG-D
 +
|X5660
 +
|6
 +
|2.80 GHz
 +
|10240 MB
 +
|1
 +
|C1060
 +
|5.5
 +
|319.37
 +
|3links
 +
|CentOS 6.4
 +
|10.0.0.208
 +
|11.1 (SP2), 12.0 (SP2), 12.1 (no SP), 12.1 (no SP)
 +
|2.1a (28/10/2011)
 +
! scope="col" |YES
 +
!C2
 +
|-
 +
! scope="col" |apeiron1
 +
|X8DAH+
 +
|X5650
 +
|6
 +
|2.67 GHz
 +
|24576 MB
 +
|1
 +
|C2050
 +
|4.2
 +
|304.60
 +
|6links (nuova - 002)
 +
|CentOS 5.9
 +
|No IPMI
 +
|11.0 (no SP), 11.1 (SP2), 12.1 (no SP)
 +
|2.0 (01/06/2010)
 +
! scope="col" |NO
 +
|-
 +
! scope="col" |apeiron2
 +
|X8DTG-QF
 +
|X5650
 +
|6
 +
|2.67 GHz
 +
|20480 MB
 +
|1
 +
|C2050
 +
|4.2
 +
|304.60
 +
|6links
 +
|CentOS 5.9
 +
|10.0.0.199
 +
|11.0 (no SP), 12.1 (no SP)
 +
|2.0c (30/12/2011)
 +
! scope="col" |NO
 
|-
 
|-
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|agape6
 
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|X8DTG-D
 
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|E5620
 
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|16
 
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|2.40Mhz
 
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:10%; background-color: #ffffff; text-align:center"|12280MB
 
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:30%; background-color: #ffffff; text-align:center"|S2050
 
−
|style="border-width: 0px 0px 1px 0px; border-style: solid; border-color: #999999; width:20%; background-color: #ffffff; text-align:center"|3.2
 
 
|}
 
|}
 +
 +
== Node topology (X+Y Torus) ==
 +
 +
            ||                ||                ||                ||
 +
            \/                \/                \/                \/
 +
        +----------+      +----------+      +----------+      +----------+   
 +
        |  agape2  |      |  agape3  |      |  agape4  |      |  agape5  |
 +
    ==> |          | <==> |          | <==> |          | <==> |          | <==
 +
        |  0 1 0  |      |  1 1 0  |      |  2 1 0  |      |  3 1 0  |
 +
        +----------+      +----------+      +----------+      +----------+
 +
            /\                /\                /\                /\   
 +
            ||                ||                ||                ||   
 +
            ||                ||                ||                ||   
 +
            \/                \/                \/                \/   
 +
        +----------+      +----------+      +----------+      +----------+ 
 +
        | apeiron1 |      | apeiron2 |      |  agape6  |      |  agape7  |
 +
    ==> |          | <==> |          | <==> |          | <==> |          | <==
 +
        |  0 0 0  |      |  1 0 0  |      |  2 0 0  |      |  3 0 0  |
 +
        +----------+      +----------+      +----------+      +----------+
 +
            /\                /\                /\                /\   
 +
            ||                ||                ||                ||

Latest revision as of 09:24, 2 August 2013

Node type

Most nodes run CentOS 5.x, with openmpi-1.4.

On COS 5.x, to properly get a functional MPI environment, it is sufficient to create a file named ~/.mpi-selector with the content:
openmpi-1.4-gcc-x86_64

On COS 6.X, simply add the line below to your .bashrc:
module load openmpi-x86_64

Node list

The current node list with their characteristics
Host name MTHBRD CPU #cores Freq RAM #GPU GPU model CUDA vers. NVIDIA driver vers. APEnet card Op. Sys. IPMI address Quartus II versions installed: BIOS Version IB board North Bridge Rev.
agape0 X8DTG-D X5570 4 2.93 GHz 24576 MB 1 M2070 4.2 319.21 3links CentOS 5.8 10.0.0.200 11.0 (no SP), 11.1 (SP1), 12.0 (SP2), 12.1 (no SP) 2.1a (28/10/2011) YES
agape1 X8DTG-D X5570 4 2.93 GHz 49152 MB 1 C1060 5.0 310.19 CentOS 6.3 10.0.0.201 11.1 (SP2), 12.0 (SP2), 12.1 (no SP), 13.0 (SP1) 2.1 (15/09/2011) NO
agape2 X8DTG-D X5650 6 2.67 GHz 32768 MB 1 M2050 5.5 319.37 6links (nuova - 001) CentOS 5.9 10.0.0.202 11.0 (no SP), 11.1 (SP1), 12.0 (no SP), 12.1 (no SP) 2.1a (28/10/2011) YES C2
agape3 X8DTG-D E5620 4 2.40 GHz 12280 MB 1 M2050 4.2 304.60 6links (nuova - 00?) CentOS 5.9 10.0.0.203 11.0 (no SP), 11.1 (SP2), 12.0 (no SP), 12.1 (no SP) 2.0 (03/09/2010) YES
agape4 X8DTG-D E5620 4 2.40 GHz 12280 MB 1 M2050 4.2 304.60 6links CentOS 5.9 10.0.0.204 11.0 (no SP), 11.1 (SP2), 12.0 (SP2), 12.1 (no SP) 2.0 (03/09/2010) YES
agape5 X8DTG-D E5620 4 2.40 GHz 12280 MB 2 S2050 4.2 304.60 6links CentOS 5.9 10.0.0.205 11.0 (no SP), 11.1 (SP2), 12.1 (no SP) 2.0 (03/09/2010) YES
agape6 X8DTG-D E5620 4 2.40 GHz 12280 MB 2 S2050 4.2 304.60 6links CentOS 5.9 10.0.0.206 10.0 (SP1), 11.0 (no SP), 11.1 (no SP), 12.1 (no SP) 2.0 (03/09/2010) YES
agape7 X8DTG-D X5660 6 2.80 GHz 10240 MB 1 M2070 4.2 304.60 6links CentOS 5.9 10.0.0.207 11.0 (no SP), 11.1 (SP2), 12.1 (no SP) 2.1a (28/10/2011) YES
agape8 X8DTG-D X5660 6 2.80 GHz 10240 MB 1 C1060 5.5 319.37 3links CentOS 6.4 10.0.0.208 11.1 (SP2), 12.0 (SP2), 12.1 (no SP), 12.1 (no SP) 2.1a (28/10/2011) YES C2
apeiron1 X8DAH+ X5650 6 2.67 GHz 24576 MB 1 C2050 4.2 304.60 6links (nuova - 002) CentOS 5.9 No IPMI 11.0 (no SP), 11.1 (SP2), 12.1 (no SP) 2.0 (01/06/2010) NO
apeiron2 X8DTG-QF X5650 6 2.67 GHz 20480 MB 1 C2050 4.2 304.60 6links CentOS 5.9 10.0.0.199 11.0 (no SP), 12.1 (no SP) 2.0c (30/12/2011) NO

Node topology (X+Y Torus)

            ||                ||                ||                ||
            \/                \/                \/                \/ 
       +----------+      +----------+      +----------+      +----------+     
       |  agape2  |      |  agape3  |      |  agape4  |      |  agape5  |
   ==> |          | <==> |          | <==> |          | <==> |          | <==
       |  0 1 0   |      |  1 1 0   |      |  2 1 0   |      |  3 1 0   | 
       +----------+      +----------+      +----------+      +----------+
            /\                /\                /\                /\    
            ||                ||                ||                ||    
            ||                ||                ||                ||     
            \/                \/                \/                \/    
       +----------+      +----------+      +----------+      +----------+  
       | apeiron1 |      | apeiron2 |      |  agape6  |      |  agape7  |
   ==> |          | <==> |          | <==> |          | <==> |          | <==
       |  0 0 0   |      |  1 0 0   |      |  2 0 0   |      |  3 0 0   |
       +----------+      +----------+      +----------+      +----------+
            /\                /\                /\                /\    
            ||                ||                ||                ||
Retrieved from "https://apegate.roma1.infn.it/mediawiki/index.php?title=GPUcluster&oldid=3524"