- Obtaining an Account
- Connection on the I-Cluster2
- Storage and file distribution
- General development
- Setting Up your Environment
- Batch Scheduling : OAR

Obtaining an Account

An account on the I-Cluster2 is created for each scientific project validated by the I-Cluster2 committee.
We ask you to fill the register form to introduce yourselves and present the purpose of you activities on the cluster.
You must fill the I-Cluster2 Login Application Form to present your scientific project.

Please provide information as accurate as possible when filling form.

The registration information is kept private and will not be disclosed to third parties, except project name and URL which should be publish on I-Cluster2 web site as project user.

Connection on the I-Cluster2

I-cluster2 is reachable using ssh onto frontal server ita.imag.fr.
From this machine, you although have to connect on nodes by using oar commands to compile or launch jobs on the cluster.

To connect on I-cluster2 :

ssh ita.imag.fr

Storage and file distribution

To copy files to|from the cluster use scp:

scp file ita.imag.fr:destfile

General development

All development work must be done on the cluster nodes not from the frontal node (ita.imag.fr)

Setting Up your Environment

Setting Up The Compiler Environment

Intel® C/C++ Compiler : icc/icpc

If you use a bourne shell (bash, sh), run :

source /usr/local/intel_cc_80/bin/iccvars.sh

If you use a C shell (csh, tcsh) , run :

source /usr/local/intel_cc_80/bin/iccvars.csh

Intel® Fortran Compiler : ifort

If you use a bourne shell (bash, sh), run :

source /usr/local/intel_fc_80/bin/ifortvars.sh

If you use a C shell (csh, tcsh) , run :

source /usr/local/intel_fc_80/bin/ifortvars.csh

Intel® Debugger : idb

If you use a bourne shell (bash, sh), run :

source /usr/local/intel_idb_80/bin/idbvars.sh

If you use a C shell (csh, tcsh) , run :

source /usr/local/intel_idb_80/bin/idbvars.csh

Batch Scheduling : OAR

OAR is a PBS-like system. Thus, it is useful to have some knowledge concerning this kind of cluster tool. As it provides a PBS-compatible software interface, it should not be difficult to work with it. This page gives an overview of the most used tools. For further information, I suggest you read the manual. ;-)

Once you're connected to the front-end server (ita), you can use the following commands.

oarnodes

oarnodes displays the state of the cluster : which nodes are up and ready, which are not available for OAR. It does not require any argument.

oarstat

oarstat gives the list of the OAR running jobs on the whole cluster. No arguments.

oarsub

The most useful command. It is used to submit a job. You may specify the number of nodes to use or the maximum walltime the job should run. OAR will select the node names according to the cluster availability

submission mode :

There are two ways to use oarsub : interactive and passive.

The interactive way allows you to open a shell on a cluster node and type your commands. When you quit this shell, your job finishes and the node is freed for other users. In order to select the interactive mode, use the -I flag in your oarsub command. The following example is roughly equivalent to a rsh to a free node :

oarsub -I

The passive mode is used to schedule batch and time-consuming jobs. To use this mode, you have to provide a program or script name as the last argument of your oarsub command. Example :

oarsub script.sh

Once your job is done, you will find two files per node allocated for your job. They are named after the following scheme:

• OAR.«script_name».«node_number».stdout (for standard output)
• OAR.«script_name».«node_number».stderr (for standard error)

You should read them to get the result of your job.

Warning : the commands and softwares used in the script must be available on each nodes concerned by your job. It could be software in a NFS-exported directory.

additional parameters :

In addition to the submission mode, you can specify several requirements for your job. Some examples :

More than one node (where N is the number of nodes you need) :

oarsub -l nodes=N submission mode

To reserve a specific named node :

oarsub -p "sql expression" submission mode

Examples:

To reserve the ita12 nodes :
oarsub -p "hostname='ita12.imag.fr'" submission mode

To reserve ita10 to ita19 :
oarsub -l nodes=10 -p "hostname like 'ita1_.imag.fr'" submission mode

To reserve 20 nodes on the same switch (fast ethernet) number 2:
oarsub -l nodes=10 -p "switch = '2' " submission mode

With walltime. The job will be killed if not terminated 2 hours later :

oarsub -l walltime=2 submission mode

oardel

oardel deletes a job identified by its ID (oarstat).