- RSH
...........................
<jvm name="jvm1">
<creation>
<processReference refid="rshProcess"/>
</creation>
</jvm>
.................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="rshProcess">
<rshProcess class="org.objectweb.proactive.core.process.rsh.RSHJVMProcess" hostname="sea.inria.fr">
<processReference refid="jvmProcess"/>
</rshProcess>
</processDefinition>
</processes>
|
|
For the Jvm2
the creation process is rshProcess(still
an abstract name), which is defined in the infrastructure
section. To define this process you have to give the class to
instantiate to create the rsh
process. ProActive
provides org.objectweb.proactive.core.process.rsh.RSHJVMProcess
to create rsh
process. You must give the remote host name to log on with rsh. You can
define as well username="toto"
if you plan to use rsh with -l
option. As said before this rsh
process must
reference a local process, and in the example, it references the
process defined with the id jvmProcess.
It means that once logged on sea.inria.fr with rsh, a local JVM will be
launched, ie a ProActive node will be created on sea.inria.fr thanks to
the process defined by jvmProcess.
- RLOGIN
...........................
<jvm name="jvm1">
<creation>
<processReference refid="rloginProcess"/>
</creation>
</jvm>
.................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="rloginProcess">
<rloginProcess class="org.objectweb.proactive.core.process.rlogin.RLoginProcess" hostname="sea.inria.fr">
<processReference refid="jvmProcess"/>
</rloginProcess>
</processDefinition>
</processes>
|
|
You can use rlogin
in the same way that you would use rsh
- SSH
...........................
<jvm name="jvm1">
<creation>
<processReference refid="sshProcess"/>
</creation>
</jvm>
.................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="sshProcess">
<sshProcess class="org.objectweb.proactive.core.process.ssh.SSHProcess" hostname="sea.inria.fr">
<processReference refid="jvmProcess"/>
</sshProcess>
</processDefinition>
</processes>
|
|
ProActive
provides org.objectweb.proactive.core.process.ssh.SSHProcess
to create ssh
process.
In order to use ssh to log on a remote host, you must perform some
actions. First you need to copy your public key (located in
identity.pub under ~/.ssh on your local machine) in the
authorized_keys(located under ~/.ssh) file of the remote host. Then to
avoid interactivity, you will have to launch on the local host the
ssh-agent command: ssh-agent
$SHELL, this command can be put
in your .xsession file, in order to run it automatically when logging
on your station. Then launching ssh-add
command to add your identity. Running this command will ask you to
enter your passphrase,
it is the one you provided when asking for an ssh key pair.
Note also that if the generated key pair is not encrypted (no
passphrase), you do not need to run neither the ssh-agent, nor the
ssh-add command. Indeed it is sufficient when using non encrypted
private key, to only copy the public key on the remote host (as
mentionned above) in order to get logged automatically on the remote
host.
These steps must be performed before
running any
ProActive application using ssh
protocol. If you are
not familiar with ssh, see openSSH
- Process list
ProActive provides a way to define a list of processes for RSH, SSH, RLOGIN protocols. Using processList or processListbyHost elements avoids having a long deployment file when many machines with similar names are going to be connected with protocols mentionned before. The first example below shows how to use processList tag, the second how to use processListbyHost.
...........................
<jvm name="jvm1">
<creation>
<processReference refid="processlist"/>
</creation>
</jvm>
.................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="processlist">
<processList class="org.objectweb.proactive.core.process.ssh.SSHProcessList" fixedName="node-" list="[0-100;2]^[10,20]" padding="3" domain="sophia.grid5000.fr">
<processReference refid="jvmProcess"/>
</processList>
</processDefinition>
</processes>
|
|
When using processList tag, the class atrricute can take 3 values: org.objectweb.proactive.core.process.ssh.SSHProcessList, org.objectweb.proactive.core.process.rsh.RSHProcessList, org.objectweb.proactive.core.process.rlogin.RLoginProcessList according to the protocol being used is ssh, rsh or rlogin. The fixedName attribute is mandatory and represents the fixed part shared by all machine's names. The list attribute is also mandatory and can take several forms: [m-n] means from m to n with a step 1, [m-n;k] means from m to n with a step k (m, m+k, m+2k, ....), [m-n]^[x,y] means from m to n exluding x and y, [m-n;k]^[x,y] same as before except that the step is k. The padding attribute is optional (default is 1) and represents the number of digit. Finally the domain attribute is mandatory and represents the last part shared by all machine's names. So in the exemple above, a jvm is going to be created using ssh on machines: node000.sophia.grid5000.fr, node002.sophia.grid5000.fr,..., node098.sophia.grid5000.fr, node100.sophia.grid5000.fr (note that step is 2) excluding machines: node010.sophia.grid5000.fr and node020.sophia.grid5000.fr.
...........................
<jvm name="jvm1">
<creation>
<processReference refid="processlist"/>
</creation>
</jvm>
.................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="processlist">
<processListbyHost class="org.objectweb.proactive.core.process.ssh.SSHProcessList" hostlist="crusoe waha nahuel" domain="inria.fr">
<processReference refid="jvmProcess"/>
</processListbyHost>
</processDefinition>
</processes>
|
|
Using processListbyHost element allows to give a hostlist separated with a whitespace. The class attribute is defined as described in the processList tag. The domain attribute is optional since the complete hostname can also be provided in the hostlist attribute. In the example, a jvm is going to be created using ssh on crusoe.inria.fr, waha.inria.fr, nahuel.inria.fr.
- LSF
This protocol is used to create Nodes(JVMs) on a cluster. ProActive
provides org.objectweb.proactive.core.process.lsf.LSFBSubProcess
to create bsub
process.
In this part we assume that you want to submit a job from a machine
which is outside the cluster. As described before, you can combine
protocols. In this case , you will have to define a process to log on
the front-end of the cluster(rlogin
if your machine is on the same LAN than the cluster front-end, else ssh
(Remember that to use ssh
you will have to run some commands as explained above).
<jvm name="Jvm2">
<creation>
<processReference refid="sshProcess"/>
</creation>
</jvm>
...................................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="bsubInriaCluster">
<bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess">
<processReference refid="jvmProcess"/>
<bsubOption>
<hostlist>cluster_machine1 cluster_machine2<hostlist/>
<processor>6</processor>
<scriptPath>
<absolutePath value="/home/ProActive/scripts/cluster/startRuntime.sh"/>
</scriptPath>
</bsubOption>
</bsubProcess>
</processDefinition>
<processDefinition id="sshProcess">
<sshProcess class="org.objectweb.proactive.core.process.ssh.SSHProcess" hostname="sea.inria.fr">
<processReference refid="bsubInriaCluster"/>
</sshProcess>
</processDefinition>
</processes>
|
|
In this example, the JVM called Jvm2
will be created using ssh
to log on the cluster front end. Then a bsub
command will be generated thanks to the process defined by bsubInriaCluster.
This bsub
command will create Nodes on several cluster machines, since bsubInriaCluster
references the jvmProcess
defined process. All tags defined under <bsubOption>
are not mandatory, but they can be very usefull. The
<hostlist> tag defines possible candidates in the job
attribution, if not set the job will be allocated among all cluster's
machines. The <processor> tag defines the number of
processor requested, if not set, one processor is requested. The
<resourceRequirement> tag defines the expected number of
processors per machine. For instance <resourceRequirement
value="span[ptile=2]"/> ensures that 2 processors per machines
will be used, whereas value="span[ptile=1]" forces that LSF allocates
only only one processor per machine. It represents the -R
option of LSF. At last <scriptPath> defines the path on
the cluster front end of the script startRuntime.sh which is necessary
to run ProActive on a cluster. This script is located under
Proactive/scripts/unix/cluster. If not set the default location is set
as ~/Proactive/scripts/unix/cluster.
It is exactly the same with rlogin
instead of ssh.
If you want to submit the job directly from the cluster entry point,
define only the bsubProcess,
like in the above example and skip the ssh definition.
<jvm name="Jvm2">
<creation>
<processReference refid="bsubInriaCluster"/>
</creation>
</jvm>
...................................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="bsubInriaCluster">
<bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess" interactive="true" queue="short">
<processReference refid="jvmProcess"/>
<bsubOption>
<hostlist>cluster_machine1 cluster_machine2<hostlist/>
<processor>6</processor>
<scriptPath>
<absolutePath value="/home/ProActive/scripts/unix/cluster/startRuntime.sh"/>
</scriptPath>
</bsubOption>
</bsubProcess>
</processDefinition>
</processes>
|
|
Note that in the example above two attributes: interactive
and queue
appear. They are optional, and have a default value: respectively false
and normal.
They represent option in the bsub command: interactive mode, and the
name of the queue.
- PBS
This protocol is used to create jobs on cluster managed by PBS or
PBSPro. ProActive provides org.objectweb.proactive.core.process.pbs.PBSBSubProcess
to create pbs
processes. As explained for LSF you can combine protocols in order for
instance to log on the cluster's frontal with ssh, then to create nodes
using PBS, or you can also use only PBS without ssh if you are already
logged on the frontend. Example below shows how to combine an ssh
process to log on the cluster, then a PBS process that reference a
local process in order to create nodes on processors requested by PBS.
<jvm name="Jvm2">
<creation>
<processReference refid="sshProcess"/>
</creation>
</jvm>
...................................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="pbsCluster">
<pbsProcess class="org.objectweb.proactive.core.process.pbs.PBSSubProcess">
<processReference refid="jvmProcess"/>
<pbsOption>
<hostsNumber>4</hostsNumber>
<processorPerNode>1</processorPerNode>
<bookingDuration>00:15:00</bookingDuration>
<outputFile>/home1/rquilici/out.log</outputFile>
<scriptPath>
<absolutePath value="/home/ProActive/scripts/unix/cluster/pbsStartRuntime.sh"/>
</scriptPath>
</pbsOption>
</pbsProcess>
</processDefinition>
<processDefinition id="sshProcess">
<sshProcess class="org.objectweb.proactive.core.process.ssh.SSHProcess" hostname="frontend">
<processReference refid="pbsCluster"/>
</sshProcess>
</processDefinition>
</processes>
|
|
Note that not all options are listed here, and some options mentionned
in the example are optionnal: hostsNumber
represents the number of host requested using pbs(default is 1), processorPerNode
represents the number of processor per hosts requested(1 or 2, default
is 1), bookingDuration
represents the duration of the job(default is 1 minute), outputFile
represents the file where to put the ouput of the job(default is
specified by pbs), scriptPath
represents the location on the frontend_host of the script
pbsStartRuntime.sh(default is
/user.home/ProActive/scripts/unix/cluster/pbsStartRuntime.sh).
- Sun
Grid Engine
This protocol is used to create jobs on cluster managed by Sun Grid
Engine. ProActive provides org.objectweb.proactive.core.process.gridengine.GridEngineSubProcess
to create grid engine
processes. As explained above you can combine protocols in order for
instance to log on the cluster's frontal with ssh, then to create nodes
using SGE, or you can also use only SGE without ssh if you are already
logged on the frontend. Example below shows how to combine an ssh
process to log on the cluster, then a SGE process that reference a
local process in order to create nodes on processors requested by SGE.
<jvm name="Jvm2">
<creation>
<processReference refid="sshProcess"/>
</creation>
</jvm>
...................................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="sgeCluster">
<gridengineProcess class="org.objectweb.proactive.core.process.gridengine.GridEngineSubProcess">
<processReference refid="jvmProcess"/>
<gridEngineOption>
<hostsNumber>4</hostsNumber>
<bookingDuration>00:15:00</bookingDuration>
<scriptPath>
<absolutePath value="/home/ProActive/scripts/unix/cluster/gridEngineStartRuntime.sh"/>
</scriptPath>
</gridEngineOption>
</gridengineProcess>
</processDefinition>
<processDefinition id="sshProcess">
<sshProcess class="org.objectweb.proactive.core.process.ssh.SSHProcess" hostname="frontend">
<processReference refid="sgeCluster"/>
</sshProcess>
</processDefinition>
</processes>
|
|
As mentionned previously, many options exist, and correspond to the
main options specified in an SGE system. ScriptPath
represents the location on the frontend_host of the script
gridEngineStartRuntime.sh (default is
/user.home/ProActive/scripts/unix/cluster/gridEngineStartRuntime.sh).
- GLOBUS
Like ssh,
using globus
requires some steps to be performed. Of course globus
must be installed(at least globus client) on the machine that will
originates the RSL
request. See Globus
Installation for how to install
globus. Then you have to initialize some variables by running the
command $GLOBUS_INSTALLATION/etc/globus-user-env.sh,
then $GLOBUS_INSTALLATION/bin
/grid-proxy-init will initialize
the proxy, you will be asked for a passphrase, it is the one you
provided when requesting a user certificate at globus.org. Once these
steps are performed you can run ProActive
application using GRAM
protocol.
ProActive
provides org.objectweb.proactive.core.process.globus.GlobusProcess
to create globus
process.
<jvm name="Jvm2">
<creation>
<processReference refid="globusProcess"/>
</creation>
</jvm>
...................................................
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="globusProcess">
<globusProcess class="org.objectweb.proactive.core.process.globus.GlobusProcess" hostname="globus1.inria.fr">
<processReference refid="jvmProcess"/>
<environment>
<variable name="DISPLAY" value="machine_name0.0"/>
</environment>
<globusOption>
<count>10</count>
</globusOption>
</globusProcess>
</processDefinition>
</processes>
|
|
In this example, Jvm2
will be created using GRAM.
An RSL
request will be generated with informations provided in the descriptor.
For instance, the <environment> tag is not mandatory, but
for the globus host to export the DISPLAY on your machine, you can
define the value in the descriptor as well as other environment
variable, except the classpath(or java path,...) which must be defined
in the local process referenced by globusProcess
as explained before. <globusOption> is neither manatory.
Default value for <count> element is 1. It represents the
number of processor requested.
- OAR:
OAR is a cluster protocol develloped at INRIA Alpes to manage their cluster.
ProActive provides org.objectweb.proactive.core.process.oar.OARSubProcess
to use such protocol.
- PRUN:
PRUN is a cluster protocol develloped at Amsterdam to manage their cluster.
ProActive provides org.objectweb.proactive.core.process.prun.PrunSubProcess
to use such protocol.
- Complete
example of XML Deployment Descriptor
<?xml version="1.0" encoding="UTF-8"?>
<ProActiveDescriptor xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="DescriptorSchema.xsd">
<virtualNodesDefinition>
<virtualNode name="PenguinNode" property="multiple"/>
<virtualNodesDefinition/>
<deployment>
<mapping>
<map virtualNode="PenguinNode">
<jvmSet>
<vmName value="Jvm1"/>
<vmName value="Jvm2"/>
<vmName value="Jvm3"/>
<vmName value="Jvm4"/>
</jvmSet>
</map>
</mapping>
<jvms>
<jvm name="Jvm1">
<creation>
<processReference refid="jvmProcess"/>
</creation>
</jvm>
<jvm name="Jvm2">
<creation>
<processReference refid="jvmProcess"/>
</creation>
</jvm>
<jvm name="Jvm3">
<creation>
<processReference refid="sshInriaCluster"/>
</creation>
</jvm>
<jvm name="Jvm4">
<creation>
<processReference refid="globusProcess"/>
</creation>
</jvm>
</jvms>
</deployment>
<infrastructure>
<processes>
<processDefinition id="jvmProcess">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="jvmProcess1">
<jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess">
<classpath>
<relativePath origin="userHome" value="/ProActive/classes"/>
<relativePath origin="userHome" value="/ProActive/lib/bcel.jar"/>
<relativePath origin="userHome" value="/ProActive/lib/asm.jar"/>
<relativePath origin="userHome" value="/ProActive/lib/jini-core.jar"/>
<relativePath origin="userHome" value="/ProActive/lib/jini-ext.jar"/>
<relativePath origin="userHome" value="/ProActive/lib/reggie.jar"/>
.............
</classpath>
<javaPath>
<absolutePath value="/usr/local/jdk1.4.0/bin/java"/>
</javaPath>
<policyFile>
<absolutePath value="/home/ProActive/scripts/proactive.java.policy"/>
</policyFile>
<log4jpropertiesFile>
<absolutePath value="/home/ProActive/scripts/proactive-log4j"/>
</log4jpropertiesFile>
<ProActiveUserPropertiesFile>
<absolutePath value="/home/config.xml"/>
</ProActiveUserPropertiesFile>
</jvmProcess>
</processDefinition>
<processDefinition id="bsubInriaCluster">
<bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess">
<processReference refid="jvmProcess1"/>
<bsubOption>
<hostlist>cluster_group1 cluster_group2<hostlist/>
<processor>4</processor>
<resourceRequirement value="span[ptile=2]"/>
<scriptPath>
<absolutePath value="/home/ProActive/scripts/unix/cluster/startRuntime.sh"/>
</scriptPath>
</bsubOption>
</bsubProcess>
</processDefinition>
<processDefinition id="sshInriaCluster">
<sshProcess class="org.objectweb.proactive.core.process.ssh.SSHProcess" hostname="sea.inria.fr">
<processReference refid="bsubInriaCluster"/>
</sshProcess>
</processDefinition>
<processDefinition id="globusProcess">
<globusProcess class="org.objectweb.proactive.core.process.globus.GlobusProcess" hostname="cluster.inria.fr">
<processReference refid="jvmProcess1"/>
<environment>
<variable name="DISPLAY" value="machine_name0.0"/>
</environment>
<globusOption>
<count>10</count>
</globusOption>
</globusProcess>
</processDefinition>
</processes>
</infrastructure>
</ProActiveDescriptor>
|
|
This xml deployment descriptor shows how to deploy the Penguin
application on several places. Two Nodes will be created locally. We
can see that with the definition of Jvm1
and Jvm2.
These JVMs will be created locally since they reference directly the
process defined by jvmProcess.
Jvm3
will be created on the cluster using ssh
to log on galere1(cluster entry point). Two nodes will then be created
on galere10.inria.fr since processor tag is set to 2, and the hoslist
tag gives galere10.inria.fr as the only one candidate. At Last Jvm4
will be created using globus It will access cluster.inria.fr and
request 10 processors. We can notice that two local processes were
defined, the reason is that the first one jvmProcess
will use default value for the classpath, java path and policyfile
path, whereas for the second one jvmProcess1
, we need to define these value, since the home directory is different
between the local machine, and globus and the cluster(home dir is the
same on globus machines and on the cluster, that is why both processes
reference the same local process : jvmProcess1).
Even if quite a lot of things can be configured in the xml files,
sometimes you will have to perform additional steps to get everything
working properly, it is the case when using ssh, or globus as seen
before. In this example, DISPLAY variable is defined for the globus
process, that means that we want the penguin icon to appears on the
local machine, be carefull to authorize your X server to display such
icons by running the following command before launching the
application: xhost +cluster.inria.fr. If you want to do the same for
the cluster machine, that you plan to access with ssh you must run
xhost +galere10.inria.fr, and on the cluster side you need to create
under ~/.ssh a file called environment where you define the DISPLAY
variable. If you are not familiar with ssh, see openSSH