|
![]() |
Parameters tied to the deployment of an application should be totally described in a xml deployment descriptor. Hence within the source code, there are no longer any references to :
A ProActive application can be deployed on different hosts, with different protocols without changing the source code
newActive(String, Object[], Node);
But VirtualNodes are names in program source, to which corresponds one or a set of Nodes after activation
newActiveAsGroup(String, Object[], VirtualNode);
turnActiveAsGroup(Object, String, VirtualNode);
|
This example shows a VirtualNode called Dispatcher, that is mapped to a jvm called Jvm.
This Jvm1will be created using the process called jvmProcess which is defined in the infrastructure part(This part will be discussed later, just notice that there are two parts in the descriptor, an abstract one containing VirtualNode definition and deployment informations and a more concrete one containing concrete infrastructure informations, that is where all processes are defined).
ProActiveDescriptor pad = ProActive.getProactiveDescriptor(String xmlFile);
//--------- Returns a ProActiveDescriptor object from the xml fileVirtualNode dispatcher = pad.getVirtualNode("Dispatcher");
//-------- Returns the VirtualNode Dispatcher described in the xml file as a java objectdispatcher.activate();
// -------- Activates the VirtualNodeNode node = dispatcher.getNode();
// -------- Returns the first node available among nodes mapped to the VirtualNodeC3DDispatcher c3dDispatcher = newActive("org.objectweb.proactive.core.examples.c3d.C3DDispatcher",param, node);
..........................
Set of methods are provided in org.objectweb.proactive.descriptor.ProActiveDescriptor to manipulate VirtualNodes, to activate several VirtualNodes at the same time .... and in org.objectweb.proactive.core.descriptors.VirtualNode to manipulate and get nodes associated to VirtualNodes.
|
Another possibility for the one to one mapping is to map 1 VN to the jvm running the program. In that case the lookup protocol can be specified but is optionnal(default value is the property proactive.communication.protocol) as it is shown in the following:
|
Since it is the current jvm, it has not to be redifined later in the descriptor. This will be shown in a complete example
|
Note that the property attribute is set to multiple if you want to map 1 VN to multiple JVMs, and then a set of JVMs is defined for the VirtualNode
Renderer. Four values are possible for the property attribute: unique which means one to one mapping, unique_singleAO: one to one mapping and only one AO deployed on the corresponding node, multiple: one to N mapping, multiple_cyclic: one to N mapping in a cyclic manner. This property is not mandatory but an exception can be thrown in case of incompatibility. For instance property set to unique, and more than one jvm defined in the jvmSet tag. In case of property set to unique_singleAO method getUniqueAO() in class org.objectweb.proactive.core.descriptor.data.VirtualNode called on such VirtualNode returns the unique AO created
Three other attributes timeout, waitForTimeout, minNodeNumber can be set when defining a virtualNode
|
The timeout attribute represents an amount of time(in milliseconds) to wait before accessing Nodes mapped on the VirtualNode. The waitForTimeout attribute is a boolean. If set to true, you will have to wait exaclty timeout seconds before accessing Nodes. If set to false, timeout represents the maximum amount of time to wait, it means that if all nodes are created before the timeout expires, you get access to the Nodes. Defaut value for waitForTimeout attribute is false. The minNodeNumber attribute defines the minimum number of nodes to be created before accessing the nodes. If not defined, access to the nodes will occur once the timeout expires, or the number of nodes expected are effectively created. Setting this attribute allows to redefine the number of nodes expected, we define it as the number of nodes needed for the VirtualNode to be suitable for the application. In the exammple above, once 3 nodes are created and mapped to the VirtualNode Renderer, this VirtualNode starts to give access to its nodes. Those options are very usefull when there is no idea about how many nodes will be mapped on the VirtualNode(which is often unususal). Those attributes are optional.
|
In this example both VirtualNodes Dispatcher and Renderer have a mapping with Jvm1, it means that at deployment time, both VirtualNodes will get nodes created in the same JVM. Here is the notion of co-allocation in a JVM.
Descriptors provide the ability to register a VirtualNode in a registry such RMIRegistry, JINI Lookup, HTTP registry, IBIS/RMI Registry Service. Hence this VirtualNode will be accessible by another application as it is described in the VirtualNodes Acquisition section. The protocol(registry) to use can be specified in the descriptor, if not specified, the VirtualNode will register using the protocol specified in proactive.communication.protocol property.
|
The register tag allows to register the VirtualNode Dispatcher when activated, on the local machine in the RMIRegistry. As said before this VirtualNode will be accessible by another application using the lookup tag(see below) or using method: ProActive.lookupVirtualNode(String).
Descriptors provide the ability to acquire a VirtualNode already deployed by another application. Such VirtualNodes are defined in VirtualNodes Acquisition tag as it is done for VirtualNodesDefinition except that no property and no mapping with jvms are defined since such VNs are already deployed. In the deployment part, the lookup tag gives information on where and how to acquire the VirtualNode. Lookup will be performed when activating the VirtualNode.
|
As mentioned in the previous section, in order to acquire VirtualNode Dispatcher, it must have been previously registered on the specified host by another application. Sometimes, the host where to perform the lookup will only be known at runtime, it that case it is specified in the descriptor with "*" for the host attribute
|
Then when the host name is available, ProActive provides method setRuntimeInformations in class org.objectweb.proactive.core.descriptor.data.VirtualNode to update the value and to perform the lookup. Typical example of code:
ProActiveDescriptor pad = ProActive.getProactiveDescriptor(String xmlFileLocation);
pad.activateMappings;
vnDispatcher =
pad.getVirtualNode("Dispatcher");
..........................
vnDispatcher.setRuntimeInformations("LOOKUP_HOST","machine_name);
To summarize all VirtualNodes are activated by calling activate methods
except if "*" is set for a VirtualNode to be acquired. In that case the
lookup will be performed when giving host informations.
Registration and lookup can be performed automatically when using tags
in the descriptor as well as programmatically using static methods
provided in org.objectweb.Proactive class:
ProActive.registerVirtualNode(VirtualNode
virtualNode, String registrationProtocol, boolean
replacePreviousBinding );
ProActive.lookupVirtualNode(String
url, String protocol);
ProActive.unregisterVirtualNode(VirtualNode
virtualNode);
|
In this example, jvm1 will be created using the process called jvmProcess (discussed later, this process represents a java process and can be seen as java ProActiveClassname command)
|
This is the case when the process referenced is a cluster process(LSF, PBS, GLOBUS, ....) or a process list
Descriptors give the ability to acquire JVMs instead of creating them. To do so, it must be specified in the acquisition tag which service to use in oder to acquire the JVMs. Services will be described below, in the infrastructure part. At this point 2 services are provided: RMIRegistryLookup and P2PService service.
|
In this example, Jvm1 will be acquired using the service called lookup (discussed later, this service represents a way to acquire a JVM). Note that the name lookup is totally abstract, with the condition that a service with the id lookup is defined in the infrastructure part
To avoid mistake when building XML descriptors, ProActive provides an XML Schema called DescriptorSchema.xsd. Then to validate your file against this schema, the following line must be put at the top of the xml document as it is done for all ProActive examples.
<ProActiveDescriptor xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Location_of_DescriptorSchema.xsd">
Note that this schema is available in the ProActive distribution package under ProActive\descriptor directory. Using descriptors related methods (Proactive.getProactiveDescriptor(file)) triggers automatic and transparent validation of the file using Xerces2_4_0 if the ProActive property schema.validation is set to enable(see configuration file for more details). If a problem occurs during the validation, an error message is displayed otherwise if the validation is successful no message appear. An XML validation tool such as XMLSPY5.0(windows) can also be used to validate XML descriptors.
Following XML files examples are used for the C3D application. The first file is read when launching the C3DDispatcher. The second one is read every time a C3DUser is added. Both files contain many features described earlier in this document.
|
This example represents xml deployment descriptor for the C3D application. The abstract part containing VirtualNodes definition and deployment informations has already been explained. To summarize, two VirtualNodes are defined Dispatcher and Renderer. Dispatcher is mapped to the jvm running the main(), and will be exported using the protocol specified in proactive.communication.protocol property. This VirtualNode will be registered in a Registry(still using the protocol specified in proactive.communication.protocol property) when activated. Renderer is mapped to a set of JVMs called Jvm1, ..., Jvm4.
|
This file is read when addind a C3DUser. Two VirtualNodes are defined User which is mapped to the jvm running the main(), whose acquisition method is performed by looking up the RMIRegistry and Dispatcher in the virtualNodesAcquisition part which will be the result of a lookup in the RMIRegistry of a host to be specified at runtime.
In the previous example, all defined JVMs will be created using jvmProcess process. This name is abstract like the other ones, it means that it can be changed. This process is totally defined in the infrastructure part. Of course the process name in the creation part must point at an existing defined process in the infrastructure part. For instance if the name in the creation tag is localJVM, there must be a process defined in the infrastructure with the id localJVM
As said before, all processes are defined in the infrastructure
part, under the processes
tag. In the previous example, the defined process jvmProcess
will create local JVMs. The class attribute defines the class to
instantiate in order to create the process. ProActive library provides
one class to instantiate in order to create processes that will launch
local JVMs:
org.objectweb.proactive.core.process.JVMNodeProcess
|
As shown in the example above, ProActive
provides the ability
to
define or change the classpath
environment variable, the java
path, the policy
file path, the log4j
properties file path, the ProActive
properties file path (see configuration
file
for more details) and also to pass parameters
to the JVM to be created. Note that
parameters to be passed
here are related to the jvm in opposition to properties given in the configuration file(more
focused on ProActive or application behaviour). In fact parameters
given here will be part of the java command to create other jvms,
whereas properties given in the config file will be loaded once the jvm
is created.
If not specified, there is a default value (except for the
jvmParameters element) for each of these variables. In the first
example of this section, just the Id
of the process, and the class to
instantiate are defined. You might want to define the classpath
or java path
or policyfile path,
etc, when creating remote JVMs(discussed later) if the home directory
is not the same on your machine and on the machine where you want to
create the JVM or for instance if you want to interact with Windows
OS if you work on Linux and vice
versa. As shown in the example paths
to files can be either absolute
or relative.
If relative, an origin must be provided, it can be user.home
or user.dir
or user.classpath
and it is resolved locally,
i.e on the jvm reading the descriptor and not on the remote jvm that is
going to be created.
As mentionned in the configuration file,
if the <ProActiveUserPropertiesFile> is not defined for
remote jvms, they will load a default one once created.
Even if not shown in this example, a specific tag is provided for
XbootClasspath option under the form
|
With XML Deployment Descriptor, ProActive
provides the ability to create remote Nodes (remote JVMs). You can
specify in the descriptor if you want to access the remote host with rsh, ssh, rlogin, lsf, pbs, oar, prun, globus, unicore, arc (nordugrid), glite.
How to use these protocols is explained in the following examples. Just
remind that you can also combine these protocols.The principle of combination is fairly simple, you can imagine for instance that you
will log on a remote cluster frontend with ssh, then use pbs to book nodes and to create jvms on each. You will also notice that
there is at least one combination for each remote protocol. Indeed each remote protocol must have a pointer either on another remote protocol or on a jvmProcess to create a jvm(discussed previoulsy).
You can find here several examples of supported protocols and useful combinations.
Note that it is mandatory for using
all these features, that ProActive is installed on each host, of course
on the local host as well as on each host where you want to create Nodes
|
For the Jvm2
the creation process is rshProcess(still
an abstract name), which is defined in the infrastructure
section. To define this process you have to give the class to
instantiate to create the rsh
process. ProActive
provides org.objectweb.proactive.core.process.rsh.RSHProcess
to create rsh
process. You must give the remote host name to log on with rsh. You can
define as well username="toto"
if you plan to use rsh with -l
option. As said before this rsh
process must
reference a local process, and in the example, it references the
process defined with the id jvmProcess.
It means that once logged on sea.inria.fr with rsh, a local JVM will be
launched, ie a ProActive node will be created on sea.inria.fr thanks to
the process defined by jvmProcess.
Check here for a complete rsh deployment example.
|
You can use rlogin in the same way that you would use rsh
|
ProActive
provides org.objectweb.proactive.core.process.ssh.SSHProcess
to create ssh
process.
In order to use ssh to log on a remote host, you must perform some
actions. First you need to copy your public key (located in
identity.pub under ~/.ssh on your local machine) in the
authorized_keys(located under ~/.ssh) file of the remote host. Then to
avoid interactivity, you will have to launch on the local host the
ssh-agent command: ssh-agent
$SHELL, this command can be put
in your .xsession file, in order to run it automatically when logging
on your station. Then launching ssh-add
command to add your identity. Running this command will ask you to
enter your passphrase,
it is the one you provided when asking for an ssh key pair.
Note also that if the generated key pair is not encrypted (no
passphrase), you do not need to run neither the ssh-agent, nor the
ssh-add command. Indeed it is sufficient when using non encrypted
private key, to only copy the public key on the remote host (as
mentionned above) in order to get logged automatically on the remote
host.
These steps must be performed before
running any
ProActive application using ssh
protocol. If you are
not familiar with ssh, see openSSH
Check here for a complete ssh deployment example.
ProActive provides a way to define a list of processes for RSH, SSH, RLOGIN protocols. Using processList or processListbyHost elements avoids having a long deployment file when many machines with similar names are going to be connected with protocols mentionned before. The first example below shows how to use processList tag, the second how to use processListbyHost.
|
When using processList tag, the class attribute can take 3 values: org.objectweb.proactive.core.process.ssh.SSHProcessList, org.objectweb.proactive.core.process.rsh.RSHProcessList, org.objectweb.proactive.core.process.rlogin.RLoginProcessList according to the protocol being used is ssh, rsh or rlogin. The fixedName attribute is mandatory and represents the fixed part shared by all machine's names. The list attribute is also mandatory and can take several forms: [m-n] means from m to n with a step 1, [m-n;k] means from m to n with a step k (m, m+k, m+2k, ....), [m-n]^[x,y] means from m to n exluding x and y, [m-n]^[x,y-z] means from m to n exluding x and values from y to z, [m-n;k]^[x,y] same as before except that the step is k. The padding attribute is optional (default is 1) and represents the number of digits. Finally the domain attribute is mandatory and represents the last part shared by all machine's names. So in the exemple above, a jvm is going to be created using ssh on machines: node000.sophia.grid5000.fr, node002.sophia.grid5000.fr,..., node098.sophia.grid5000.fr, node100.sophia.grid5000.fr (note that step is 2) excluding machines: node010.sophia.grid5000.fr and node020.sophia.grid5000.fr.
|
Using processListbyHost element allows to give a hostlist separated with a whitespace. The class attribute is defined as described in the processList tag. The domain attribute is optional since the complete hostname can also be provided in the hostlist attribute. In the example, a jvm is going to be created using ssh on crusoe.inria.fr, waha.inria.fr, nahuel.inria.fr.
Check SSHList_example.xml or SSHListbyHost_Example.xml for list examples.
This protocol is used to create Nodes(JVMs) on a cluster. ProActive
provides org.objectweb.proactive.core.process.lsf.LSFBSubProcess
to create bsub
process.
In this part we assume that you want to submit a job from a machine
which is not the cluster frontend. As described before, you can combine
protocols. In this case , you will have to define a process to log on
the front-end of the cluster(rlogin
if your machine is on the same LAN than the cluster front-end, else ssh
(Remember that to use ssh
you will have to run some commands as explained above).
|
In this example, the JVM called Jvm2
will be created using ssh
to log on the cluster front end. Then a bsub
command will be generated thanks to the process defined by bsubInriaCluster.
This bsub
command will create Nodes on several cluster machines, since bsubInriaCluster
references the jvmProcess
defined process. All tags defined under <bsubOption>
are not mandatory, but they can be very usefull. The
<hostlist> tag defines possible candidates in the job
attribution, if not set the job will be allocated among all cluster's
machines. The <processor> tag defines the number of
processor requested, if not set, one processor is requested. The
<resourceRequirement> tag defines the expected number of
processors per machine. For instance <resourceRequirement
value="span[ptile=2]"/> ensures that 2 processors per machines
will be used, whereas value="span[ptile=1]" forces that LSF allocates
only only one processor per machine. It represents the -R
option of LSF. At last <scriptPath> defines the path on
the cluster front end of the script startRuntime.sh which is necessary
to run ProActive on a cluster. This script is located under
Proactive/scripts/unix/cluster. If not set the default location is set
as ~/Proactive/scripts/unix/cluster.
It is exactly the same with rlogin
instead of ssh.
If you want to submit the job directly from the cluster entry point,
define only the bsubProcess,
like in the above example and skip the ssh definition.
|
Note that in the example above two attributes: interactive and queue appear. They are optional, and have a default value: respectively false and normal. They represent option in the bsub command: interactive mode, and the name of the queue.
Check also SSH_LSF_Example.xml.
This protocol is used to create jobs on cluster managed by PBS, PBSPro or Torque. ProActive provides org.objectweb.proactive.core.process.pbs.PBSBSubProcess
to create pbs
processes. As explained for LSF you can combine protocols in order for
instance to log on the cluster's frontal with ssh, then to create nodes
using PBS, or you can also use only PBS without ssh if you are already
logged on the frontend. Example below shows how to combine an ssh
process to log on the cluster, then a PBS process that references a
jvmProcess in order to create nodes on processors requested by PBS.
|
Note that not all options are listed here, and some options mentionned in the example are optionnal: hostsNumber represents the number of host requested using pbs(default is 1), processorPerNode represents the number of processor per hosts requested(1 or 2, default is 1), bookingDuration represents the duration of the job(default is 1 minute), outputFile represents the file where to put the ouput of the job(default is specified by pbs), scriptPath represents the location on the frontend_host of the script pbsStartRuntime.sh(default is /user.home/ProActive/scripts/unix/cluster/pbsStartRuntime.sh).
Check also SSH_PBS_Example.xml.
This protocol is used to create jobs on cluster managed by Sun Grid
Engine. ProActive provides org.objectweb.proactive.core.process.gridengine.GridEngineSubProcess
to create grid engine
processes. As explained above you can combine protocols in order for
instance to log on the cluster's frontal with ssh, then to create nodes
using SGE, or you can also use only SGE without ssh if you are already
logged on the frontend. Example below shows how to combine an ssh
process to log on the cluster, then a SGE process that references a
jvmProcess in order to create nodes on processors requested by SGE.
|
As mentionned previously, many options exist, and correspond to the main options specified in an SGE system. ScriptPath represents the location on the frontend_host of the script gridEngineStartRuntime.sh (default is /user.home/ProActive/scripts/unix/cluster/gridEngineStartRuntime.sh).
Check also SSH_SGE_Example.xml.
OAR is a cluster protocol developed at INRIA Alpes and used on Grid5000.
ProActive provides org.objectweb.proactive.core.process.oar.OARSubProcess
to use such protocol.As explained above you can combine protocols in order for
instance to log on the cluster's frontal with ssh, then to create nodes
using OAR, or you can also use only OAR without ssh if you are already
logged on the frontend. Example below shows how to combine an ssh
process to log on the cluster, then an OAR process that references a
jvmProcess in order to create nodes on processors requested by OAR.
|
As mentionned previously, many options exist, and correspond to the main options specified in an OAR system. ScriptPath represents the location on the frontend_host of the script oarStartRuntime.sh (default is /user.home/ProActive/scripts/unix/cluster/oarStartRuntime.sh).
Check also SSH_OAR_Example.xml and SSH_OARGRID_Example.xml
PRUN is a cluster protocol develloped at Amsterdam to manage their cluster.
ProActive provides org.objectweb.proactive.core.process.prun.PrunSubProcess
to use such protocol.
Check also SSH_PRUN_Example.xml.
Like ssh,
using globus
requires some steps to be performed. In particular the java COG Kit (no need for the whole GT) must be installed on the machine that will
originates the RSL
request. See COG Kit
Installation for how to install
the client kit. Then you have to initialize your proxy by running COG_INSTALLATION/bin
/grid-proxy-init, you will be asked for a passphrase, it is the one you
provided when requesting a user certificate at globus.org. Once these
steps are performed you can run ProActive
application using GRAM
protocol.
ProActive
provides org.objectweb.proactive.core.process.globus.GlobusProcess
to create globus
process.
|
In this example, Jvm2 will be created using GRAM. An RSL request will be generated with informations provided in the descriptor. For instance, the <environment> tag is not mandatory, but for the globus host to export the DISPLAY on your machine, you can define the value in the descriptor as well as other environment variable, except the classpath(or java path,...) which must be defined in the jvmProcess referenced by globusProcess as explained before. <globusOption> is neither manatory. Default value for <count> element is 1. It represents the number of processor requested.
Check also Globus_Example.xml.
ProActive provides org.objectweb.proactive.core.process.unicore.UnicoreProcess
to use such protocol.
Check also Unicore_Example.xml.
ProActive provides org.objectweb.proactive.core.process.nordugrid.NGProcess
to use such protocol.
Check also NorduGrid_Example.xml.
ProActive provides org.objectweb.proactive.core.process.glite.GLiteProcess
to use such protocol.
Check also SSH_GLite_Example.xml.
|
This xml deployment descriptor shows how to deploy the Penguin
application on several places. Two Nodes will be created locally. We
can see that with the definition of Jvm1
and Jvm2.
These JVMs will be created locally since they reference directly the
process defined by jvmProcess.
Jvm3
will be created on the cluster using ssh
to log on sea.inria.fr (cluster entry point)and then bsub to request processors and to create jvms on each. Here, two nodes will be created
on machines that belong to cluster_group1 or cluster_group2 since processor tag is set to 2, and the hoslist
tag gives cluster_group1 cluster_group2 as candidates. At Last Jvm4
will be created using globus It will access cluster.inria.fr and
request 10 processors. We can notice that two local processes were
defined, the reason is that the first one jvmProcess
will use default value for the classpath, java path and policyfile
path, whereas for the second one jvmProcess1
, we need to define these value, since the home directory is different
between the local machine, and globus and the cluster(home dir is the
same on globus machines and on the cluster, that is why both processes
reference the same local process : jvmProcess1).
Even if quite a lot of things can be configured in the xml files,
sometimes you will have to perform additional steps to get everything
working properly, it is the case when using ssh, or globus as seen
before. In this example, DISPLAY variable is defined for the globus
process, that means that we want the penguin icon to appears on the
local machine, be carefull to authorize your X server to display such
icons by running the following command before launching the
application: xhost +cluster.inria.fr. On the cluster side you need to create
under ~/.ssh a file called environment where you define the DISPLAY
variable. If you are not familiar with ssh, see openSSH
As mentionned previously, instead of creating jvms, ProActive gives the possibility to acquire existing jvms. To do so, as shown in the example below, a service must be referenced in the acquisition tag. At this point two services are implemented: RMIRegistryLookup: this service performs a lookup in an RMIRegistry at the url specified in the service definition to find a ProActiveRuntime(a jvm) with the given name. P2PService service allows when using ProActive's P2P infrastructure to get as many jvms as desired.
|
The RMIRegistryLookup
service needs only an url
to perform the lookup. Many options exists for the P2PService
service: nodesAsked
represents the number of JVMs to acquire, the peer
element represents an entry point in the P2P system, many peers can be
specified. Elements acq and port represent the
communication protocol
and the listening port for the P2P Service, if a P2P Service is already
running with this configuration the descriptor will use this one
else a new one is started.
See P2P
documentation for more
information.
The example above shows a VirtualNode VnTest,
that is mapped with two jvms, Jvm1
and Jvm2. Jvm1
represents one jvm that will be acquired using an RMI Lookup, Jvm2
represents two jvms that will be found in the P2P infrastructure, so at
the end 3 acquired jvms are expected.
Fault Tolerance can also be defined at the service level. See Fault Tolerance documentation for more information.
ProActive gives the ability to kill all JVMs and Nodes deployed with an XML descriptor with the method: killall(boolean softly) in class ProActiveDescriptor:
ProActiveDescriptor pad =
ProActive.getProactiveDescriptor(String xmlFileLocation);
pad.activateMappings();
--------------------------------------------
--------------------------------------------
--------------------------------------------
pad.killall(false);
If softly is set to false, all jvms created when activating the descriptor are killed abruptely. If true a jvm that originates the creation of a rmi registry waits until registry is empty before dying. To be more precise a thread is created to ask periodically the registry if objects are still registered.
There is the possiblity to use only the infrastructure part in order to
create processes. A Schema called ProcessSchema located in the examples
directory allows to validate XML files for processes.
ProActive provides also the ability to use all processes defined above
without using XML Deployment Descriptor. You can programmatically
create such processes.
In order to get familiar on how to create processes programmatically
see Process
package