back to API     back to index     back to guided tour index     prev     next  

4. SPMD Programming

OO SPMD on a Jacobi example

 

1. Execution and first glance at the Jacobi code

1.1 Source files: ProActive/src/org/objectweb/proactive/examples/jacobi

The Jacobi example is made of two Java classes:

- Jacobi.java: the main class

- SubMatrix.java: the class implementing the SPMD code

Have a first quick look at the code, especially the Jacobi class, looking for the strings "ProActive", "Nodes", "newSPMDGroup".


The last instruction of the class:

matrix.compute();

is an asynchronous group call. It sends a request to all active objects in the SPMD group, triggering computations in all the SubMatrix.

We will get to the class SubMatrix.java later on.

1.2 Execution

ProActive examples come with scripts to easily launch the execution under both Unix and Windows.

For Jacobi, launch:

ProActive/scripts/unix/jacobi.sh

or

ProActive/scripts/windows/jacobi.bat

The computation stops after minimal difference is reached between two iterations (constant MINDIFF in class Jacobi.java), or after a fixed number of iteration (constant ITERATIONS in class Jacobi.java).


The provided script, using an XML descriptor, creates 4 JVMs on the current machine. The Jacobi class creates an SPMD group of 9 Active Objects; 2 or 3 AOs per JVM.

Look at the traces on the console upon starting the script; in the current case, remember that all JVMs and AOs send output to the same console. More specifically, understand the following:

- "Created a new registry on port 1099"

- "Reading deployment descriptor ... Matrix.xml "

- "created VirtualNode"

- "**** Starting jvm on"

- "ClassFileServer is reading resources from classpath"

- "Detected an existing RMI Registry on port 1099""

- "Generating class : ... jacobi.Stub_SubMatrix "

- "ClassServer sent class ... jacobi.Stub_SubMatrix successfully"


You can start IC2D (script ic2d.sh or ic2d.bat) in order to visualize the JVMs and the Active Objects. Just activate the "Monitoring a new host" in the "Monitoring" menu at the top left.

To stop the Jacobi computation and all the associated AOs, and JVMs, just ^C in the window where you started the Jacobi script.

2. Modification and compilation

2.1 Source modification

Do a simple source modification, for instance changing the values of the constants MINDIFF (0.00000001 for ex) and ITERATIONS in class Jacobi.java.

Caveat: Be careful, due to a shortcoming of the Java make system (ant), make sure to also touch the class SubMatrix.java that uses the constants.

2.2 Compilation

ProActive distribution comes with scripts to easily recompile the provided examples:

linux>ProActive/compile/build

or

windows>ProActive/compile/build.bat

Several targets are provided (start build without arguments to obtain them). In order to recompile the Jacobi, just start the target that recompile all the examples:

build examples

2 source files must appear as being recompiled.

Following the recompilation, rerun the examples as explained in section 1.2 above, and observe the differences.

3. Detailed understanding of the OO SPMD Jacobi

3.1 Structure of the code

Within the class SubMatrix.java the following methods correspond to a standard Jacobi implementation, and are not specific to ProActive:

- internalCompute ()

- borderCompute ()

- exchange ()

- buildFakeBorder (int size)

- buildNorthBorder ()

- buildSouthBorder ()

- buildWestBorder ()

- buildEastBorder ()

- stop ()

The methods on which asynchronous remote method invocations take place

are:

- sendBordersToNeighbors ()

- setNorthBorder (double[] border)

- setSouthBorder (double[] border)

- setWestBorder (double[] border)

- setEastBorder (double[] border)

The first one sends to the appropriate neighbors the appropriate values, calling set*Border() methods asynchronously. Upon execution by the AO, the methods set*Border() memorize locally the values being received.

Notice that all those communication methods are made of purely functional Java code, without any code to the ProActive API.

On the contrary, the followings are ProActive related aspects:

- buildNeighborhood ()

- compute ()

- loop ()

We will detail them in the next section.

Note: the classes managing topologies are still under development. In the next release, the repetitive and tedious topology related instructions (e.g. methods buildNeighborhood) won't have to be written explicitly by the user, whatever the topology (2D, 3D).

3.2 OO SPMD behavior

Let us detail the OO SPMD techniques and ProActive related methods.

First of all, look for the definition and use of the attribute "asyncRefToMe". Using the primitive "getStubOnThis()", it provides a reference to the current active object **on which method calls are asynchronous**. It permits the AO to send requests to itself.

For instance in

this.asyncRefToMe.loop();

Notice the absence of classical loop. The method "loop()" is indeed asynchronously called from itself; it is not really recursivity since it does not have the drawback of the stack growing. It features an important advantage: the AO will remain reactive to other calls being sent to him. Moreover, it eases reuse since it is not necessary to explicitly encode within the main SPMD loop all the messages that have to be taken into account. It also facilitates composition since services can be called by activities outside the SPMD group, they will be automatically executed by the FIFO service of the Active Object.

The method "buildNeighborhood ()" is called only once for initialization. Using a 2D topology (Plan), it constructs references to north, south, west, east neighbors -- attributes with respective names. It also construct dynamically the group of neighbors. Starting from an empty group of type SubMatrix

this.neighbors = (SubMatrix) ProActiveGroup.newGroup

(SubMatrix.class.getName());

such typed view of the group is used to get the group view: Group neighborsGroup = ProActiveGroup.getGroup(this.neighbors); Then, the appropriate neighbors are added dynamically in the group, e.g.:

neighborsGroup.add(this.north);

Again, the classes managing topologies will permit to simplify this code.

3.3 Adding a Method barrier for a step by step execution

Let say we would like to control step by step the execution of the SPMD code. We will add a barrier in the SubMatrix.java, and control the barrier from input in the Jacobi.java class.

In class SubMatrix.java, add a Method barrier() of the form:

String[] st= new String[1];

st[0]="keepOnGoing";

ProSPMD.barrier(st);

Do not forget to define the keepOnGoing() method that indeed can return void, and just be empty. Find the appropriate place to call the barrier() Method in the loop() Method.

In class Jacobi.java, just after the compute() Method, add an infinite loop that, upon a user's return key pressed, calls the method keepOnGoing() on the SPMD group "matrix". Here are samples of the code:

while (true) {

printMessageAndWait();

matrix.keepOnGoing();

}

...

private static void printMessageAndWait() {

java.io.BufferedReader d = new java.io.BufferedReader(

new java.io.InputStreamReader(System.in));

System.out.println(" --> Press return key to continue");

System.out.println(" or Ctrl c to stop.");

try {

d.readLine();

} catch (Exception e) {

}

}

Recompile, and execute the code. Each iteration needs to be activated by hitting the return key in the shell window where Jacobi was launched. Start IC2D (./ic2d.sh or ic2d.bat), and visualize the communications as you control them. Use the "Reset Topology" button to clear communication arcs. The green and red dots indicate the pending requests.

You can imagine and test other modifications to the Jacobi code.

3.4 Undestanding various different kind of barriers

The group of neighbors built above is important wrt synchronization. Below in method "loop()", an efficient barrier is achieved only using the direct neighbors:

ProSPMD.barrier("SynchronizationWithNeighbors"+ this.iterationsToStop, this.neighbors);

This barrier takes as a parameter the group to synchronize with: it will be passed only when the 4 neighbors in the current 2D example have reached the same point. Adding the rank of the current iteration allows to have a unique identifier for each instance of the barrier.

Try to change the barrier instruction to a total barrier:

ProSPMD.barrier("SynchronizationWithNeighbors"+ this.iterationsToStop);

Then recompile and execute again. Using IC2D observe that many more communications are necessary.

In order to get details and documentation on Groups and OO SPMD, have a look at:

ProActive/src/org/objectweb/proactive/doc-files/

TypedGroupCommunication.html

OOSPMD.html

4. Virtual Nodes and Deployment descriptors

4.1 Virtual Nodes

Get back to the source code of Jacobi.java, and understand where and how the Virtual Nodes and Nodes are being used.

4.2 XML Descriptors

The XML descriptor being used is:

ProActive/descriptors/Matrix.xml

Look for and understand the following definitions:

- Virtual Node Definition

- Mapping of Virtual Nodes to JVM

- JVM Definition

- Process Definition

A detailed presentation of XML descriptors is available at:

ProActive/docs/api/index.html

entry 9. XML Deployment Descriptors

4.3 Changing the descriptor

Edit the file Matrix.xml in order to change the number of JVMs being used. For instance, if your machine is powerful enough, start 9 JVMs, in order to have a single SubMatrix per JVM.

You do not need to recompile, just restart the execution. Use IC2D to visualize the differences in the configuration.

5. Execution on several machines and Clusters

5.1 Execution on several machines in the room

ProActive/examples/descriptors/Matrix.xml is the XML deployment file used in this tutorial to start 4 jvms on the local machine. This behavior is achieved by referencing in the creation tag of Jvm1, Jvm2, Jvm3, Jvm4 a jvmProcess named with the id localProcess. To summarize briefly at least one jvmProcess must be defined in an xml deployment file. When this process is referenced directly in the creation part of the jvm definition (like the example below), the jvm will be created locally. On the other hand, if this process is referenced by another process(rshProcess for instance, this is the case in the next example), the jvm will be created remotely using the related protocol (rsh in the next example).

Note that several jvmProcess can be defined, for instance in order to specify different jvm configurations (e.g classpath, java path,...).

<ProActiveDescriptor xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="DescriptorSchema.xsd">
 <componentDefinition>
  <virtualNodesDefinition>
   <virtualNode name="matrixNode" property="multiple"/>
  </virtualNodesDefinition>
 </componentDefinition>
 <deployment>
  <mapping>
   </map>
   <map virtualNode="matrixNode">
    <jvmSet>
     <vmName value="Jvm1"/>
     <vmName value="Jvm2"/>
     <vmName value="Jvm3"/>
     <vmName value="Jvm4"/>
    </jvmSet>
   </map>
  </mapping>
  <jvms>
   <jvm name="Jvm1">
    <creation>
     <processReference refid="localProcess"/>
    </creation>
   </jvm>
   <jvm name="Jvm2">
    <creation>
     <processReference refid="localProcess"/>
    </creation>
   </jvm>
   <jvm name="Jvm3">
    <creation>
     <processReference refid="localProcess"/>
    </creation>
   </jvm>
   <jvm name="Jvm4">
    <creation>
     <processReference refid="localProcess"/>
    </creation>
   </jvm>
  </jvms>
 </deployment>
 <infrastructure>
  <processes>
   <processDefinition id="localProcess">
    <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
   </processDefinition>
  </processes>
 </infrastructure>
</ProActiveDescriptor>

Modify your XML deployment file to use the current jvm (i.e the jvm reading the descriptor) and also to start 4 jvms on remote machines using rsh protocol.

Use IC2D to visualize the machines and the JVMs being launched on them.

<ProActiveDescriptor xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="DescriptorSchema.xsd">
 <componentDefinition>
  <virtualNodesDefinition>
   <virtualNode name="matrixNode" property="multiple"/>
  </virtualNodesDefinition>
 </componentDefinition>
 <deployment>
  <mapping>
   </map>
   <map virtualNode="matrixNode">
    <jvmSet>
     <currentJvm />
     <vmName value="Jvm1"/>
     <vmName value="Jvm2"/>
     <vmName value="Jvm3"/>
     <vmName value="Jvm4"/>
    </jvmSet>
   </map>
  </mapping>
  <jvms>
   <jvm name="Jvm1">
    <creation>
     <processReference refid="rsh_titi"/>
    </creation>
   </jvm>
   <jvm name="Jvm2">
    <creation>
     <processReference refid="rsh_toto"/>
    </creation>
   </jvm>
   <jvm name="Jvm3">
    <creation>
     <processReference refid="rsh_tata"/>
    </creation>
   </jvm>
   <jvm name="Jvm4">
    <creation>
     <processReference refid="rsh_tutu"/>
    </creation>
   </jvm>
  </jvms>
 </deployment>
 <infrastructure>
  <processes>
   <processDefinition id="localProcess">
    <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
   </processDefinition>
   <processDefinition id="rsh_titi">
    <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="titi">
     <processReference refid="localProcess"/>
    /rshProcess>
   </processDefinition>
   <processDefinition id="rsh_toto">
    <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="toto">
     <processReference refid="localProcess"/>
    /rshProcess>
   </processDefinition>
   <processDefinition id="rsh_tata">
    <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="tata">
     <processReference refid="localProcess"/>
    /rshProcess>
   </processDefinition>
   <processDefinition id="rsh_tutu">
    <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="tutu">
     <processReference refid="localProcess"/>
    /rshProcess>
   </processDefinition>
  </processes>
 </infrastructure>
</ProActiveDescriptor>

Pay attention of what happened to your previous XML deployment file. First of all to use the current jvm the following line was added just under the jvmSet tag

<jvmSet>
 <currentJvm />
 ...
<jvmSet>

Then the jvms are not created directly using the localProcess, but instead using other processes named rsh_titi, rsh_toto, rsh_tata, rsh_tutu

<jvms>
 <jvm name="Jvm1">
  <creation>
   <processReference refid="rsh_titi"/>
  </creation>
 </jvm>
 <jvm name="Jvm2">
  <creation>
   <processReference refid="rsh_toto"/>
  </creation>
 </jvm>
 <jvm name="Jvm3">
  <creation>
   <processReference refid="rsh_tata"/>
  </creation>
 </jvm>
 <jvm name="Jvm4">
  <creation>
   <processReference refid="rsh_tutu"/>
  </creation>
 </jvm>
</jvms>

Those processes as shown below are rsh processes. Note that it is mandatory for such processes to reference a jvmProcess, in this case named with the id localProcess, to create, at deployment time, a jvm on machines titi, toto, tata, tutu, once connected to those machines with rsh.

<processDefinition id="localProcess">
 <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/>
</processDefinition>
<processDefinition id="rsh_titi">
 <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="titi">
  <processReference refid="localProcess"/>
 /rshProcess>
</processDefinition>
<processDefinition id="rsh_toto">
 <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="toto">
  <processReference refid="localProcess"/>
 /rshProcess>
</processDefinition>
<processDefinition id="rsh_tata">
 <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="tata">
  <processReference refid="localProcess"/>
 /rshProcess>
</processDefinition>
<processDefinition id="rsh_tutu">
 <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHProcess" hostname="tutu">
  <processReference refid="localProcess"/>
 /rshProcess>
</processDefinition>

5.2 Execution on Clusters

If you have access to your own clusters, configure the XML descriptor to launch the Jacobi on them, using the appropriate protocol:

ssh, LSF, PBS, Globus, etc.

Have a look at XML Descriptors documentation to get the format of the XML descriptor for each of the supported protocols.