CMI (Clustered Method Invocation)

Introduction

CMI is the protocol cluster for JOnAS ensuring:

CMI architecture

Getting started with the CMI Protocol

CMI can be enabled in JOnAS by:

Clustered objects and CMI Registry

CMI brings its own registry for implementing the jndi replication. Each CMI registry instance over the cluster contains two areas:

When an object is registered in the registry, the routing to the local or global area is done according to:

The entries of the distributed area are lists providing the ability, for example, to gather several stubs for the the same jndi-name and thus to return a stubs list.

JNDI HA

Registry Replication

CMI relies on JGroups group-communication protocol for ensuring the global registry replication. The parameters are gathered in the:

CMI registry

All the members of a cluster share the same JGroups configuration.

If several cluster partitions are required over a single LAN, several JGroups configurations must be configured with different values for the following parameters:

When a new node appears in the cluster, its registry content is synchronized automatically.

When a node disappears, JGroups notifies the other's member of the node leaving and the registry entries related to this node are removed.

Registry Fail-over

On the client side, the high availability of the registry is provided by the capability to set several JOnAS instances in the registry url. At the lookup time, the client chooses (round-robin algorithm) one of the available servers to get the home stub. If the server fails, the request is sent to another server. The CMI url registry is specified in the $JONAS_BASE/conf/carol.properties file using the following syntax:

carol.cmi.url=cmi://server1:port1[,server2:port2...] 

CMI Cluster Stub or Cluster-aware Stub

Load-balancing and fail-over on the client side are provided through cluster-aware stubs. These stubs are generated on the fly through ASM and rely on:

CMI cluster stub

Cluster Map on the Client Side

The CMI cluster stub handles a cluster map on the client side. The CMI cluster stub is created:

In these two cases, the call gets a stubs list from the global registry and the CMI cluster stub updates the local cluster map. Afterwards, the local cluster map can be updated dynamically during the invocation of the business methods calls (through the HA interceptors) when a new view is detected in the cluster.

If a communication error with a server occurs during a remote call invocation, the server is removed from the cluster map.

CMI Class

CMI classes are generated by GenIC when compiling with the protocol CMI. They are built from the velocity templates located in $JONAS_ROOT/templates/genic directory. By default the templates used are:

The templates inherit the org.objectweb.carol.cmi.Distributor class and contain the following methods:

Customizing the Load-balancing and Fail-over Logic

The user has the ability to customize the load-balancing and fail-over logic for each EJB by specifying the velocity template to use in the JOnAS- specific descriptor on deployment of the ejb-jar file. The XML elements are:

<cluster-home-distributor>MyHomeDistributor.vm</cluster-home-distributor>
<cluster-remote-distributor>MyRemoteDistributor.vm</cluster-remote-distributor>

If not set, the default velocity templates are used.

If set with the value 'disabled', the CMI classes are not generated and the EJB will not be distributed.

If set with a file name, this file must be located in the $JONAS_ROOT/templates/genic directory.

The 'cluster-home-distributor' element is valid for the SSB, SFSB and EB.

The 'cluster-remote-distributor' element is valid for the SSB.

High Availability with Horizontal Replication

Stateful session beans (SFSBs) can be replicated since JOnAS 4.7 in order to provide high availability in the case of failures in clustered environments. A new service called High Availability (HA) has been included in JOnAS to provide replication mechanishms. JOnAS HA also requires the cluster method invocation (CMI) protocol.

Compared to JOnAS 4.7, JOnAS 4.8 implements a new replication algorithm based on a horizontal replication approach. The algorithm improves the algorithm implemented for JOnAS 4.7 with the following enhancements:

  • Replication of SFSBs with references to EBs: The algorithm can replicate SFSBs that reference EB by means of both, local or remote interfaces.
  • Transaction awareness: The algorithm is transaction aware, meaning that the state is not replicated if the transaction aborts.
  • Exactly-once semantics: Each transaction is committed exactly once at the DB if the client does not fail.
  • Replication Description

    JOnAS implements an update-everywhere replication protocol according to the database replication terminology (See the J. Gray et al.'s paper ''The dangers of replication and a solution'' in proceedings of the ACM SIGMOD 96's conference, Canada). In this protocol, a client can connect to any server. When the client calls the create() method on the SFSB's Home interface, the server the client connects to is selected following a round-robin scheme. All the requests from the client to the SFSB will be processed by this server until the client calls the remove() method on the remote interface. The rest of the servers will act as backups for that client. Before sending the response to the client, the SFSB's state is sent to the backups.

    If the server fails, another server among the backups will be selected to serve the client requests, first restoring the current state of the SFSBs from the state information stored in the HA local service. From this point on, this server will receive the new client requests.

    The supported replication scenarios are shown in the following figure:

    Replication scenarios in JOnAS 4.8

    Configuring JOnAS for SFSB Replication

    The High Availability (HA) service is required in JOnAS in order to replicate SFSBs. The HA service must be included in the list of available services in JOnAS. This is done in the jonas.properties file placed in $JONAS_BASE/conf.

    ...
    jonas.services registry,jmx,jtm,db,dbm,security,resource,ejb,ws,web,ear,ha
    ...

    The HA service must be configured also in the jonas.properties file:

    ...
    jonas.service.ha.class org.objectweb.jonas.ha.HaServiceImpl
    jonas.service.ha.gcl jgroups
    ...
    

    The HA service uses JGroups as a group communication layer (GCL). JGroups behavior is specified by means of a stack of properties configured through an XML file (See JGroups documentation for more information:http://www.jgroups.org). The default configuration of the HA service uses the $JONAS_BASE/conf/jgroups-ha.xml file and the sfsb-rep group name. The HA service can be told to use a particular stack configuration or a particular group name by modifying the following lines in jonas.properties :

    ...
    jonas.service.ha.jgroups.conf jgroups-ha.xml
    jonas.service.ha.jgroups.groupname jonas-rep
    ...
    Finally, the CMI protocol must be specified in the carol.properties file in $JONAS_BASE/conf:
    ...
    carol.protocols=cmi...
    ...

    Transaction Table Configuration

    The new horizontal replication algorithm uses a database table to keep track of current running transactions. This table is accessed from the new elected node during fail-over to detect whether the current transaction committed or not at the former local node, ensuring exactly-once semantics. The table contains only one column: the transaction identifier (txid).

    In JOnAS 4.8 this table must be created by hand with the following SQL command:

    create TABLE ha_transactions (txid varchar(60));

    This table should be located preferably in the database used by the replicated application, but it is not mandatory. If the table is not created in the database used by the replicated application, it is necessary to configure a new datasource for the database that contains this transaction table. This datasource must be configured to use the serializable transaction isolation level.

    The database that holds the transaction table is accessed by the replication service with the JNDI name configured in jonas.properties.

    ...
    jonas.service.ha.datasource example_ds
    ...

    Configuring Garbage Collection

    Due to the fact that the replication algorithm stores information associated with clients' transactions and that the server is not notified when a client dies, the HA service might have been storing unnecessary replication information over time. In order to automatically clean this unnecessary replication information, the HA service includes a garbage collection mechanism. It is possible to configure the number of seconds the system waits to execute this mechanism by changing the following property in the jonas.properties file:
    ...
    jonas.service.ha.timeout 600
    ...

    Configuring an Application for Replication

    In order to configure an application for replication, it is required to add the <cluster-replicated/> element to the bean definition of every bean requiring replication in the jonas-ejb-jar.xml deployment descriptor file. This element can have two possible values: true or false (default value). In addition, if the programmer wants to change the behavior of the CMI stubs (e.g., the server selection policy), it is possible to specify different distributor implementations by means of <cluster-home-distributor/> and <cluster-remote-distributor/> elements. In this case, the value corresponds to the .vm file that implements the distributor in its home and remote parts respectively. If the <cluster-replicated/> element is present without the <cluster-*-distributor/> elements, the default values are used (ClusterHomeSFSBRepDistributor.vm and ClusterRemoteSFSBRepDistributor.vm).

    The following is an example description for a replicated SFSB in jonas-ejb-jar.xml file:

    ...
    <jonas-session>
       <ejb-name>DummySFSB</ejb-name>
       <jndi-name>DummySFSB</jndi-name>
       ...
       <cluster-replicated>true</cluster-replicated>
       <cluster-home-distributor>Dummy_HomeDistributor.vm</cluster-home-distributor>
       <cluster-remote-distributor>Dummy_RemoteDistributor.vm</cluster-remote-distributor>
    </jonas-session>
    ...

    The lock policy for the Entity Beans in a replicated application must be configured as database in the jonas-ejb-jar.xml deployment descriptor file.

    The following is an example description for a replicated EB in the jonas-ejb-jar.xml:

    ...
    <jonas-entity>
        <ejb-name>MyEntitySLR</ejb-name>
        <jndi-name>MyEntityHome</jndi-name>
        <cluster-replicated>true</cluster-replicated>
        <shared>true</shared>
        <jdbc-mapping>
            <jndi-name>example_ds</jndi-name>
        </jdbc-mapping>
        <lock-policy>database</lock-policy>
    </jonas-entity>
    ...

    The datasources used by replicated applications must be configured to use the serializable transaction isolation level.

    The following is an example for a datasource configuration file for the Postgres DBMS:

    ...
    datasource.name         example_ds
    datasource.url          jdbc:postgresql://xxx.xxx.xxx.xxx:xxxx/database
    datasource.classname    org.postgresql.Driver
    datasource.username     jonas
    datasource.password
    datasource.mapper       rdb.postgres
    datasource.isolationlevel       serializable
    ...

    Finally, when compiling the application that includes the replicated beans, the CMI protocol must be specified in order to generate the classes that include the replication logic.