Clustering JBoss in the Amazon EC2 Cloud

I have been working on a number projects that involved deploying infrastructure to the Amazon EC2 Cloud. For those you of you that do have a life (shame on you, remember in school, the Turincratic oath, you shall not have a life outside of your work? Dating is a NP-incomplete problem?  Anyone? I am starting to believe that was a practical joke somebody played on me) and do something else except Software, here is a link that explains EC2 http://www.amazon.com/gp/browse.html?node=201590011 .

The problem with deploying clusters in EC2 is that multicasting is not enabled in the EC2 network. This means that JGroups will not work by default in the Cloud, which in turns means that JBoss Clustering will not work either. There is plenty of debate of why Amazon does not allow multicasting and what they can do to fix it. Until Amazon enables multicasting this is what you need to set up a JBoss cluster:

Start a Gossip Server:

java org.jgroups.stack.GossipRouter -port 5555 -bindaddress localhost

 

Then in the JGroups configuration of the file %JBOSS_HOME/server/all/deploy/cluster-service make the following modifications in all nodes of the cluster:

<UDP ip_mcast="false" mcast_addr="224.0.0.35" mcast_port="45566" ip_ttl="32"

 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"/>

<PING gossip_host="gossip_server_name" gossip_port="5555"     
gossip_refresh="15000" timeout="2000" num_initial_members="3"/>

 

Now, instead of nodes multicasting to find other nodes, they will register and get information about the cluster from the gossip server. This brings one interest point. The gossip server is a single point of failure. Therefore UDP gossip multicasting will not work for a solution that does not permit single points of failure. The solution is then to use TCP Gossiping.

To enable TCP Gossiping you start Gossip Servers in various nodes. Then in the same file as before you make this change in all the nodes of the cluster:

<TCPGOSSIP initial_hosts="gossip_server1[5555],gossip_server2[5556]" 
gossip_refresh_rate="10000" num_initial_members="3" />

You also need to disable other types of multicasting in that configuration 
file. gossip_server1 and gossip_server2 are the servers where you started the
 gossip server and 5555 and 5556 are the respective ports.

There is one more strategy you could try and it involves configuring J
Groups to use TCP ping. With TCP ping you don’t need to start Gossip Servers. The tradeoff here
is that you assign the JVM of a few nodes to handle the extra Gossip Server load, but it is easier 
to maintain since you don’t have start another process. Here is how to you configure the nodes 
in cluster-service.xml:

<TCP start_port="7800" />

<TCPPING initial_hosts="HostA[7800],HostB[7800]" port_range="5" timeout="3000" 
num_initial_members="3" />

HostA and HostB are your designed Gossip Servers. 

For more information take a look at the relevant JGroups documentation at 
http://www.jgroups.org/javagroupsnew/docs/manual/html/user-advanced.html .

Comments

[sureshrk19]

Hi Hugo,

Thanks for the nice post :-) I have been trying to setup JBoss cluster on EC2 and got stuck at multicasting.Your post helped me moving forward...
But, I couldn't finish the setup yet... I followed teh steps mentioned in the post but, the nodes are not discovered yet.
On both the nodes I'm getting following error...
------------
14:41:34,926 ERROR [GossipClient] exception connecting to host 10.254.163.159:5556
....
...
14:39:49,611 ERROR [GossipClient] exception connecting to host 10.255.199.176:5555

--------

I started Gossip server on 10.255.199.176 at 5555
On
10.254.163.159 at 5556

Am I missing anything?

Any pointers would be highly appreciated.

Thanks,
Suresh

[shibyniju]

That was indeed a good post and will be helpful for Jee deployments.

I have a doubt regarding deploying web applications in Cloud (any). How will be the communication between the application server (say JBoss) and the database server (MySql) of a web application in the following cases.
Case 1: The whole application in the cloud.
Case 2: The database alone in the cloud.

Any reply would be highly appreciated.

Thanks in advance.