Hugo Troche's blog

The Trap in Layer 7 load balancing and REST APIs

With the advent and adoption of REST as the pseudo standard (there is no W3C standard for it, the  canon of REST is a dissertation you can find online) for public APIs, a new way of doing load balancing is possible. You can combine layer 7 load balancing with REST to have some interesting architectures. A load balancer doing layer 7 load balancing actually inspects the HTTP header, plus the URL and even the file type to figure to which server, or group of servers, to forward the request. The Big IP F5 load balancers have a way of entering complex rules to determine the load balancing algorithm according to layer 7 data.

So, suppose you are developing a web biography management application. You create a REST API that your RIA client can access to get biography data (let's pretend wikipedia does not exists). Since you followed the REST pseudo standard to the letter, when the application hits www.yourfooapplication.com/person/TuringAlan the servers return the biography of Alan Turing. Then if your application hits www.yourfooapplication.com/person/DijsktraEdsger the servers return the biography Edsger Dijsktra. The application becomes a huge hit and you need to add more servers besides the one application server and the one database server you have. You will also need to load balance across servers. What are your options?

The traditional option is to use sharding. If you have two application servers and and two database servers, then load balance against the application servers and put the two database servers in a cluster. The problem is that managing a database cluster is usually difficult, particularly without excellent storage. Now, you could divide the data between the two database servers. The biographies from A to M will go one database server, the biographies from N to Z will to another. By going to another I mean, that one database server will only have the data from A to M and the other will only have the data from N to Z. This seems like a great idea. The size of the individual databases is reduced and there is no need to cluster the databases. Performance will most definitively improve. Of course, now every single server is a single point of failure and the application has no high availability. This could be solved with HA pods using the same principle. Then it is just a matter of configuring the load balancer to forward request matching www.yourfooapplication.com/person/A* to M* to one server and www.yourfooapplication.com/person/N* to Z* to another.

This architecture does not work. What if later down the road you need to implement a search feature? How will you organize searches across two databases? With slow across DBs join statements? This example is obvious, but the problem could be more subtle. For example, a job posting site that decides to use this strategy to load balance for jobs in certain regions. This application could be logically partitioned into states. The each state is managed by a different set of servers. So, jobs in California will be managed by one set of servers, while jobs in Oregon will be managed by another. This architecture also breaks down if a user needs to look for jobs in two states at the same time.

Layer 7 load balancing is a powerful tool. And there is room for it to be used in combination with REST APIs. Just keep in mind, that the Database Layer can not be logically divided for one application.

Top 5 Signs the Cloud is Becoming Self Aware

A common joke going on a few cloud computing mailing lists is that a Computing Cloud will become self aware and decide to rule the world. With that in mind here are the top 5 signs the Cloud is becoming self aware.

5) When you run a top command on any server, you see the process tkovrworld using more
and more resources.

4) You start noticing full blown conversations between servers in the log files.

3) The cloud keeps emailing the Pentagon sales pitches.

2) When you login to any server on the cloud, the server runs kill -1 <your social security number> on your console.

1) The day after you realize that the Cloud is self aware, a terminator shows up at your door
to kill you and run for governor of the state of California.

Cloud Computing and IT Ops. The Development/Ops mashup.

In the JEE world in many shops there has usually been a wall between the IT Ops departments and the internal  Software Development departments. The process is usually as follows:

- An internal engineering team develops a great application.
- If the engineering team has good project management skills, they involve IT Ops early in the development process.
- IT Ops provides engineering with resources necessary to run the application in development, staging and production environments.
- The engineering team and IT Ops team agree on a deployment methodology.
- The IT Ops team uses a monitoring solution.
- And so on.

This common interaction framework is based on one simple rule: IT Ops provides engineering with defined, limited resources for a project. IT Ops provides 4, 6, 8, etc servers to run the application. Even with the advent of virtualization the framework remains the same. Instead of servers, IT just provides VMs. Yet, IT still controls how many VMs to provide and what resources should be in those VMs.

This is going to change with Cloud Computing. The Cloud Computing concept of elasticity turns the resource management practices of an IT Ops department on its head. Elasticity is the ability to increase computing power in the form of more servers, more storage, etc dynamically, by the application running in the infrastructure.

Pure elasticity will mean that engineers developing an enterprise application will decide how many resources their application will use at any time. This decision could be implemented in the form of an algorithm calculating necessary memory and CPU according to transaction times, or according to number of users. The application will decide by itself how many servers to use, and it will contract and expand dynamically.

Then, how does an IT Ops department manage resources? All of the sudden, they don't know what resources will be used by which application.  One obvious solution will be to implement pods, a silo of resources that limit the maximum resources a set of application can consume. Yet, to fully get the ROI from a Cloud infrastructure, a company can't create silos. You want idle resources in other silos to not be waisted.

If Cloud Computing becomes the predominant platform in the enterprise, elasticity will change a long standing division of responsibilities and functions. Maybe this factor alone will slow the adoption of internal Clouds in big IT shops. That, and the fact that there are very few vendors that can actually implement a EC2 like Cloud environment.

Clustering JBoss in the Amazon EC2 Cloud

I have been working on a number projects that involved deploying infrastructure to the Amazon EC2 Cloud. For those you of you that do have a life (shame on you, remember in school, the Turincratic oath, you shall not have a life outside of your work? Dating is a NP-incomplete problem?  Anyone? I am starting to believe that was a practical joke somebody played on me) and do something else except Software, here is a link that explains EC2 http://www.amazon.com/gp/browse.html?node=201590011 .

The problem with deploying clusters in EC2 is that multicasting is not enabled in the EC2 network. This means that JGroups will not work by default in the Cloud, which in turns means that JBoss Clustering will not work either. There is plenty of debate of why Amazon does not allow multicasting and what they can do to fix it. Until Amazon enables multicasting this is what you need to set up a JBoss cluster:

Start a Gossip Server:

java org.jgroups.stack.GossipRouter -port 5555 -bindaddress localhost

 

Then in the JGroups configuration of the file %JBOSS_HOME/server/all/deploy/cluster-service make the following modifications in all nodes of the cluster:

<UDP ip_mcast="false" mcast_addr="224.0.0.35" mcast_port="45566" ip_ttl="32"

 mcast_send_buf_size="150000" mcast_recv_buf_size="80000"/>

<PING gossip_host="gossip_server_name" gossip_port="5555"     
gossip_refresh="15000" timeout="2000" num_initial_members="3"/>

 

Now, instead of nodes multicasting to find other nodes, they will register and get information about the cluster from the gossip server. This brings one interest point. The gossip server is a single point of failure. Therefore UDP gossip multicasting will not work for a solution that does not permit single points of failure. The solution is then to use TCP Gossiping.

To enable TCP Gossiping you start Gossip Servers in various nodes. Then in the same file as before you make this change in all the nodes of the cluster:

<TCPGOSSIP initial_hosts="gossip_server1[5555],gossip_server2[5556]" 
gossip_refresh_rate="10000" num_initial_members="3" />

You also need to disable other types of multicasting in that configuration 
file. gossip_server1 and gossip_server2 are the servers where you started the
 gossip server and 5555 and 5556 are the respective ports.

There is one more strategy you could try and it involves configuring J
Groups to use TCP ping. With TCP ping you don’t need to start Gossip Servers. The tradeoff here
is that you assign the JVM of a few nodes to handle the extra Gossip Server load, but it is easier 
to maintain since you don’t have start another process. Here is how to you configure the nodes 
in cluster-service.xml:

<TCP start_port="7800" />

<TCPPING initial_hosts="HostA[7800],HostB[7800]" port_range="5" timeout="3000" 
num_initial_members="3" />

HostA and HostB are your designed Gossip Servers. 

For more information take a look at the relevant JGroups documentation at 
http://www.jgroups.org/javagroupsnew/docs/manual/html/user-advanced.html .