What is a Message Queue?

One of the underlying premises behind message queues is that you are working on a distributed application. If your entire application is capable of running in a single executable, then don't waste your time with this article (other than as an opportunity to learn). If you do have a distributed application, then there are a number of issues with which you must contend. Ones that you are probably already aware of - integration with legacy systems and network reliability.
Since the term legacy system has a different meaning depending on whom you talk to, let me give you mine quickly. A legacy system is one that has a database interface that is markedly different than the one that will be used on your current database. We realize that this is not the classic definition, but it is necessary for our exercise, since a newly developed application on a foreign database is still one with which we need to integrate. Our definition includes such an application in our scope of the problem.

The issue with legacy systems in our narrowly defined world is how do we access business rule procedures that may already be defined in these systems. We certainly don't want to rewrite into Visual Basic the accounts receivable system that has been developed over many years on an AS400. And simply having access to the data won't solve our problem since the rules are probably implemented in an archaic language, like COBOL or (gasp) RPG. So the question of how do we communicate with the AS400 at the business rule level.

With respect to network reliability, we're sure that none of you have any issues with the uptime of your network. In other words, whenever you request that some kind of transaction take place on a remote system, it happens immediately, and with no latency of any kind. Hands up all who qualify. Didn't think so. This problem is compounded when you try to run your environment across a wide area network or the Internet. So how do we ensure that the transactions we require are actually performed? The answer to both of these problems lies in message queues.

Consider a message queue to be like an in box on your desk. As people come up with work for you to do, they put the request into the pile on your in-tray. As you finish whatever task you are currently working on, you reach into you tray and pull out the next task to work on. Since you are a fair person, you perform the tasks on a first come, first served basis. What you have built is a message queue, albeit a manual one.

There are two basic functions in a computer-based message queue. First is a mechanism that allows one process to place a message into a queue. The second occurs when another process retrieves a message from the queue. The true beauty of the solution is that the two processes can not only be on different machines but running under different operating systems. It is this ability that allows message queues to transcend the problem with connecting to legacy systems.

The other principal attribute of a message queue is that the delivery of the message is guaranteed. In other words if a message is sent to a queue and the system on which the receiving process would be found is not running, the message remains in the queue. Then when the receiving system comes back to life, the message is sitting there waiting to be processed.

There are a number of other benefits offered by using a message queue. One of the main ones is the ability to scale your application by adding machines as the load demands it. The message queues that we are discussing can have messages added by more than one system. For example, if you are running a web site on a sever farm with a load balancing application to apportion the requests as they come in, query and update requests could come from any one of the web servers. And on the back end, you may want to have more than one server handle the middle tier (i.e. business rules) of your application. By using a message queue, you can have each of the web servers feed a single queue. Then as each middle tier server finishes its previous task, it can take the incoming requests from the queue and process them on a first come, first served basis. Additional processing power is created by tying new machines to either the load balancing application or the message queue with a minimal impact on the design of the application.

Another benefit of message queues can be derived if a portion of your computing environment is mobile. You can easily design a standalone application that allows salespeople to enter order information onto their laptop computers while they are sitting in the client's office. These orders are placed into a message queue that resides on their computer. Once they connect up to the corporate network, the guaranteed delivery portion of message queuing causes the entered orders to be updated to the central database. And all of this processing happens with no active action taken by the user, other than dialing into the network.