responsivstrap transparent positive 300x83 1

Author
Richard Whyte, CEO
at Responsiv 

PDF

Messaging Middleware

Messaging products use the “store and forward” technique to provide connectivity between applications. It’s one purpose is to deliver packets of information, known as messages, from one place to another. It does this thing very well, demonstrating surprisingly high performance and reliability.

Middleware sits in the middle of applications and the operating system to deliver capabilities that most applications need, and operating systems do not provide. Databases, Messaging, Transformation, and Web Application Servers are common examples.

Connect applications and microservices in private datacentres, across hybrid or multi-cloud environments, and at the edge of your enterprise. Tap into the value of your existing mission-critical data to gain real-time insights.

Middleware has been developed by humans, and you could develop your own. Why would you? The middleware exists to allow application developers to concentrate on the business problem rather than worrying about connectivity and data storage.

Responsiv Messaging

Responsiv Unity Messaging Node is based on IBM MQ. It simplifies monitoring and includes the ability to be connected to the Responsiv Service Desk. We offer variations on the IBM licence terms to help align cost and benefit. We provide a remote managed service that includes patching and upgrades as well as incident response.

  • End to end message encryption
  • Assured once-only delivery
  • Managed File Transfer
  • Transactional handoff between databases and message fabric (XA compliant)
  • Automatic load balancing with failover/fail back

IBM MQ was launched by IBM in 1993 as a product named MQSeries. Since then, the product has been renamed many times and been maintained to improve performance and functionality. The capabilities it offers are no less relevant now than in the late 21st century.

The Responsiv Cloud Automation Platform delivers the benefits of process automation and in-motion message manipulation in a way that seamlessly integrates with your existing MQ installation.

Fortune.com reports that 73% of the Fortune 100 companies use IBM MQ to protect data and save money. There are other products that use similar approaches, including Rabbit MQ, Azure Service Bus, Tibco, and MSMQ. Only IBM MQ protects your business from incorrect data and application errors with exactly once message delivery.

MQ performance information is available here.

Store and Forward

The idea of store and forward conjures the idea that messaging must be slow. This is far from the truth and in many circumstances the approach delivers faster throughput, response, and better latency than direct connections.

“Store and forward” is the way postal letters are handled. The sender places the letter in a known place (post box) with an outer wrapper instructing the delivery service where to route the package. Sometime later the delivery service collects the letter and delivers it, through a series of drops, to the destination.

The sender and receiver do not have to be awake at the same time, they can have different reading and writing speeds but do not have to wait for each other, and while the delivery service is operating bot sender and receiver can be doing other useful work.

Like letters, MQ messages can be sent in one order and received in another. This is an advantage when things don’t work as planned. Imagine a postal service that could not deliver anything because one letter was slow to process.

Print Spooler

You may not realise, but the “Store and forward” technique is used when you print a file on your desktop. The printer is slower than the program you are using and may be busy when you want to print. The computer gains significant benefit from allowing you to “print” (store) to a file called a “spool file”. This file is then forwarded to the printer as soon as its available.

Imagine having to wait for the printer to be free, and then be locked out of your document while Microsoft Word controls the slow printer.

Six ways that IBM MQ saves you money

Now you understand what MQ is doing, let’s see how these things can save money:

  1. Reduce workload on the sender application by offloading responsibility for accurate delivery to the messaging product
  2. Remove the need for polling* to reduce system overheads and avoid retry effort in applications
  3. Avoid applications having to wait for slower or more busy destinations to accept messages
  4. Arrange messages to allow them to be processed in any order to improve reliability, throughput, latency, and recovery from component failures
  5. Destination controlled load balancing
  6. Reduced need to scale for the peaks

[1] Reduce effort for the sender

Remember trying to phone a friend and having problems getting through? Many people prefer to use text messages because the receiver can decide when to read them and may be able to multi-task at their own speed. Well texting is another “store and forward” product.

We prefer to text because there is no real reason to retry or be blocked from sending. Even when your friend’s phone is switched off.

Reduced overheads and effort for the sender translates into a reduced demand for license and compute capacity – saving you money on cloud hosting and software licenses.

[2] Trigger the receiver to avoid polling

Polling is when a computer program continually asks, “are we there yet”. Not only irritating but very inefficient. Consider that a system receives one message each day. If it polls every 15 minutes, then it will poll 24×4=96 times and only once be successful. The message, when it is sent, will likely take an average 7.5 minutes to be delivered.

Using messaging, the receiver is triggered once, and the message arrives immediately with no artificial delays.

I have used a low volume and trivial example but with higher volumes of messages the advantages are far greater, for example from a stock control system, a point-of-sale device, an IoT device or a selection of SAP, MS-Dynamics, and other enterprise systems.

Polling can bring the biggest machines to their knees. Use messaging to avoid polling and reduce the delays and overheads that polling systems introduce.

[3] Avoid waiting for the slowest to keep up

You go for a brisk walk and end up crawling along with the slowest in the group. Systems that connect directly rather than using a store and forward approach, are forced to operate at the pace of the slowest machine. In the case of a printer and a supercomputer this can be a very expensive ball and chain.

Use store and forward to benefit from the performance of your application servers without waiting for the slowest webservices.

[4] Process workloads in any order

When workloads are arranged to be processed in any order the computers are released to work in parallel and to fail and recover in their own time. Processing can be arranged to avoid bottlenecks.

Several years ago, I designed a solution to gather data from hundreds of IoT devices. The existing solution polled in a round-robin fashion and several minutes to complete a cycle. Failed devices added significant delays to the cycle.

Using a messaging technology, it was possible to ask all of them for information at once, and to correlate and aggregate the responses as they returned. The result was a far more accurate and timely view of how things were performing. Failed devices responded too late and were ignored.

[5] Load balancing by the receiver

When an HTTP sender is load balanced the balancing happens before the data reaches the destination. This approach can result in some destinations working harder than others and unnecessary delay.

Using a messaging product ensures that all messages are delivered to a queue (store) that can be read by multiple destinations. Each destination only reads the next message when it has the capacity to process it.

The result is that messages are processed efficiently by all available capacity, significantly reducing capacity demand.

[6] Process work when capacity allows

For situations where latency is not a problem, the throughput can be reduced to save capacity without undermining the system. For example, a company experiences peak loads for a few minutes on a random basis or at specific times of the day. By storing those peaks, they can be completed a few seconds later.

The line is the new maximum capacity. Peaks that show above the line are processed in the next trough.

The technique can be used with websites and human interactions to give the appearance of fast response and good UI experience. The user request is stored and immediately a response is given that the request has been accepted for processing. Later, the response is delivered to the user.

Alternatively, a request is sent for information that is anticipated by the website so that it is waiting in the queue when needed.

Conclusion

There are simply too many benefits to list in a short POV. Responsiv has experts with the experience needed to successfully implement messaging middleware or simply to discuss the art of the possible.

Why not contact us with any questions?

    Last Name*

    First Name

    E Mail*

    Company*

    Lead Status*


    *By pressing submit you agree to receiving communication from Responsiv. You may unsubscribe from communications at any time.

    Richard Whyte

    Richard Whyte

    Richard Whyte has been building enterprise IT solutions for over 20 years. He is known for creating innovative practical solutions that provide a strong foundation for future development, whilst solving immediate problems. Previously the European CTO and Principal Architect for IBM Systems Middleware at IBM, he has an MBA, a degree in Statistics and Computing, is a Chartered Engineer, a Chartered IT Professional, and Fellow of both the Institute of Technology and the British Computer Society.