POINT OF VIEW

Move to Responsiv: Save Money and Create Opportunities

This POV is for business leaders of companies that have outgrown their investment in an integration solution

Developing applications and database systems is difficult, but their relative isolation from the rest of the business means that many common mistakes can be fixed without business-wide disruption.

Integration solutions are different. The job of an integration solution is to connect many systems to each other, and to expose their functions and data for new applications.

Good integration solutions create opportunities for businesses

A good integration solution creates opportunities for the business, and isolates applications and database systems to extend your ability to change and adapt to new challenges.

When business leaders recognise, and focus on extracting value from cooperation and information, they are immediately faced with two significant decisions; (1) what is my adoption strategy, and (2) what is the technology.

Adoption Strategy

The adoption strategy lays out how the stakeholders will work together to move from the current state to the new “thing” being in place. It should include how investment will be handled and paced through the schedule, the risks that need to be considered, and how departmental disruption will be accommodated. A critical decision will be the roadmap of projects and the technology to be adopted.

Integration solutions can be adopted incrementally and are often initially targeted at new projects or specific problem areas. This approach can lead to false pressures on the adoption strategy that can bite later.

First, the initial target problem does not justify a large investment and the strategic landscape is not included in the technology adoption decision.

Second, the initial problem is small and well understood, making it a good candidate to support learning about integration and having a reduced risk of project failure.

The result is that for all the right reasons, a lower cost and less capable solution is chosen and used for the first project. The relative simplicity and management focus means that the project is successful. The technology is adopted, and our research suggests that it will take approximately two years before the accumulated frustrations come to a decision point.

Does this make any sense to you?

If this resonates with your experience, you are not alone. If not, then congratulations. In either case, Responsiv is here to provide skilled resources, advice, and experience to help resolve problems.

The rest of this POV is to help business leaders that are frustrated by their integration strategy and want to investigate ways to make it better.

Common Misconceptions

It is important to appreciate how the business got to the current reality because the business requirements and reasons for decisions often remain relevant. The problem is with the tactics and the technology.

This is particularly true when businesses are wrong footed by claims of simplicity that are only partially true and distracted away from complex problems that become important later in the adoption roadmap. Examples of these misconceptions are

Transaction Integrity 

I am not a bank. Transaction integrity is not a concern for me.

While your systems are operating properly, and the volumes are low, this statement may be true. The examples below may take weeks to recognise and may happen on a regular basis.

Now imagine being a manufacturer that buys stock but forgets to record its arrival in the warehouse, or that sells stock and forgets to record the sales. Perhaps it’s easier to imagine being a retailer that forgets about several hours of home-shopping deliveries.

All of these situations are caused by lack of transactional integrity, and all have the ability to cause brand risk, significant cost, and disruption.

Performance and Throughput

I am a small company. Performance and throughput are not a concern for me. Think efficiency and sustainability, and green planet. High performance integration means that fewer cloud cores are needed to deliver the same workload. This reduces carbon emissions and reduces your cloud costs.

Realtime is not needed – file and batch is fine 

Realtime means that information is where it needs to be in time for it to be used. Telling me my house burned down last week is not as useful as telling me its burning now, or that it will be burning later.

Realtime integration reduces the problems of inconsistent information, and information that is simply out of date. It often reduces the need for steps in business processes and in some cases avoids processes all together.

I was told that APIs were the answer to life the universe and everything

Exposing information using APIs is great but I need to expand and connect various business systems that do not have defined interfaces or that are not standardised. I now understand that one-size-fits is not quite accurate. It’s not just about exposing data, it’s about intelligently combining and flowing data across different vendors and clouds.

My integration only does APIs. I am forced to use intermediary technologies to fill the gap.

Common Technology Adoption Mistakes

Decisions made by most business leaders are rational and reflect the state of understanding and working constraints. Over time, those constraints change because the decision opens new opportunities, or because the business imperatives move.

The initial problem does not justify a large investment and the strategic landscape is not included in the technology adoption decision.

Often for information sharing and coordination problems the initial symptoms and need do not warrant a large investment, however initial success will quickly drive demand for similar solutions across the enterprise.

The initial problem is small and well understood, making it a good candidate for learning about integration and having a reduced risk of project failure.

We all look for cost effective solutions to the problems presented and the choice of problem often encourages adoption of a solution with low capital cost but high and increasing operational cost. This is partly because quantifying operational cost is difficult.

Common misconceptions that lead to integration solutions that do not scale are:

1. A single API platform will solve all my problems
2. The initial cost is cheap, and I won’t need more connectors or capacity
3. The demonstration was great I will have no problem using this product
4. I need to connect people and my integration only does APIs

Misconceptions

A single API platform will solve all my problems

Having a converged technology strategy is a useful way to reduce the complexity of enterprise technology and should be a key consideration for any technology adoption process (TAP).

The idea of a single API platform for the enterprise is seductive. It will reduce technology proliferation and my staff will have a single set of skills to learn.

The reality is that I have struggled to separate workloads with different ownership, behaviour, performance, and security needs. The security team are insisting on additional firewalls and don’t like that the API host is seeing data that cannot be released through the APIs to external parties.

My attempts to use the platform in the DMZ and internally have confused the network teams and the solution has ended up with unexpected new technology to protect APIs that are hosted in two places anyway.

Responsiv point of view

An enterprise architecture should map capability not product. This means that the same technology may be installed in different places to deliver different capabilities. The integration software should be efficient and compact with a license model that allows it to be deployed in different parts of the enterprise.

Benefits from integration are low and licenses are expensive

Integration is a drug. Once you realise the potential the sky is the limit. The opportunities are endless, and the number of connectors and volume of transactions you end up with will be frightening.

“Can you speak Bocce?” “Of course I can, sir. It’s like a second language to me…”
“Yeah, all right. Shut up.”; “Shutting up, sir.” ―Owen Lars and C-3PO

A “simple to use” single solution that is cheap to start will get me going with a low adoption risk. I have a small department/business and the volumes cannot be that big?

The reality. The business users found that integration solved a lot of seemingly unrelated problems. They started looking at forgotten problems and demanding more connectors than our budget estimated. The costs are increasing as fast as the benefits.

My attempts to leverage the simplicity of my purchase have been undermined by a lack of skilled resources and the need to write loads of Java – in a tool that suggested low code.

Responsiv point of view

The adoption strategy needs to consider future projects and the problems they represent. Include two “difficult” or even fabricated projects when you are collecting requirements for the target projects. One of my favourites is the inside-leg-measurement. Imagine that a new regulation requires you to implement a field on every sales application to capture the customer’s inside-leg-measurement. Now assess the impact on your integration and how quickly and at what cost and disruption to add the field.

The demonstration was great I will have no problem using this product

Product vendors will demonstrate the strengths of their products and minimise the importance of their weaker aspects. To varying extents all vendors will show off their best side. The demonstration is by its nature a simple use case that can be shown quickly and chosen to be genuinely easy.

The reality. The product strengths will be demonstrated. This often does not include high volume, failure recovery, ease of upgrades, or conceptually difficult aspects of the product. Where code is needed, it is generally pre-written and shown to the customer.

Pre-conceived and written code hides a multitude of sins. It has been tested, and you have no way to know how long it took to write. Einstein’s famous E=MC² equation is quick to write, but could you come up with it? How many people could? A line of software is the same.

My attempts to leverage the simplicity of what was demonstrated has led to complex layers of logic that seem to work but are difficult to support. Some things seem to be impossible.

For example, I need to distribute data from a trusted source to multiple consumers. When I have a failure the status of data in downstream systems becomes undefined and untrusted.

The value of this weakness is unlikely to be highlighted in a short demonstration.

Responsiv point of view

The author’s experience with low code and no code products is that they are often not very flexible and can lack condition handling features that are needed for unreliable environments and the more complex situations encountered after a systems failure. Failures during cross platform replication are notoriously difficult to recover when the product does not provide detailed and trusted information about its progress and point of failure.

It is important to consider your requirements before looking at products. This way you know the questions that need answers, and it is more difficult for vendors to direct the review away from their weaknesses.

I need to connect people and my integration only does APIs

While some believe that APIs are the future, humans do struggle to learn and speak Bocce (Java). You need a tool that allows machine to machine and to human connections to be made and maintained. This new tool needs to remember what was happening over extended timescales to allow for human holidays and coffee breaks.

I believed that APIs were the answer to everything. I focused on that and at the time we had no requirement for human integration or the ability to compensate for errors that cannot be rolled back.

The reality. When you send an email it cannot be rolled back. The compensation is a phone call or email of apology/correction. Some information needs to be authorised before it is posed to a system, and on occasion I want to put a human process over the top of multiple back-end systems. My API-only platform is not up to the job.

My attempts to simplify have led to increased complexity, additional technology, and escalating costs. I am not looking forward to finding Java programmers to maintain the specialised integration code.

Responsiv point of view

The adoption strategy needs to consider future projects and the problems they represent. This includes the potential need to include human decisions in integration logic, and the possibility of human-only process management.

Recovering from the wrong integration technology

Businesses that recognise and address the problem quickly will tend to reduce the cost of the problem and reduce the complexity of the solution.

There are several ways to approach adoption of an alternative integration technology. The choice depends on the specific situation.

Understand the problem

In this POV we have illustrated some common mistakes and misconceptions. If you are here, then you have your own list of problems and perceptions about the state of the integration strategy. You need to write down what those perceptions are and gather the evidence.

It is possible that the perception of problems is wrong, or that once it is written down, the problems are simply resolved by changes to processes or staff movements.

If the problem is with the product because of a lack of function, unavoidably high license costs, vendor allergy, or lack of project progress then read-on.

Define target state

The target state should include a definition of what success looks like, and target improvements, for example a 20% cost saving over 2 years, or a 50% reduction in project delivery times.

Include requirements for two projects that are important and need to be delivered quickly, and for two that may be made up, but that represent the unexpected, for example integrations that require a different protocol SFTP, FTPS, MQTT, JMS, etc; or that have 4x expected volumes, or 10x connectors.

Don’t limit this to projects already implemented unless these remain problematic.

Review technologies

There are many different technologies from Opensource, MuleSoft, IBM, Responsiv, and Microsoft to name but a few. They are all capable of delivering the simple cases.

Many are focused on ASCII integrations using JSON or XML based protocols. You need to consider the mainframe, Sun, IoT, Industrial Robots, Medical Equipment, and older systems that may require Cobol, Fortran, Binary, or other integrations.

Consider transaction integrity and performance to be important for any business of any size. Make sure that the product supports XA transaction standards, and that it can connect to databases like Oracle, SQL Server, DB2, Postgres, Sybase and any that you have.

Think about the case of an error. Check that the product support compensation logic, transaction rollback and recovery, and restart recovery.

If development speed and project turnaround are important take a long look at the developer tools. Does the product have a distributed debugger, does it perform declarative (no code) parsing to a consistent state regardless of input structure and without writing any Java code? Can the developer parse complex files with no-code configuration, are simple things simple without making complicated things impossible.

Adopt and migrate

Businesses with just a few integrations can often rewrite those components in the new technology faster than trying to move them. The result is a cleaner implementation and improved understanding of the new platform.

Businesses with a mixture of critical and non-critical integrations need to consider moving them is groups of associated integrations using an iterative approach.

Businesses that are actively writing new integrations should move new development to the new platform as soon as practicable and have a separate project work stream to migrate existing projects.

In every case having a specialist company like Responsiv at your side, for their experience, additional capacity, and knowhow, is critical to your success.

Responsiv Automation

Responsiv specialise in integration and automation for businesses of all sizes and industries. Our consultants have decades of experience with many different vendor technologies and approaches to integration.

We can support your decision process as explained above, and we can provide objective and valuable insights to support your decisions and if necessary, your migration.

Responsiv partner with Red Hat, Microsoft, and IBM to allow us to access some of the most capable cloud, operating systems, and integration technologies around. We do not limit ourselves to these vendors. Rest assured that our consulting advice is not driven by our partners.

Responsiv Cloud Automation Platform

Our consultants use pre-existing assets to accelerate adoption of the Responsiv Cloud Automation Platform. Our cloud platform is deployed to datacentres located in the UK and can support businesses that need to integrate systems with or without human intervention.

We provide development services and ongoing support for the code we develop, as well as flexible support for your own developers.

Responsiv Unity Automation Platform

Responsiv Unity Automation Platform delivers similar functionality to that available from our cloud platform from the comfort of your own cloud of datacentre. It is fully compatible with the cloud platform and can be deployed across multiple clouds for resilience and right-placing of workloads.

Responsiv Migration

Responsiv has several custom offerings to reduce the cost and distraction of a move to Responsiv Cloud. We can perform assessments and provide pricing that reduces duplicate platform costs and spreads the cost of migration over a committed term.

If any of the mistakes or misconceptions resonate with you then please contact Responsiv for an informal chat.

Conclusion

Choosing the right solution for you isn’t always as easy as it should be. And even when we think we’ve found the answer it can turn out to be a hinderance down the line as we look to expand. Outgrowing a solution is unfortunately a very common thing. 

If you feel you have outgrown your integration solution, speak to Responsiv. Find out how you can future-proof your business with solutions that grow with you! 

*By pressing submit you agree to receiving communication from Responsiv. You may unsubscribe from communications at any time.