Journey to Cloud Systems: From Mainframe Monoliths to Cloud Microservices

The evolution of software development has been closely linked to the advancement of hardware technology. Years ago, hardware was expensive and bulky, with limited processing ability. Due to this, developers needed to write small, efficient applications for this hardware

As programming languages became more sophisticated, developers were able to create more complex applications. However, these applications were still constrained by the hardware limitations, which made a more distributed service impossible. Thus, the monolithic architecture emerged, where the application was composed of one interconnected entity that performed all the functions

But what if we could take the modular mindset a step further? What if we could break down the application into smaller, independent services that communicate with each other? This is the idea behind microservices, an architectural style that has gained popularity in recent years. Thanks to the decrease in hardware costs and the rise of cloud computing, microservices have become a viable option for many businesses.

77% of businesses have adopted microservices, with 92% experiencing success with them. In this article, we explore the benefits and challenges of monoliths and microservices, along with the approaches of both.

Cloud vs Mainframe?

Before we dive into microservices, let’s first understand the infrastructure: the cloud and the mainframe. These are the two most common ways of hosting and running applications.  There are pros and cons to each, so it’s important to assess the application’s long-term goals, considering the various merits and demerits of both.

Cloud services are internet-based platforms which provide similar functions to mainframe infrastructure but are hugely customizable. Cloud services offer high flexibility and scalability, as they allow users to access and adjust the resources on demand. They also have high availability and reliability, as they often boast 99%+ uptime. Moreover, cloud services are cost-efficient, as they only charge users for the resources they use, and they do not require dedicated staff or infrastructure to maintain. Cloud-based services also offer improved application monitoring and AI solutions to enhance their operations and decision-making processes. These technologies offer valuable insights and optimizations that can significantly benefit a business. However, they do have some drawbacks, such as security and privacy risks, vendor lock-in, and performance variability.

Mainframe systems are essentially large, powerful computers that process enormous amounts of data and transactions. Mainframe systems are known for their unparalleled security and reliability, as they have built-in mechanisms to protect data and prevent failures. They also have high performance and efficiency, as they can handle complex and intensive workloads with minimal resources. However, mainframe systems also have drawbacks, including high initial setup costs, a lack of flexibility and agility, and difficulty in finding and retaining skilled staff.

Businesses often face the dilemma of having to choose between mainframe and cloud computing for their operations. However, there’s a middle ground that combines the strengths of both approaches: The Modern, Cloud Connected Mainframe approach.

To help work out which applications belong where, it helps to picture your business as a house. The mainframe is the sturdy foundation, providing unparalleled security and robust data processing capabilities. It’s like the basement where you store your most valuable possessions, keeping them safe from external threats.

Cloud on the other hand is flexible, allowing you to change and extend your house to add more as required. It offers scalability, agility, and cost-efficiency, allowing you to expand your operations rapidly without the constraints of physical infrastructure.

Now your house has a strong foundation (mainframe) and multiple floors (cloud services) seamlessly integrated. This is the essence of a hybrid mainframe/cloud approach. It enables you to leverage the security and reliability of the mainframe for critical operations while harnessing the agility and scalability of the cloud for dynamic workloads and innovations.

Breaking Down Your Monolith

If you are using a monolithic architecture, you might be wondering how to transition to a microservices architecture. The first step is to identify the different services or functions your application performs and how they interact with each other. For example, if you’re running a fast-food application, you might have services such as orders, restaurants, delivery, payments, and users. These services are currently bundled together in one application, with one database serving all the services. Here is a diagram of a monolithic fast-food application.

There are some advantages to the monolithic design, such as simplicity and consistency. Everything is consolidated in one place, making it easier to understand the overall picture and how services interact with each other. This simplicity results in fewer deployed resources and a straightforward infrastructure that is easy to manage.

This simplicity also aids in resolving any bugs in the application, as it provides a clearer understanding of which service is causing the issue and allows for better tracking of the flow of operations.

However, there are also some drawbacks to the monolithic design, such as rigidity and fragility. If you want to deploy any fixes or updates to any service, you must redeploy the whole application. This can make organizing deployments tricky, especially as the business grows. A bug fix might have to be bundled into the next release of the whole application, making reacting to bugs a slower process.

Moreover, if a particular service is receiving an increased workload, it is impossible to scale just that service to deal with that workload. The entire application must be scaled instead, which can lead to increased costs as services are being given resources they do not need.

Similarly, if there is a problem in one service, every service suffers. If a service or the database encounters a fault, the entire operation grinds to a halt. Additionally, while having all the services in one place aids finding bugs, this is only a prominent advantage in smaller monoliths. Over time, as the application grows, this adds to the confusion and complexity. There is much more to maintain and test.

If you want to break down your monolith into microservices, you need to separate the different services or functions into independent units that communicate with each other. This way, each service has its own codebase, database, and deployment process. Here is a diagram of a microservices fast-food application.

Building Up Your Microservices

Once you have your microservices, you need to decide how they will interact with each other. There are two main ways of communication between microservices: HTTP requests and messaging. With HTTP requests, services talk directly to each other using standard web techniques. It’s like calling someone on the phone – you make a direct connection and they, or in this case the service, need to be available to pick up.

Messaging is like sending an email instead of calling. Services communicate through a middleman, like how you might trust Outlook to deliver your email. Messages do not need to be actioned immediately either, making this approach much more flexible than HTTP requests. Like email, messages can also be sent to multiple recipients. For example, the event of the user making a payment occurs. This is something the delivery and orders services would need to know so the event can be sent to both simultaneously. This is not something that is possible with HTTP requests.

In the end, it’s about picking what works for your application and business needs. Use HTTP requests for simplicity, messaging for flexibility, or a mix of both for the best of both worlds.

Transitioning to a microservices architecture also presents an opportunity to evaluate the data being utilized and identify its usefulness. This allows for the elimination of any unnecessary data, optimizing resource utilization and improving system efficiency.

In the example above, I’ve introduced a hybrid of both HTTP requests and messaging for communication. We have our app interacting with each service directly through HTTP requests, which triggers the services to interact with each other through messaging. This allows for quick communication with the app and end user, but also allows the decoupled services to communicate with each other behind the scenes.

Additionally, each service now has its own database. The efficiency gain of having dedicated databases offsets the increase in cost of having multiple databases, especially as storing and interacting with data is inexpensive. Each database can be tailormade to the service, each service can store, manage and retrieve data in completely different ways.

This segregation is what eliminates single points of failure. For example, if the user service or database experiences an issue or an outage, the end user may not be able to see their profile details but can still place an order. In the monolith example, this could have resulted in a loss of business.

Beware the Distributed Monolith

As your microservice ecosystem grows, care needs to be taken not to fall into creating monoliths again. This is often referred to as a “distributed monolith”, where your small, independent services over time become monoliths in themselves. This is often much worse than a single monolith, as you end up with all the disadvantages of a monolith while also having multiple of them which are all interlinked.

To avoid this, make sure to properly assess additions to microservices. Sometimes, the addition is needed and will fit in perfectly in the current architecture. However, if you feel the service is straying from its original intended purpose, or perhaps you’re planning a great new expansion, it may be time to create a new microservice. For example, what if we wanted to expand our application to also deliver groceries to users. It might be easy to add supermarkets as “restaurants” as they function in a similar manner to begin with but what happens if the way they operate starts to differ? We can easily end up down the road of “just one more feature” and end up with a distributed monolith.

Conclusion

To sum up, the choice between mainframe, cloud, monolith and microservices depends on various factors, including scalability, flexibility, security, and cost. Each approach has its own merits and challenges, so it’s essential to evaluate your specific needs and long-term goals when making architectural decisions.

Mainframe systems are ideal for applications that require high security, reliability, and performance, but they are costly, inflexible, and hard to maintain. Cloud services are ideal for applications that require high flexibility, scalability, and cost-efficiency, but they pose security and privacy risks, vendor lock-in, and performance variability. Microservices are ideal for applications that require high agility, modularity, and innovation, but they introduce complexity, coordination, and testing challenges.

Whether you opt for a monolithic architecture or embrace microservices, the key is to design a system that aligns with your business objectives and enables future growth and innovation.