Messaging: There are at least 2 common paradigms to resolve communication between different services in the current software architecture landscape: synchronous and asynchronous.
The first form of communication can be implemented using the most common model of calls between services, which is the exchange of HTTP requests. The second, in turn, is more often implemented using messaging brokers.
Both communication patterns have their pros and cons – and it is about these tradeoffs that we will talk a little bit here.
Synchronous Communication: Conventional HTTP Model
The HTTP protocol has been with us for decades. Therefore, in software development, it is already widely known and is the fundamental piece for accessing all kinds of content on the internet.
In practice, synchronous communication includes all forms of communication in real-time; that is, responses and interactions are immediate, like a face-to-face conversation, a video call, or even a phone call.
Among the pros that the synchronous inter-service communication model using HTTP can offer, we can highlight:
- Ease of implementation;
- It is well suited to most communication scenarios between services that do not demand a high rate of competition;
- It has a vast universe of documentation, libraries, and examples to inspire and implement.
On the other hand, as points of attention, it is worth mentioning:
- Generally speaking, it is not a model that scales to high levels of concurrency (> 1K TPS, for example);
- Requires implementation of retries by the person sending a request;
- By default, it is synchronous, so the entire flow can be interrupted if there is degradation on the receiving side.
Asynchronous Communication: Using Events And Messaging
In this model, an application-independent structure comes into play: messaging brokers.
This type of middleware offers communication and data structures that allow different services to talk to each other in an asynchronous, secure, replicated, and highly fault-tolerant manner.
In this communication paradigm – and specifically in Apache Kafka – there are some fundamental basic concepts:
- Brokers -> messaging cluster components, which allow communication with services, store, replicate and provide all data integrity guarantees.
- Topics -> data structures that store the messages produced and consumed by services. They are created in a way that makes sense for real life, for example, a topic for shopping events in e-commerce and new orders in a delivery app. Many call topics queues, but that’s for another post!
- Producers -> services that produce messages in the broker, indicating that a new event has happened. Following the example used in the topics, the checkout service of e-commerce can produce events with messages for each new purchase made in an online store.
- Consumers -> services that consume messages in the broker, starting a new process flow according to its mission. An example can be a production order service, which consumes events produced by the checkout of e-commerce, starting the process. It can still produce a new event for other services to perform specific production, logistics, and tax activities. And, already answering the question that popped into your head, services can be producers and consumers of different topics!
But what are the pros of using Apache Kafka or a similar solution?
- By nature, the messaging cluster is asynchronous, distributed, and scalable, making it fault-tolerant.
- It allows enormous visibility of the flow due to the way topics are organized and the collected metrics.
- Different teams can consume the same events to generate different information. For example, operations teams can consume order events to measure throughput on the platform and graph potential bottlenecks, big data teams can consume the same events to generate business insights, and dev teams consume the targeted initially events. Your applications to move the process flows forward.
- You can integrate events generated in other data sources such as relational databases, IoT sensors, etc.
Also Read: Application Development: Why Create Yours?