Looking for something else? Check the categories of Messaging:
Apache Kafka Apache Pulsar RabbitMQ
If not, below you can find all articles belonging to Messaging.
If you're working with Katka, rebalancing is maybe a word the most commonly met. It's also an important word because it helps to ensure correct message consumption.
Since Kafka is a distributed system, it naturally has to coordinate its members somehow. The synchronization mechanism is ensured by coordinators.
Until now we've almost used only default configuration values. In this article we'll see some of configuration possibilities more in detail.
An intrinsic part of each messaging system are messages. After learnt previously about producing and consuming messages, it's good moment to see what these messages really are.
After presenting consumers in Kafka, it's a good moment to pass to producers.
In one of previous articles we've discovered how to implement Kafka to publish/subscribe pattern, ie. one consumer per consumer group. This time we'll describe another aspect of Kafka consuming, messages queue.
When you need to write about messaging, you'll certainly meet a dilemma about 'which part, consuming or producing, describe first ?'. I decided to start with simple consumers. By 'simple', I mean only 1 consumer per group. Thanks to that we can avoid the point of partition sharing.
As already told during quick introduction, Apache ZooKeeper is an inseparable part of Apache Kafka. Knowing what happens between these two actors is important to start to work with Kafka correctly.
In RabbitMQ we can use temporary messages and queues. We can create a queue which is automatically deleted within some time of inactivity.
Sometimes messages are not equal between them. Some are more prioritized than the others and they should be handled first. RabbitMQ implements also a feature to deal with the problems of priorities.
RabbitMQ is not only one-direction system. Queues can communicate between them, just as two people talk together.
Sometimes consumers can be overloaded by the number of messages to handle. RabbitMQ helps to control this number with the parameter called prefetch.
RabbitMQ Java API brings a lot of methods called the same but accepting different parameters. Very often, these parameters are flags which can apply as well on messages, as on exchanges level.
It's not a surprise, transactions slow down RabbitMQ performances. It's the reason why a suggested way to manage guaranteed delivery is publisher confirms concept.
Recently we discovered some theoretical concepts about Apache Kafka. So it's a good moment to discover Java API.
This article starts a small succession of posts about Apache Kafka, considered often as one of solutions to data ingestion.
At the first look, transactions appears as an element strictly related to databases. However, we retrieve them also in messaging.
Messaging reserves surprises, especially when things go bad. It can occur when, for example, message time to live is exceeded or when there are no more space in the queue to handle new messages. To avoid messages loosing, a special kind of exchange, dead-letter exchange, exists in RabbitMQ.
Persistence is an important part of RabbitMQ configuration. Nobody would like to lose key business messages because of temporary power loss or human manipulation error.
Exchanges are key element in RabbitMQ messages transmission. Every message is firstly routed to these exchanges and only after it's dispatched to appropriate queues. A queue can't be declared without associated exchange.
