Before we learn something new we kinda wanna need it for something first, so let’s see why we need “queueing systems” and what are they.
Divide and conquer is the way to go when we need some system to do a whole lot of work and fast. We divide the nature of the tasks and conquer them by making separate modules dedicated to that task specifically, we make them send each other messages to pass data or context. When we add a queue to the equation we can be sure that modules with tasks that take the time can be optimized by simply letting them do one task at a time without blocking the work of the other modules. So this is about two or more different software components communicating without the need for them to be on the same system, written in the same language or platform and still creating fine modular and complex systems.
So let’s assume we wanna create a website that converts something from one format to another. In this case, we want to create a system with one part that takes care of the people that want to convert something ( the interface/ the page ), another part for converting data and another part for distributing the converted data. For this to work properly we will add two queues, one for all the data that needs to be converted and another for the converted data that needs to be distributed and voila our system is done! Now the three modules can do their thing and communicate with each other to pass the load. We separated a different kind of work that the system will do AND we’ve made it capable to handle a big load. Why? Because when 100 people (hypothetically) open the same page our system will simply add their data to a queue, as opposed to block and try to figure out which convert to do first and what to do with the other 99.
Ok, so we saw why we need it, now let’s see what is it.
A message queuing systems are systems that handle message queues. They contain queues, messages that go into and out of queues. It must go by some protocol so that the modules on both sides of a queue would understand each other and most importantly it contains a message broker. The message broker is like the brains behind the whole operation, it takes messages from senders, stores them, puts them onto queues, takes them out of the queue and distributes them to receivers. Meanwhile validating and transforming the messages from the language of the sender to the language of the receiver even partition the messages into smaller parts if the message is too big for the queue. It is the real deal! So in the future when we talk about queueing systems we will actually talk about different message brokers.