Posts

Showing posts from May, 2020

Rebalancer for RabbitMQ, SQS, etc.

Image
Rebalancer (forked from Jack Vanlightly  and improved)      Create Kafka style consumer groups in other technologies. Rebalancer was born of the need for consumer groups with RabbitMQ. But Rebalancer is completely technology agnostic and will balance activity over any group of resources across a group of participating nodes. Use cases Create Kafka-like "consumer groups" with messaging technologies like RabbitMQ, SQS, etc. Consume a group of resources such as file shares, FTPs, S3 buckets between the instances of a scaled out application. Single Active-Consumer / Active-Backup Create an application cluster that consumes a single resource in a highly available manner. The cluster leader (Coordinator) consumes the single resource and the slaves (Followers) remain idle in backup in case the leader dies. Balancing assignment of multiple resources over multiples nodes Consuming from a single resource with backup nodes in case of failure Concepts ...