An introduction to Forest Bus
Forest Bus is a standalone message bus / distributed commit log that allows clients to send messages to topics and receive messages from topics. This simple mechanism allows for a number of uesful design patterns:
|Loosely coupled integration||Software components can trigger activities in other sub-systems by publishing a message to a Forest Bus topic. Any errors or downtime in other sub-systems are isolated from the publisher.|
|Real time reporting||Publish business events to a Forest Bus topic as they happen, rather than producing multiple hourly or daily extracts. This isolates operational systems from analytic reports, while enabling real time reports to be created.|
|Operational Reslience||Clients specify which message to start reading from, allowing them to catch-up after downtime and reprocess data after bug fixes or feature enhancements|
Once a Forest Bus cluster is up and running (see the quick start guide) use forest-bus-admin to create a topic that will hold messages. Producers use one of the Client libraries to connect to the server(s) that form a particular cluster, and send batches of messages to the topic. Multiple clients can send messages to the same topic at once.
When sending messages to a cluster, Clients can choose to wait for the cluster to confirm the messages are committed before returning. If message delivery isn't critical then messages can be sent without waiting for the commit to take place. The Forest Bus cluster will return the list of IDs created for a batch of messages - these IDs uniquely identify the message within the topic.
Subscribers that are interested in messages in this topic also connect to the cluster using a Client library. Each subscriber is responsible for recording the last message index (ID) that they processed. Multiple clients can read from the same topic and can choose to re-read the same messages again if required. Clients can either wait for new messages to arrive on a topic and read them in real time, or they can connect periodically and read those messages that are available.
Forest Bus can be run as a single server or as a cluster of nodes that provide resliency. When multiple servers are used they coordinate the distribution of the commit log using the RAFT algorithm, essentially this means that: