Apache ActiveMQ is a mature messaging middleware. It is offered as a managed service by AWS, which makes it a good choice for low latency messaging needs for applications on AWS. This post explains how message delivery guarantees can be achieved with ActiveMQ and STOMP clients.

Concepts

Persistent Messages

Persistent messages will not be lost between broker restarts.

SEND {destination: 'some-topic', persistent: true}

Message Expiry

To expire the messages, send an expires header, the value of which is the UTC UNIX timestamp in milliseconds after which the message is discarded. An expired message is not routed by a broker, and clients should not process expired messages either.

SEND {destination: 'some-topic', expires: 1539580330000}

Durable Consumers

Durable consumers receive messages lost between consumer restarts or reconnections.

CONNECT {'client-id': CLIENT_ID} SUBSCRIBE {'activemq.subscriptionName': SUBSCRIPTION_NAME}

• CLIENT_ID should be unique to a consumer and must be the same across consumer restarts.
• SUBSCRIPTION_NAME needs to be same across consumer restarts.

Explicit Consumer ACK

Consumers can acknowledge each message explicitly instead of an auto acknowledge mode where the broker assumes the message is received after sending it to a consumer. Unacknowledged messages are sent again by the broker. There are two ack modes -

• client: acknowledges all messages received so far.
• client-individual: acknowledges an individual message received.

SUBSCRIBE {'ack': 'client'}

In the subscribe callback, call message.ack() to acknowledge receipt when you have processed the message.

Retroactive Consumers

A retroactive consumer receives messages sent to a topic before the consumer connected with the broker. To specify a consumer as retroactive, use the activemq.retroactive header.

SUBSCRIBE {'activemq.retroactive': true}

ActiveMQ XML

<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
    <destinationPolicy>
        <policyMap>
            <policyEntries>
                <policyEntry topic=">">
                    <subscriptionRecoveryPolicy>
                        <fixedCountSubscriptionRecoveryPolicy maximumSize="1000000" />
                    </subscriptionRecoveryPolicy>
                </policyEntry>
            </policyEntries>
        </policyMap>
    </destinationPolicy>
</broker>

For complete list of subscriptionRecoveryPolicy, see Active Documentation for Subscription Recovery Policy.

Message Delivery Guarantees

There are 3 kinds:

• At least once: messages are guaranteed to be delivery at least once. The consumer must take care of duplicate messages.

• At most once: messages are guaranteed to be delivered at most once. The producer and consumer must take care of lost messages.

• Exactly once: messages are guaranteed to be delivered exactly once. This is a hard problem and can most likely be solved under special circumstances. This post will not examine this guarantee.

At most once

In order to implement this guarantee, it must be ensured that any message successfully sent by a producer to a broker is delivered to one or more consumers at most once. In this scheme, it is acceptable for a consumer to not receive a message, but it is not acceptable to receive it more than once.

The key concepts are:

• Persistent Messages
• Message Expiry

Producers need to make sure not to send duplicate messages. The messages must persist across a broker restart. Message expiry should be added to prevent delivery of expired messages when the broker starts.

At least once

In this scheme, a consumer must receive a message at least once. This means it is acceptable to receive duplicate messages. The key concepts are:

• Persistent Messages
• Durable Consumers

Messages need to be persistent across a broker restart. A durable consumer will receive any messages it may have lost across a restart.

Conclusion

It is important to agree on the delivery guarantees of a messaging middleware before building systems around it. Different systems may need different guarantees, and an improper or insufficient configuration of the messaging middleware will make developers build unnecessary "defences" in their systems.