Advice and answers from the Realtime Team

The new guaranteed message delivery mode provides an "at-least-once" delivery semantics, quite stronger than the default "at-most-once".

What's the motivation?

As it's implied, the goal of this new feature is to guarantee that all subscribers get a copy of each message sent through the channel, even if they were not connected at the time the message was sent.

This will be highly useful in Realtime apps using mobile data, where common network failures cause reconnections. Currently with the "at-most-once" mode if a message is sent when a subscriber is reconnecting, that message is lost, the subscriber won't get it. With the new mode, subscribers will still get the message when they reconnect, even if it's many seconds later.

"At-most-once" vs "at-least-once"

A messaging system implements an "at-most-once" delivery mode when the only guarantee is that each message will be delivered only one time to each subscriber, if it's delivered at all.

The first part is great, "only one delivery", but the second part not so much, "if delivered at all". This basically means that messages can be lost.

This mode is good for apps where messages contain full data updates sent frequently (not incremental updates), like dashboards or stock tickers, where only the last message is important since it holds the last known "truth". In this scenario if a message is lost the next one will contain the necessary data to correctly update the UI.

However, if each message contains only a small incremental update to the app state, like in a chat app, the "at-most-once" guarantee will become insufficient (the chat can become out-of-sync) and will force developers to "hack" some workarounds, like frequently syncing the chats querying the app backend or more optimally, syncing when the app detects a Realtime reconnection.

The "at-least-once" simplifies this since it offers a stronger guarantee: all messages will be delivered at least once. This means each chat participant will always receive each chat message, independently of it's connection state at the moment the message was sent. Frequent re-syncing "hacks" are no longer required.

Why "at-least-once" and not "exactly-once"? Honestly, most times the new mode will be "exactly-once and in order", but under some abnormal circumstances (like severe network failures) a subscriber can receive the same message more than once. Your app should be aware of this and detect these unfrequent (but possible) duplications. Hopefully, as I'll explain below, each message has a unique identifier that you can use to easily identify duplications.

What can cause message duplication?

The Realtime Platform knows that a subscriber has successfully received a message when the subscriber sends an acknowledge (this happens automatically inside the Realtime SDK). If there's a severe network failure at the moment this acknowledge is being sent, the Realtime Platform may not receive the acknowledge in due time and will consider that message was lost, adding the message to a "virtual" subscriber buffer. This buffer will be flushed and the message sent again as soon as possible, possibly causing a duplication. But duplication is a small price to pay for such a higher degree of delivery guarantee, right?

Which API methods implement the guaranteed mode?

Two methods were added to the Realtime Messaging API: subscriberWithBuffer and publish.

The subscribeWithBuffer method will establish a buffered subscription of a channel uniquely identified by a subscriberId parameter. 

Through the subscriberId the platform will know exactly which was the last message successfully delivered to that specific subscriber. To avoid "strange" behaviours in your app you should guarantee that each subscriber will always use the same unique subscriberId throughout the app lifecycle. Your app's internal userId is a good option since it will never change during the user "lifetime".

On the sender side you must use the new publish method. This method "simply" adds the message to the channel sequence log, gives it an unique sequenceId and finally routes the message to the subscribers. 

The sequenceId will never be repeated for any other message and it will allow easy duplication detection.

Most times the messages sent with publish will be delivered in real-time, only a few milliseconds after sending, but in some extraordinary case, like a reconnect, the non-delivered (buffered) messages will be delivered later and in strict order, when the subscriber is ready again to receive them (we call this process subscriber re-sync).

Another cool feature of the publish method is the message time-to-live (TTL) parameter. This will allow automatic non-delivered message expiration from any subscriber buffer. The minimum expiration interval will be 1 second and the maximum will vary according to the Realtime plan being used by the app.

To sum it up, when you send a message with the new publish method, you can rest assured that message will be delivered in order to each subscriber that used the subscribeWithBuffer method. As simple as that!

Any limitations?

The first limitation is related to the time-to-live (TTL) maximum value. It will be 2 minutes for the Free plan and 72 hours for the paid plans.

Another global limitation is related to publish frequency. The Realtime Platform will enforce a 10 messages per second rate limit for each client connection. It will allow some bursts above that rate limit but messages will be buffered and sent with a 10 messages per second throttling. If the pending messages buffer exceeds 100 messages, newly published messages will be discarded and a client exception will be throw'ed (you can easily handle this exception and resend the message later when the buffer is flushed).

If you need to publish messages with a higher frequency you must use several client connections to distribute message publishing or the use send method instead (without delivery guarantee).

What about pricing?

The new guaranteed delivery mode uses the same Realtime pricing and plans as the default "at-most-once" mode. However, every message delivered to a subscriber from the message buffer (not delivered in real-time but during a subscriber re-sync procedure) will count as two (2) messages towards the monthly usage.



Did this answer your question?