Realm Sync - The Details

Realm Sync is designed to empower offline-first use cases with shared data between users in disparate locations. Each user or application will have their own copy of the realm sent from the master copy on the Realm Object Server (Realm Cloud). Because users can be editing the same data when they’re offline at the same or different times, Realm Sync is designed with a robust conflict resolution engine that is guaranteed to always deterministically converge to the same result, both on the server and all clients. In this way Realm Sync is strongly eventually consistent.

In order to achieve this strong eventual consistency, Realm Sync keeps a running log of all transactions that occur on a realm then sends these logs to the Realm Object Server. When the Realm Object Server receives the logs, it triggers the Realm Sync merge algorithm, which uses vector clocks and timestamps to infer causality and place the master transaction log of that realm in the correct order.

In the simplest example of the merge algorithm, the sync-client sends a transaction that appends an item to the list, the sync-server receives the transaction and sees that the sync-clients list state is the same as its master list state - an append-only operation. In the most computationally expensive use case, multiple sync-clients have been working offline for a long time, all editing the same list of objects, deleting, inserting, and changing the order of the list. The sync-server must compare timestamps and causality of each transaction from each sync-client to determine the final state.

Transaction logs are created by interacting with objects within each Realm SDKs’ write transaction API. Realm Sync is always available, so when a developer performs a write transaction:

  • the state of the realm is changed in an ACID compliant way, and

  • a transaction log entry is generated to account for this change.

A single transaction can contain one or many different instructions to the realm database which are then translated to what we call changesets. The Adapter API emits instructions that correspond to these changeset log entries - you can see a list here: build-your-own-adapter

These changesets are then sent up to sync-server or down to the sync-client in a transactional manner. While the sync protocol may split up large transactions into chunks simply to make the network transfer of data more efficient, the receiving side will hold all the changesets in memory until the full transaction is received before committing to disk. In this way, atomicity is preserved.

Once the transaction is received by the sync-server, the merge algorithm is applied to create the master transaction log of the realm, and committed to disk. The sync-client will then receive acknowledgement that it can then proceed to the next transaction. The sync-client will then prune the instructions that made up the previous transaction and iterate its history version to match the version now present on the sync-server - this preserves disk space while maintaining the current state. On the other hand, the sync-server needs to preserve the master record of all instructions that have occurred since the realm was created in case an old offline client comes back online that needs to have its transactions merged. Over time your realm on the sync-server will grow as your app generates transactions. Keep in mind that instructions are sent and received in the order they were emitted which can lead to slower performance for new clients that connect to an old realm, because they are downloading all of the instructions that may have occurred during the lifetime of that realm.

This is why it is important to regularly prune your transaction logs with the sync-server's historyTTL setting which will trim superfluous transaction logs. For Realm Cloud, the TTL is set to 30 days by default.

Sync Modes

Query-based Sync is not recommended. For applications using Realm Sync, we recommend Full Sync. Learn more about our plans for the future of Realm Sync here.

When opening the realm, you can choose between two sync modes:

  • Full Sync, which replicates the entire realm; and

  • Query Based Sync, which replicates only a subset of the objects specified with a set of queries. While this allows you to use sophisticated permissions rules on specific objects within the realm, it supports significantly fewer concurrent users. Therefore, query-based sync is generally not recommended for production workloads.

It is no problem to open both query-based reference realms and fully-synced realms within the same app. Thus it is possible to mix and match the types of realms you use on any single device to adapt to the use-case. The only restriction is that each realm has to be designated as either a reference or full realm on creation and will stay as such from then on (a full realm can later be upgraded to a reference realm though).