Sync Types

Overview

Currently, Realm has two modes of sync: Full and Query-based. You probably want to use Full Sync. Query-based sync is not recommended for production scale apps as it does not support a large number of concurrent users. This document will help you understand the differences between the two types.

Full Sync

Full Sync is analogous to fetching a file at a particular path - your mobile client will get all data contained within the realm found at that URI path. This makes Full Sync ideal for apps that need to scale to tens of thousands of concurrent connections or more.

Full Sync is ideal for scaling out the ROS system horizontally because realms are pinned to individual sync-workers. As each realm is a compartmentalized bucket of data where the user has either all or nothing, it is important to define from the beginning how the app will split the data into these buckets (explained in greater detail here). This requires some thinking as at this time there is not automatic way to link objects across realms which will not offer automatic conflict resolution across realms (though retains within each realm). Thus the ability to query across multiple realms will have to be achieved manually via an algorithm analogous to map-reduce.

In general with sync the conflict resolution algorithm is CPU intensive and this can be evident if there are many writers submitting inserts of objects or sending updates on the same realm concurrently. Every use case is different but we recommend a team size of 20-30 writers per realm for a shared realm for real-time performance. An ideal use case for full-sync is for a single writer, such as a backend application which inserts data into a global Realm which has read-only permissions associated to all mobile users - this can easily scale past tens of thousands of concurrent users. Another ideal use case for full-sync is where each user has their own individual or private realm; for instance, this could be used for user account info or preferences, or for an individual user’s shopping cart as part of a retail app. As long as only the user and/or a backend application interacts with the mobile app the performance is ideal and scales linearly.

As your app grows there may come a time to start creating clones of your Full Sync read-only realms. Let’s say you had a read-only catalog realm for your retail and shopping app. To do this you will need to create a clone of the catalog realm so there are two - /catalog1 & /catalog2. You can do this by either performing a file copy and rename if you are self-hosted or if you are using Realm Cloud by building a script that copies the data out of one and into the other. Anytime your backend app would need to update the catalog, it should now be aware to open and update both catalogs. Therefore on the client-side you will need to add app code that determines which catalog the sync-client should connect to - a simple check of the first character of a random GUID of each user would enable you to divide half to one realm and half to the other. This would require an app update in the event that further scale-out is needed. A more flexible method is to have the server assign you a url. This can be done though a per-user realm and allows you to transparently scale out with more replicas as needed.

Query-Based Sync

Query-based Sync is not recommended. For applications using Realm Sync, we recommend Full Sync. Learn more about our plans for the future of Realm Sync here.

Query-Based Sync is analogous to a GET request in a REST API. The mobile client will submit a query to the backend ROS where the Realm Object Server then processes this query against a master or reference realm and the result is synced down to the client. Unlike REST the query and sync persists, ensuring that all synced objects are 'live' and change in lock-step with the server. The design of query-based sync makes the queries easy to reason about because the developer can make a query on a parent object and all child objects will be automatically pulled in. This is an advantage when designing a new application where you are controlling the data model from inception. The backend architecture is also easy to understand as everything goes into a single realm (or a small number of realms) which is analogous to one large database. This makes it doubly straightforward for an admin portal or BI tool to open the reference Realm and run queries across the entire dataset.

However, as queries are run on the backend server first and then synced down to the client, there is processing overhead associated with query-based sync when compared to Full Sync. Thus query-based sync will consume more physical resources such as memory, CPU, and disk, and this overhead depends on the structure and complexity of the queries used. For example the big-O notation or efficiency of a query directly determines how fast query-sync data can be propagated to the sync-client. For instance, a query that uses the IN predicate to match against an array of fields will be far less performant than a single predicate against a list of field values.

We always improve the performance of the Realm system, though currently (late 2019) do not recommend query-based sync for production, especially not for use cases where a large number of clients will be connected concurrently. Feel free to reach out to your Realm technical contacts to confirm your app design. Sometimes small tweaks make a big difference.

Summary

In summary, full sync provides a more performant alternative for applications where larger scale or lower latency are important, but it adds a few more constraints on working with your data since it will need to be partitioned into numerous self-contained Realms. Query-based sync allows for more simplicity and flexibility when designing and querying your Realm architecture, but this comes at a cost of higher processing overhead at the server. Generally, it is recommended to use Full Sync. That said, we are happy to connect with you and give you input to structure the application for an optimal user experience.