Batch indexing
xConnect increments a sync token in the storage layer each time a batch is submitted to xConnect. If the client’s SyncToken property is set when a search is performed, results are only returned when indexing has caught up to at least that number.
The guarantee that all batches have been indexed when search results are returned is local to the client instance. If you submit data to xConnect in parallel threads, client A might return search results before client B.
The latest sync token is returned to the client alongside the results of the batch:
await client.SubmitAsync();
var searchSynchronizationToken = client.SyncToken;
Change tracking is not enabled for device profiles.
If you submit 5 batches of 10,000 contacts, indexing is complete for those 10,000 contacts when the indexing worker has indexed changes up to and including the fifth batch. This batch is represented by a specific sync token. To confirm that indexing is complete for those 50,000 contacts:
-
Submit five batches.
-
Save a reference to the
SyncTokenthat is returned with the final batch. For example, List Manager stores sync tokens in the list definition items. -
In the code that checks whether or not indexing is complete, set the
SyncTokenproperty to the saved token. -
Perform a dummy search using this client. Results will only be returned if indexing has caught up to the sync token that was supplied.
It does not matter what you search for - no results will be returned until indexing has caught up to at least the number specified. If the SyncToken is null, results are returned immediately - the client does not wait for indexing to catch up to a particular sync token.
Scenario: Saving and using a SyncToken
In the following scenario, xConnect client A adds a large number of contacts and subscribes them to a new list by setting the ListSubscriptions facet. On a separate page, xConnect client B is using contact search to return all contacts in that list. However, client B’s SyncToken property is not set - therefore, results will be returned before indexing is complete and the results count will not be accurate.
To ensure that results are only returned when indexing is complete, save the SyncToken returned to client A. For example, you might maintain a SQL table of list IDs and the sync token from the latest batch of contacts to be added to that list:
|
Job ID |
Subscription List ID |
Sync Token |
|
1 |
a256fe54-9915-4779-afe1-511b74795dde |
6e26fe54-9915-4779-afe1-511b74795dde |
Before performing a search, set the SyncToken property to the saved value as shown. Results will not be returned until indexing is complete.
using Sitecore.XConnect;
using Sitecore.XConnect.Client;
using Sitecore.XConnect.Collection.Model;
using System;
using System.Linq;
namespace Documentation
{
public class ChangeTracking
{
// Async example
public async void ExampleAsync()
{
using (XConnectClient client = Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.GetClient())
{
// Manually set sync token
client.SyncToken = Guid.Parse("6e26fe54-9915-4779-afe1-511b74795dde").ToByteArray();
try
{
var listId = Guid.Parse("a256fe54-9915-4779-afe1-511b74795dde");
var query = client.Contacts.Where(c => c.ListSubscriptions().Subscriptions.Any(s => s.ListDefinitionId == listId));
// Will only return results when indexing is complete
var results = await query.GetBatchEnumerator();
}
catch (XdbExecutionException ex)
{
// Handle exception or timeout
}
}
}
// Sync Example
public void ExampleSync()
{
using (XConnectClient client = Sitecore.XConnect.Client.Configuration.SitecoreXConnectClientConfiguration.GetClient())
{
// Manually set sync token
client.SyncToken = Guid.Parse("6e26fe54-9915-4779-afe1-511b74795dde").ToByteArray();
try
{
var listId = Guid.Parse("a256fe54-9915-4779-afe1-511b74795dde");
var query = client.Contacts.Where(c => c.ListSubscriptions().Subscriptions.Any(s => s.ListDefinitionId == listId));
// Will only return results when indexing is complete
var results = query.GetBatchEnumeratorSync();
}
catch (XdbExecutionException ex)
{
// Handle exception or timeout
}
}
}
}
}