Sitecore Experience Platform

Tracking changes to the xDB index

Abstract

Overview of how the xDB tracks changes to contacts, interactions, and facets.

The xDB tracks incoming changes to contacts, interactions, and facets. The xConnect Search Indexer uses this data to synchronize the index with data in the collection database

Tracking changes with the xDB Collection SQL provider

If you are using the xDB Collection SQL provider, xConnect uses the SQL Server change tracking feature. The change tracking retention period determines how long changes are stored in SQL server. By default, the Retention Period setting is set to 5 days and the Auto Cleanup setting is set to true for each shard:

change-tracking1.png
  • Keep the change tracking retention period as small as possible, keeping in mind the effect on live indexing and index rebuild.

  • Enable Auto Cleanup on all shards.

  • Monitor the indexer closely.

For more information, see: xConnect Search Indexer configuration (SQL).

Important

Azure Search index rebuild is only available in 9.0 Update 2 and later. If you are using an earlier version, the index retention period may need to be set to a higher value than desired.

Impact on live indexing

The retention period affects the indexer’s ability to resume indexing after being offline. If the retention period is less than the downtime period, the indexer will not be able to index the data that was collected during the downtime. In this scenario, the index must be rebuilt.

Impact on index rebuild

An index rebuild occurs in two logical steps:

  1. Indexing of all data up until the moment when the index rebuild was triggered

  2. Indexing of all data collected after the index rebuild was triggered

If step 1 takes longer than the configured retention period, the indexer will not be able to index all data collected in step 2. In this scenario, one of the following changes must be made before triggering another index rebuild:

  • Increase the retention period on every shard

  • Temporarily set Auto Cleanup to false on every shard

Impact on performance

Each table with change tracking enabled has an internal table that stores data about each changed row. Increasing the retention period affects performance in the following ways:

  • A longer retention period results in larger internal tables. Larger internal tables affects the overall database size and may cause performance issues in a busy system.

  • Larger internal tables increases the performance impact of the Auto Cleanup process. However, disabling the Auto Cleanup process will result in even larger tables over time.

Tracking changes with the xDB Collection MongoDB provider

If you are using the xDB Collection MongoDB provider, xConnect uses a custom collection named [Changes] with a TTL index. The retention period is controlled in configuration and uses the same default as SQL change tracking (5 days):

<!--
The `expireAfterSeconds` option for the Changes collection's TTL index.
Default: 432000 (5 days).
-->
<ExpireAfterSeconds>432000</ExpireAfterSeconds>

For more information, see Configuring the xConnect Search Indexer