Tracking changes to the xDB index
The xDB tracks incoming changes to contacts, interactions, and facets. The xConnect Search Indexer uses this data to synchronize the index with data in the collection database.
Tracking changes with the xDB Collection SQL provider
If you are using the xDB Collection SQL provider, xConnect uses the SQL Server change tracking feature. The change tracking retention period determines how long changes are stored in SQL Server.
We recommend that you do the following:
-
Enable auto cleanup on all shards. This is true by default.
-
Monitor the indexer closely.
-
Keep the change tracking retention period as short as possible.
When you set the change tracking value, we recommend that you allow enough time for your IT department to be able to fix the indexer during any potential system downtime. If they can fix it during downtime, you might avoid having to rebuild the index. By default, the value is set to 5 days.
For information on how to configure the xConnect search indexer, see: Configuring the xConnect Search Indexer.
Impact on live indexing
The retention period affects the indexer’s ability to resume indexing after it has been offline. If the retention period is less than the downtime period, the indexer cannot index the data that was collected during the downtime. In this scenario, you must rebuild the index from scratch.
Impact on index rebuild
An index rebuild occurs in two logical steps:
-
Indexing of all data up until the moment when the index rebuild was triggered.
-
Recurring indexing of all data collected after the index rebuild was triggered. You can set the interval for this using
SyncLatestChangesIntervalSec. For more information, see Configuring the xConnect Search Indexer.
If step 2 takes longer than the configured retention period, the indexer cannot index the data that it collected since the last synchronization. If this happens, increase the retention period on every shard, and start the rebuild from scratch.
Impact on performance
Each table with change tracking enabled has an internal table that stores data about each changed row.
Different retention times and table sizes affect performance in the following ways:
-
Short retention times produce high performance and save space. However, if the system fails, you must rebuild the indexer.
-
A longer retention period results in larger internal tables. This increases the overall database size, which can cause performance issues in a busy system.
-
Larger internal tables makes the Auto Cleanup process slower. However, we recommend that you do not disable the auto cleanup process. If you disable the auto cleanup process, the system gains even larger tables over time.