Sitecore Experience Platform

Rebuild the xDB index in Azure Search

This topic describes how to rebuild the xDB index in Azure Search. Index rebuild functionality was introduced in 9.0 Update 2, but you can take steps to refresh the index in earlier versions.

9.0 Update 2

The following index rebuild instruction instructions are relevant for Sitecore 9.0 Update 2 and later. The xConnect Search Indexer uses the connection strings to create any missing indexes on startup or at the beginning of the rebuild process. There are two indexes:

  • The primary index, named xdb by default.

  • The secondary index, named “<primaryindex>-secondary”. The default name of the secondary index is xdb-secondary.

By default, you do not need a dedicated connection string for the secondary index. However, you can configure a custom connection string if you require it.

Rebuild the index

To rebuild the index:

  1. Open xConnect Collection Search role console - for example, via the Kudu Debug console. By default, the xConnect Search Indexer is hosted as a Web Job in the context of the xConnect Collection Search role.

  2. Navigate to to the folder where xConnect Search Indexer is located. For example:

    cd site\wwwroot\App_data\jobs\continuous\IndexWorker
    
    azr-rebuild-11.png
  3. Run the following command:

    .\XConnectSearchIndexer.exe -rr
    

    The xConnect Search Indexer will return Done to indicate that the rebuild process has started.

Requests are automatically directed to the active index when rebuild process is complete. You do not need to make any changes to connection strings.

Important

Rebuilding the reporting database does not trigger a rebuild of the index. You must rebuild the index separately.

Rebuild to a different subscription or index name

The rebuild process can also be used when:

When the primary index is the active index

If the primary index is the current active index, complete the following steps to rebuild to a different index name or a different subscription.

  1. Confirm that the primary index (identified by the collection.search connection string) is the current active index.

  2. Create a patch file for the <xconnect-role-root>\App_Data\Config\Sitecore\CollectionSearch\sc.Xdb.Collection.IndexReader.AzureSearch.xml configuration file and add it to the <xconnect-role-root>\App_data\config\global folder.

  3. In the patch file, specify a secondary connection string as shown:

    <?xml version="1.0" encoding="utf-8" ?>
    <Settings>
        <Sitecore>
            <XConnect>
                <CollectionSearch>
                    <Services>
                        <IAzureSecondaryWebClientFactory>
                            <Options>
                                <ConnectionStringName>
                                    collection.search.secondary
                                </ConnectionStringName>
                            </Options>
                        </IAzureSecondaryWebClientFactory>
                    </Services>
                </CollectionSearch>
            </XConnect>
        </Sitecore>
    </Settings>
  4. Open the <xconnect-role-root>\App_Config\ConnectionStrings.config configuration file.

  5. Add a new connection string with the same name that you specified in step 2 and add in your subscription, index name, and API key. In this example, the secondary connection string name is collection.search.secondary and the index name is xdb-custom-secondary:

    <add name="collection.search.secondary" connectionString="serviceUrl=https://[service].search.windows.net/;indexName=xdb-custom-secondary;apiKey=[API Key]"/>
    <add name="collection.search" connectionString="serviceUrl=https://[service].search.windows.net/;indexName=xdb;apiKey=[API Key]" />
    
  6. Repeat on any other instances of the xConnect Collection Search service role and the xConnect Search Indexer.

  7. Rebuild the index. When the rebuild is complete, xdb-custom-secondary becomes the active index. You do not need to restart the indexer before rebuilding - if an index does not exist, the indexer will create it.

  8. When the rebuild is complete, change the default connection string (collection.search) so that the primary index is recreated in the new subscription.

  9. At this point you can delete the original index and the original subscription. Make sure no roles are referencing the original subscription.

When the secondary index is the active index

If the secondary index is the current active index, complete the following steps to rebuild to a different subscription. In this scenario, you do not need to configure a custom secondary connection string.

  1. Confirm that the primary index (identified by the collection.search connection string) is not the current active index.

  2. Change the primary collection.search connection string to point to your new subscription or index name.

    <add name="collection.search" connectionString="serviceUrl=https://[service].search.windows.net/;indexName=xdb;apiKey=[NEW API KEY]" /> <!-- MAINTENANCE> -->
  3. Rebuild the index. When the rebuild is complete, xdb becomes the active index in the new subscription and xdb-secondary becomes the maintenance index.

  4. When the rebuild is complete, you can delete the old subscription that contains the old indexes.

Note

If you want to change the name of the secondary index, you should so before rebuilding from the primary to the secondary index. You cannot change the name of the secondary index whilst it is active.

Checking if rebuild is complete

There are several ways to check if a rebuild is complete:

  • Check the xConnect Search Indexer logs - Rebuilding process. Stage: Finished indicates that the rebuild process is complete.

    az-rebuild-41.png
  • Check the status of the xdb-rebuild-status document with the following query:

    $filter=id eq 'xdb-rebuild-status'
    

    A RebuildState of 6 indicates that rebuild has completed.

Checking if an index is active

The Azure Search provider uses two physical cores: primary and secondary. At any time, one core is actively used for search and indexing of live data, and the other is used for maintenance operations such as rebuild. After a rebuild, the physical cores are swapped but not renamed.

To know which physical index is currently active, follow these steps:

  1. Open one of the indexes in Azure Search.

  2. Perform the following search:

    &$filter=id eq 'indexconfiguration'
    az-rebuild-21.png
  3. In the results field, find the “isactivecore” field. If the index is active, this setting will be true:

    az-rebuild-31.png

Note

You can delete or clean up the index that is not active. It does not matter if it is the primary or secondary index. Restart the xConnect Search Indexer to re-create the index or wait until the next rebuild.

9.0 Update 1

There is no index rebuild process available for Azure Search between 9.0 Initial Release and 9.0 Update 1. However, you can use the following approach to refresh content of the index:

  1. Stop the indexer web job.

  2. Drop the index.

  3. Restart the indexer.

This basic refresh process relies on the xDB Collection provider’s change tracking retention period being high enough to maintain a record of all incoming changes until the indexer has processed all existing data. Note that setting a very high change tracking retention period has detrimental effects on performance.

Important

Rebuilding the reporting database does not trigger a rebuild of the index. You must rebuild the index separately.