Scaling and configuring the xConnect Search Indexer

This topic explains how to configure the xConnect Search Indexer. In addition to the configuration settings discussed here, you can use additional settings to configure how it operates and how it interacts with the search provider you choose.

Scaling the xConnect Search Indexer

The following table describes the ways you can scale the xConnect Search Indexer:

Scaling option

Supported

Notes

Combined instance

Partially

The xConnect Search Indexer is a worker role and cannot be combined with any other XP services. For example, you cannot combine the xConnect Search Indexer and the xConnect Search Indexer into a single Windows Service or WebJob.

Standalone instance

Yes

You can deploy xConnect Search Indexer as:

  • A Windows Service that runs on the same server as the Marketing Automation Operations role or any combination that includes this role, such as the XP Single topology.

  • An Azure WebJob that runs in the same context as the xConnect Collection Search web application or any combination that includes this role, such as the XP Single topology.

High availability and failover

Partially

You can fail over to a passive instance of the xConnect Search Indexer or run multiple active instances.

Note

The xConnect Search Indexer does not have a health check API endpoint. You must use an external tool to monitor the WebJob or Windows Service health.

Horizontal scaling for load distribution

No

You cannot run multiple active instances of the xConnect Search Indexer. Each instance would process the same contacts and interactions and put unnecessary load on the xConnect Collection Search service and the xDB Collection database. Keep the following in mind when scaling the xConnect Collection Search service:

  • In Azure, the xConnect Search Indexer runs as a singleton. This ensures that the indexer only runs on one instance of the parent web app (the xConnect Collection Search service) even if you choose to scale out.

  • The xConnect Search Indexer is bundled with the xConnect Collection Search service by default. If you use the Sitecore Installation Framework (SIF) to deploy multiple instances of xConnect Collection Search service to multiple servers, you must manually disable all but one indexer. You can do this by stopping or uninstalling the Windows Service.

    Note

    If you choose to stop the indexer Windows Service, remember to set the startup type to disabled.

Performance tuning

Consider the following performance tuning options:

  • Change tracking may cause high CPU usage by the underlying collection database (SQL, MongoDB). If this happens, consider modifying the tracking retention period.

  • If the xConnect Search Indexer is consuming too much memory, consider reducing the SplitRecordsThreshold setting until memory usage reaches a reasonable level. SplitRecordsTreshold x number of cores determines the number of records that are loaded into memory.

  • If reducing SplitRecordsThreshold does not improve memory usage, consider reducing the NumberOfChangeVersion setting.

  • If the indexer is consuming too much memory during index rebuild, consider reducing the ParallelizationDegree and BatchSize settings. Keep in mind that new data continues to be indexed during an index rebuild.

See Configuring the xConnect Search Indexer for descriptions and locations of settings.

Default topologies and packages

The following tables list the topologies that include the xConnect Search Indexer and how the role is packaged by default.

Sitecore Installation Framework

The xConnect Search Indexer is available in the following default topologies for SIF:

Topology

Web Deploy Packages

Description

XP Single

Sitecore 9.X.X rev. XXXXXX (OnPrem)_xp0xconnect.scwdp.zip

The xConnect Search Indexer is bundled with all other XP service roles. The indexer is installed as a Windows Service and is physically located in the App_data\jobs\continuous\IndexWorker folder.

XP Scaled

Sitecore 9.X.X rev. XXXXXX (OnPrem)_xp1collectionsearch.scwdp.zip

The xConnect Search Indexer is bundled with the xConnect Collection Search. The indexer is installed as a Windows Service and is physically located in the App_data\jobs\continuous\IndexWorker folder.

Sitecore Azure Toolkit

The xConnect Search Indexer is available in the following default topologies for SAT:

Topology

Web Deploy Packages

Description

XP Single

Sitecore 9.X.X rev. XXXXXX (Cloud)_xp0xconnect.scwdp.zip

The xConnect Search Indexer is bundled with all other XP service roles. The indexer runs as an Azure WebJob in the context of the combined application.

xDB Single

XP Scaled

  • Sitecore 9.X.X rev. XXXXXX (Cloud)_xp1collectionsearchsolr.scwdp.zip (Solr provider)

  • Sitecore 9.X.X rev. XXXXXX (Cloud)_xp1collectionsearch.scwdp.zip (Azure Search provider)

The xConnect Search Indexer is bundled with xConnect Collection Search. The indexer runs as an Azure WebJob in the context of the xConnect Collection Search application.

xDB Scaled

Excluding the xConnect Search Indexer

The xConnect Search Indexer is a mandatory role if you are using the Experience Platform and cannot be excluded from any of the topologies that it is a part of.