Walkthrough: Creating a custom index

Version:

You can create your own custom indexes for your search provider if the default indexes do not suit your purposes. This walkthrough describes how to create a new custom index, create a policy for incremental updates to the index, and configure the update strategy to update the index itself.

For the purposes of this walkthrough, the new custom index uses Solr as a search provider and contains Commerce catalog items from the Sitecore Master database.

Create a new Solr core

To create a new Solr core:

Create and initialize a new Solr core and name it, for example, mysite_master_products.

Create new Commerce Engine lists to track incremental changes

All indexing configuration is stored in the Plugin.CatalogIndexing.PolicySet-1.0.0.json file. You must add a new policy that instructs the Commerce Engine to create new lists for tracking incremental updates to your custom index.

For more information on incremental index updates in Sitecore XC, see the Incremental index updates topic.

Note

You must add this configuration in all Commerce Engine environments.

To create new lists in Commerce Engine to track incremental changes:

Navigate to the c:\inetpub\wwwroot\<environment>\wwwroot\data\Environments folder and open the Plugin.CatalogIndexing.PolicySet-1.0.0.json file.

Under the "$values" list, add a policy block to create managed lists for incremental updates to your custom index:

  {
    "$type": "Sitecore.Commerce.Plugin.Catalog.SitecoreCatalogIndexingPolicy, Sitecore.Commerce.Plugin.Catalog",
    "Name": "MySiteSellableItemsIndexMaster",
    "IncrementalListName": "MySiteSellableItemsIncrementalMasterProducts",
    "DeletedListName": "MySiteSellableItemsDeletedMasterProducts",
    "EntityTypeNames": {
      "$type": "System.Collections.Generic.List`1[[System.String, mscorlib]], mscorlib",
      "$values": [
         "Sitecore.Commerce.Plugin.Catalog.SellableItem, Sitecore.Commerce.Plugin.Catalog"
      ]
    },
    "IsMasterIndex": true
  },
  {
    "$type": "Sitecore.Commerce.Plugin.Catalog.SitecoreCatalogIndexingPolicy, Sitecore.Commerce.Plugin.Catalog",
    "Name": "MySiteCategoriesMasterProducts",
    "IncrementalListName": "MySiteCategoriesIncrementalMasterProducts",
    "DeletedListName": "MySiteCategoriesDeletedMasterProducts",
    "EntityTypeNames": {
      "$type": "System.Collections.Generic.List`1[[System.String, mscorlib]], mscorlib",
      "$values": [
        "Sitecore.Commerce.Plugin.Catalog.Category, Sitecore.Commerce.Plugin.Catalog"
      ]
    },
    "IsMasterIndex": true
  },
  {
    "$type": "Sitecore.Commerce.Plugin.Catalog.SitecoreCatalogIndexingPolicy, Sitecore.Commerce.Plugin.Catalog",
    "Name": "MySiteCatalogsMasterProducts",
    "IncrementalListName": "MySiteCatalogsIncrementalMasterProducts",
    "DeletedListName": "MySiteCatalogsDeletedMasterProducts",
    "EntityTypeNames": {
      "$type": "System.Collections.Generic.List`1[[System.String, mscorlib]], mscorlib",
      "$values": [
         "Sitecore.Commerce.Plugin.Catalog.Catalog, Sitecore.Commerce.Plugin.Catalog"
      ]
    },
    "IsMasterIndex": true
  }

Bootstrap the Commerce Engine to persist your changes.
Perform an IIS reset in the IIS Manager.

Create an index configuration file

You must create a configuration file for the new custom index. The index configuration file must contain:

General information about the index, such as the index name, the Solr core it is associated with, and the property store.
References to the update strategies used to retrieve changed or deleted items for incremental updates to the index.
A definition for the crawler that populates the index and the root location in the content tree to start from.
Detailed definitions of each update strategy referenced by the index.

Example

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/" xmlns:search="http://www.sitecore.net/xmlconfig/search/" xmlns:role="http://www.sitecore.net/xmlconfig/role/">
  <sitecore search:require="solr">
    <contentSearch>
      <configuration type="Sitecore.ContentSearch.ContentSearchConfiguration, Sitecore.ContentSearch">
        <indexes hint="list:AddIndex">
          <index id=" mysite_master_products" type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, Sitecore.ContentSearch.SolrProvider">
            <param desc="name">$(id)</param>
            <param desc="core"> mysite_master_products</param>
            <param desc="propertyStore" ref="contentSearch/indexConfigurations/databasePropertyStore" param1="$(id)" />
            <configuration ref="contentSearch/indexConfigurations/defaultSolrIndexConfiguration" />
            <strategies hint="list:AddStrategy">
              <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/mysite_products_sellableItemsIntervalAsynchronousStrategyMaster" />
              <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/mysite_products_categoriesIntervalAsynchronousStrategyMaster" />
              <strategy ref="contentSearch/indexConfigurations/indexUpdateStrategies/mysite_products_catalogsIntervalAsynchronousStrategyMaster" />
            </strategies>
            <locations hint="list:AddCrawler">
              <crawler type="Sitecore.Commerce.Engine.Connect.Search.Crawlers.AllCatalogItemsCrawler, Sitecore.Commerce.Engine.Connect">
                <ItemsToTake>100</ItemsToTake>
                <Environments hint="list">
                  <environment>HabitatAuthoring</environment>
                </Environments>
                <Database>master</Database>
              </crawler>
            </locations>
          </index>
        </indexes>
      </configuration>
      <indexConfigurations>
        <indexUpdateStrategies>
          <mysite_products_sellableItemsIntervalAsynchronousStrategyMaster type="Sitecore.Commerce.Engine.Connect.Search.Strategies.SellableItemsIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
            <IncrementalIndexListName>MySiteSellableItemsIncrementalMasterProducts</IncrementalIndexListName>
            <DeletedIndexListName>MySiteSellableItemsDeletedMasterProducts</DeletedIndexListName>
            <ItemsToTake>100</ItemsToTake>
            <Environments hint="list">
              <environment>HabitatAuthoring</environment>
            </Environments>
            <param desc="interval">00:10:00</param>
            <param desc="database">master</param>
          </mysite_products_sellableItemsIntervalAsynchronousStrategyMaster>
          <mysite_products_categoriesIntervalAsynchronousStrategyMaster type="Sitecore.Commerce.Engine.Connect.Search.Strategies.CategoriesIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
            <IncrementalIndexListName>MySiteCategoriesIncrementalMasterProducts</IncrementalIndexListName>
            <DeletedIndexListName>MySiteCategoriesDeletedMasterProducts</DeletedIndexListName>
            <ItemsToTake>100</ItemsToTake>
            <Environments hint="list">
              <environment>HabitatAuthoring</environment>
            </Environments>
            <param desc="interval">00:10:00</param>
            <param desc="database">master</param>
          </mysite_products_categoriesIntervalAsynchronousStrategyMaster>
          <mysite_products_catalogsIntervalAsynchronousStrategyMaster type="Sitecore.Commerce.Engine.Connect.Search.Strategies.CatalogsIntervalAsynchronousStrategy, Sitecore.Commerce.Engine.Connect">
            <IncrementalIndexListName>MySiteCatalogsIncrementalMasterProducts</IncrementalIndexListName>
            <DeletedIndexListName>MySiteCatalogsDeletedMasterProducts</DeletedIndexListName>
            <ItemsToTake>100</ItemsToTake>
            <Environments hint="list">
              <environment>HabitatAuthoring</environment>
            </Environments>
            <param desc="interval">00:10:00</param>
            <param desc="database">master</param>
          </mysite_products_catalogsIntervalAsynchronousStrategyMaster>
        </indexUpdateStrategies>
      </indexConfigurations>
    </contentSearch>
  </sitecore>
</configuration>

To create a custom index configuration file:

Open a new text file in the App_Config\Include folder and save it with a relevant file name and .config extension (for example, my_site_master-products_index.config).
Open the file and add your index configuration (using the previous example as a guide).

In addition to configuring index configuration properties, you must define the index crawler, and include configuration for the index update strategies.
Save your changes.

Note

In this example, the custom index is configured to contain items from the Master database. For this reason, you would store the custom index configuration file in the App_Config\Include folder in your CM deployment.

Add a custom index to the list of indexes in the GetChildrenToIndex processor

You must add the new index to the Sitecore GetChildrenToIndex processor. Sitecore XC uses its own crawler to index Commerce items. The Sitecore.Commerce.Engine.Connect.Pipelines.GetChildrenToIndex processor stops the default Sitecore index crawlers from attempting to index Commerce items.

To add the custom index to the list of Sitecore children indexes:

In the C:\inetpub\wwwroot\cqa-dem-01-dk1\App_Config\Include\Y.Commerce.Engine folder, open the Sitecore.Commerce.Engine.Connectors.Index.Common.config file.

Add the custom index to the list of indexes, as shown in the following example:

<pipelines>
  <indexing.getChildrenToIndex>
     <!-- 
          Insert our processor first so that we can intercept Commmerce items that we do    not want indexed into the below referenced indexes.
          As Commerce uses a separate crawler to index Commerce items, we insert this processor to stop the default crawlers from attempting to index
          Commerce items.
          By default the processor below will skip indexing of children of the following locations or templates:
            Commerce Catalog Folder   ID {334E2B54-F913-411D-B159-A7B16D65242C}
     -->
     <processor type="Sitecore.Commerce.Engine.Connect.Pipelines.GetChildrenToIndex, Sitecore.Commerce.Engine.Connect" patch:source="Sitecore.Commerce.Engine.Connectors.Index.Common.config">
        <!-- The list of indexes that we do not want commerce items to be indexed to -->
        <Indexes hint="list:AddString" help="Add index names below to stop crawling of Commerce items by the default crawler">
          <IndexName>sitecore_master_index</IndexName>
          <IndexName>sitecore_web_index</IndexName>
          <IndexName>sitecore_testing_index</IndexName>
          <IndexName>sitecore_suggested_test_index</IndexName>
         <IndexName>mysite_master_products/IndexName>
        </Indexes

If you have suggestions for improving this article, let us know!