Optimizing custom catalog import operations

Abstract

How to use the header keys that skip pipeline blocks to gain efficiency during catalog import routine.

The Postman samples provided out of the box for importing Commerce catalog data uses keys in HTTP header that help gain efficiency during the import process by ignoring certain pipeline blocks within the Commerce Engine. These pipeline blocks perform tasks such as incremental indexing operations, for example, or other functions that can affect the performance of Catalog import operations.

This topic outlines how you can benefit from efficiencies that these header keys provide when you choose to create your own custom Catalog import routine for importing catalog data programmatically.

Blocks to skip for optimizing a Commerce catalog import routine

Following is the list of blocks that your Catalog import routine should skip to speed up Commerce Catalog import operations:

  • IndexDeletedSitecoreItemBlock - this block adds entities to a list to be deleted from an index incrementally.

  • IndexUpdatedSitecoreItemBlock - this block adds entities to a list so that they can be updated incrementally to Sitecore search indexes.

  • AddEntityToIndexListBlock -  this block adds entities to a list so that they can be updated incrementally to Commerce Engine search indexes.

  • LocalizeEntityBlock - this block creates localized entities (catalog data is expected to contain localized data).

These blocks derive from a conditional block (Sitecore.Commerce.Core.PolicyTriggerConditionalPipelineBlock) that does not run when certain flags (or keys) are included within the Sitecore.Commerce.Core.CommerceContext headers.

How to pass policy keys programmatically

You can add flags to the CommerceContext header PolicyKeys property to disable the indexing routines and other functions affecting the overall time it takes to complete a Commerce catalog data import.  

The following policy keys are available:

  • IgnoreAddEntityToIndexList

  • IgnoreIndexDeletedSitecoreItem

  • IgnoreIndexUpdatedSitecoreItem

  • IgnoreLocalizeEntity

The PolicyKeys property on the CommerceContext retrieves the current policies, but to manipulate the keys you must modify the header directly, as shown in the examples that follow.

To set new policy keys:

  1. Retrieve the current keys directly from the header string:

    policyKeysHeader =
    context.CommerceContext.Headers[CoreConstants.Headers.PolicyKeys];
  2. Append your new policy key:

    policyKeysHeader
    += "|IgnoreAddEntityToIndexList";
  3. Set the updated policy keys on the header:

    context.CommerceContext.Headers[CoreConstants.Headers.PolicyKeys]
    = policyKeysHeader;
    

    For example:

    // for catalog
    import, disable recommended blocks
    if
    (context.CommerceContext.Headers.ContainsKey(CoreConstants.Headers.PolicyKeys))
    {
    var keys =
    context.CommerceContext.Headers[CoreConstants.Headers.PolicyKeys];
    
    keys +=
    "|IgnoreIndexDeletedSitecoreItem|IgnoreIndexUpdatedSitecoreItem|IgnoreAddEntityToIndexList|IgnoreLocalizeEntity";
    
    context.CommerceContext.Headers[CoreConstants.Headers.PolicyKeys] =
    keys;
    }
    else
    {
    context.CommerceContext.Headers.Add(new KeyValuePair<string,
    StringValues> (CoreConstants.Headers.PolicyKeys,
    "IgnoreIndexDeletedSitecoreItem|IgnoreIndexUpdatedSitecoreItem|IgnoreAddEntityToIndexList|IgnoreLocalizeEntity"));
    
    }