Importing catalog data programmatically

Abstract

How you can leverage samples from the Commerce Engine SDK to create custom catalog import operations.

Most Sitecore Experience Commerce implementations have some requirement for importing catalog data programmatically. There are different approaches you can take when writing a customization to import your catalog data, depending on the volume of data, the frequency of import operations, whether you are writing an integration with an external product information management system, and where the import operation is triggered from.

This topic focuses on how you can leverage some of the samples provided in the Sitecore.Commerce.Engine.SDK to:

  • Create a custom catalog import operation that is triggered by a web client, in a HTTP request.

  • Add a catalog import operation to your custom Commerce Engine plugin.

Creating a custom catalog import triggered by a web client

The Sitecore.Commerce.Engine.SDK contains a Sitecore.Commerce.Sample.Console application that defines a sample static helper method named RunImport. You can use this RunImport method to trigger a catalog import via an HTTP request. Out-of-the-box, this method is called when you run the Initialize Environment sample request in Postman.

The path to the console application extensions file that contains the RunImport method is:

Sitecore.Commerce.Engine.SDK.*.*.**\src\Sitecore.Commerce.Sample.Console\Extensions\EngineExtensions.cs

The sample RunImport method imports the catalog and also publishes the catalog entities by default. But If you want to import a catalog without publishing its content, you can use the following code example where the option to publish is parameterized as a Boolean:

private static
   async Task RunImport(  
   string filePath,        
   string mode,         
   string importMethodName,  
   bool publish,         
   params string[] allowedErrorTerms)
{          
   using (new SampleMethodScope())        
   {            
     File.Exists(filePath).Should().BeTrue();          
     using (var importStream = new FileStream(filePath, FileMode.Open, FileAccess.Read))               
     {                 
       using (var hc = CreateHttpClientAuthoring())             
       {                 
        using (var importContent = new MultipartFormDataContent())                 
        {                     
          importContent.Add(new StreamContent(importStream), "importFile", filePath);                    
          importContent.Add(new StringContent(mode), "mode");                   
          importContent.Add(new StringContent("0"), "errorThreshold"); 
          importContent.Add(new StringContent(publish ? "true" : "false"), "publish");

          var importResponse = await hc.PostAsync(    
            $"{Program.ShopsServiceUri}{importMethodName}()", importContent);
          importResponse.IsSuccessStatusCode.Should().BeTrue();                 
          var importResponseText = await importResponse.Content.ReadAsStringAsync();                 
          var importCommand = JsonConvert.DeserializeObject<CommerceOps.Sitecore.Commerce.Core.Commands.CommerceCommand>(importResponseText);                         
          importCommand.WaitUntilCompletion(allowedErrorTerms);                     
        }                 
       }               
     }          
   }
}

Note

When you import Catalog entities in the unpublished state, a Business Tools user must push each entity through the approval workflow to publish to the storefront. You cannot use the Content Editor to publish Commerce items.

Creating a custom catalog import triggered in the Commerce Engine

Some commerce implementation require the Commerce Engine to trigger import operation. You can create your own plugin to customize the Commerce Engine for handling catalog data import operations, for example to perform a scheduled catalog import routine.

The Sitecore.Commerce.Engine.SDK contains two sample catalog management plugins that provide a good starting point for a custom catalog import plugin: the Plugin.Sample.AdventureWorks plugin and the Plugin.Sample.Habitat plugin.

In these sample plugins, the InitializeCatalogBlock class defines the catalog import process. You can use this sample class in your custom catalog management plugin to handle the import function. The full path to this pipeline block in sample Habitat catalog plugin is:

Sitecore.Commerce.Engine.SDK.*.*.**/src/Plugin.Sample.Habitat/Pipelines/Blocks/InitializeCatalogBlock.cs

The sample InitializeCatalogBlock block, shown in the following example, sets the AutoApprovePolicy policy and automatically publishes the imported entities to the storefront:

protected virtual async Task ImportCatalogAsync(CommercePipelineExecutionContext context)
{
    using (var stream = new FileStream(GetPath("Habitat.zip"), FileMode.Open, FileAccess.Read))
    {
        var file = new FormFile(stream, 0, stream.Length, stream.Name, stream.Name);
        var contextOptions = context.CommerceContext.PipelineContextOptions;
        var argument = new ImportCatalogsArgument(file, CatalogConstants.Replace)
        {
            BatchSize = -1,
            ErrorThreshold = 10
        };
            context.CommerceContext.Environment.SetPolicy(new AutoApprovePolicy());
            await _importCatalogsPipeline.RunAsync(argument, contextOptions).ConfigureAwait(false);
            context.CommerceContext.Environment.RemovePolicy(typeof(AutoApprovePolicy));
     }
  }
 private string GetPath(string fileName)
 {
    return Path.Combine(_hostingEnvironment.WebRootPath, "data", "Catalogs", fileName);
 }

You can optimize your custom catalog import operation by passing policy keys programmatically in the CommerceCommerce.Headers.

Importing catalog items without publishing

In some scenarios, you might want to import catalog data without publishing items to the storefront automatically. You can to do this by modifying the InitializeCatalogBlock in the sample plugins.

Note

If you import Catalog entities in the unpublished state (without the AutoApprovePolicy policy), then a Business Tools user must push each imported entity through the publishing workflow to publish them. You cannot use the Sitecore Content Editor to publish Commerce items.

The Plugin.Sample.AdventureWorks and the Plugin.Sample.Habitat plugins are configured to call the InitializeEnvironment pipeline before running the InitalizeCatalogBlock. As part of its default behavior, the InitializeEnvironment pipeline adds and sets the AutoApprovePolicy to publish imported catalog items. If your custom plugin uses the same logic (defined in the sample plugins Configure.Sitecore.cs file), and you want to import a catalog without publishing its items on the storefront, then you must add an extra line of code to the InitializeCatalogBlock block to explicitly remove the AutoApprovePolicy. This is necessary because the InitializeEnvironment, which runs first, has already added and set the AutoApprovePolicy.

To import the catalog without publishing it on the storefront, modify the InitializeCatalogBlock by adding a statement to remove the auto-approve policy, and comment out the two statements that enable and disable it, as shown in the following example:

context.CommerceContext.Environment.RemovePolicy(typeof(AutoApprovePolicy));
//context.CommerceContext.Environment.SetPolicy(new AutoApprovePolicy());
await _importCatalogsPipeline.RunAsync(argument, contextOptions).ConfigureAwait(false);
//context.CommerceContext.Environment.RemovePolicy(typeof(AutoApprovePolicy));