Walkthrough: Importing catalog data

This walkthrough describes how to import a ZIP file containing catalog data (previously exported from a Sitecore XC environment), into another Sitecore XC environment.

Important

In Sitecore XC release 9.0.3 and 9.1, you can only import catalog data into an environment that contains the same Storefront site as the one from which you previously exported it. If you attempt to import the catalog data into an environment that contains a different Storefront site, the import operation fails with an error indicating that the catalog is already associated to another site.

For details on the expected data structure for the import operation, see the Walkthrough: Exporting catalog and inventory data topic.

Note

Sitecore XC currently supports full catalog imports only.

During standard operations, the Commerce Engine Search service uses the incremental index minion as an automated process that periodically adds new catalog items to an indexing list. This automated process ensures the discoverability of new catalog items. To help optimize performance during catalog import, the Catalog application programming interface (API) uses policy keys that prevent incremental indexing during the catalog data import process. This reduces the total time required to import catalog data.  

After the catalog data is imported, you can proceed with importing any related inventory data set (if required).  Finally, you must rebuild catalog indexes manually at the Commerce Engine level and at the Sitecore platform level.

To import catalog data and inventory sets (if required), and rebuild indexes, perform the following procedures:

Before you begin importing catalog data, you must modify the Internet Information Services (IIS) settings (in the IIS Manager and in the web.config file) to reduce the risk of timeouts occurring during the data export process. This is important when exporting a very large catalog data set.

To reconfigure the service used to import catalog data:

  1. Start the Internet Information Services (IIS) Manager and, in the navigation tree, click Application Pools.

  2. On the Application Pools page, click the name of the relevant application pool (for example, CommerceAuthoring_Sc9). 

  3. In the Actions pane, in the Edit Application Pool section, click Advanced Settings and, in the Advance Settings dialog box, set the following values to 0:

    Section

    Parameter

    Value

    CPU

      Limit Interval (minutes) 

    0

    Process Model

     Idle Time-out (minutes)

    0

    Recycling

    Regular Time Interval (minutes)

    0

  4. click OK.

  5. Open the web.config file for the service you use to import the catalog data (for example, C:\inetpub\wwwroot\CommerceAuthoring_Sc9\web.config), and modify the settings marked in bold in the following configuration sample: 

    Note

    Before you change the IIS settings, take note of their original values so that you know what to set them back to after you have finished importing catalogs.

    <?xml version="1.0" encoding="utf-8"?>
    <configuration>
      <system.webServer>
        <security>
          <requestFiltering>
            <!-- For Microsoft IIS (Internet Information Services), maxAllowedContentLength specifies the maximum length of content in a request, in bytes. -->
            <requestLimits maxAllowedContentLength="2147483648" />
          </requestFiltering>
        </security>
        <handlers>
          <add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified" />
        </handlers>
        <modules runAllManagedModulesForAllRequests="false">
          <remove name="WebDAVModule" />
        </modules>
        <aspNetCore processPath=".\Sitecore.Commerce.Engine.exe" arguments="" forwardWindowsAuthToken="false" stdoutLogEnabled="false" requestTimeout="00:00:00"stdoutLogFile=".\logs\stdout" />
      </system.webServer>
    </configuration>
    
  6. In the memory cache plugin file for the service you use to import the catalog data (for example, C:\inetpub\wwwroot\CommerceAuthoring_Sc9\wwwroot\data\EnvironmentsPlugin.AuthoringMemoryCache.PolicySet-1.0.0.json) ensure that the Sitecore.Commerce.Plugin.Workflow.Workflow entity section has the parameter AllowCaching set to true

    If not, change the value to true and then bootstrap the Commerce Engine to persist the change.

    Note

    Optionally, you can enable logging for collecting information while importing catalog data. To do this, open the config.json file for the service you use to import the catalog data (for example, C:\inetpub\wwwroot\CommerceAuthoring_Sc9\wwwroot\config.json) and set the log level values for "Default""System" and "Microsoft" logs to "Information".

    "Logging":  {
                   "IncludeScopes": false,
                    "LogLevel":  {
                                     "Default":  "Information",
                                     "System":  "Information",
                                     "Microsoft":  "Information"
                                  }
  7. Reset the Internet Information Services.

This task describes how to import a ZIP file containing catalog data (previously exported from a Sitecore XC environment), into another Sitecore XC environment. This procedure is based on the Postman sample provided with the Sitecore Commerce Engine Software Development Kit (SDK).

The following instructions assume that you have installed and set up the Postman application, imported the sample collections from the Sitecore Commerce Engine SDK, configured your environment, and retrieved a bearer token to access the Commerce Engine API.

To import catalog product data:

  1. In the Collections pane, expand the CatalogAPISamples collection.

  2. Open the Catalog - API folder and click one of the following requests:Import Catalogs (with publishing) requests.

    Note

    When you import with publishing, you by-pass the publishing workflow. As a result, sellable items are imported as the "published" version, and do not need to transition from the various unpublished status (for example, draft, waiting for approval, and so on) in order to be published.

    (The Postman samples also include a request to import catalogs without publishing. In this case, imported sellable items are in the draft status and must then go though the publishing workflow.)

  3. In the Environment field, specify the Commerce environment where you need to import data  (for example, the HabitatAuthoring environment).

  4. Click the Headers tab and, in the HTTP request, specify the following header key and value:

    • Key: PolicyKeys

    • Value: IgnoreIndexDeletedSitecoreItem|IgnoreAddEntityToIndexList|IgnoreIndexUpdatedSitecoreItem|IgnoreLocalizeEntity

  5. On the Body tab, in the importFile key value, click Select Files and browse to the ZIP file that contains the catalog data to import.  Optionally, set other key values as required. The following table lists available key values:

    Key

    Value

    Description

    mode

    replace

    Deletes all existing catalogs, and adds the new catalog file to import.

    Caution

    The replace mode is the default value. This mode deletes all existing catalog data (even if the new catalogs have different names.) If you want to keep existing catalog data, use the add mode.

    add

    Adds the new catalog data file, and keeps all existing catalogs.

    Note

    If an existing catalog has the same name as the catalog file to import, the content of the existing catalog is replaced with the content of the new catalog file.

    errorThreshold

    <int>

    Indicates the maximum number of errors allowed during the import operation. After this threshold is reached, the import operation fails.

  6. To begin the catalog import operation, click Send

    You can use the DevOps API to check the status of long running import commands.

    Note

    If you have configured logging to collect information during the import operation, set the logging levels back to their original value (for example, from "Information" back to "Warning").

  7. Repeat this procedure for each additional catalog .zip file that you want to import into your Sitecore Commerce deployment.

    Caution

    After you have finished importing all your catalogs (and there is no related inventory data to import), you must set the settings you previously changed in the IIS Manager and in the web.config file back to their original values. 

After you have finished importing catalog data files into your deployment, you can import related inventory data.

The following instructions assume that you have installed and set up the Postman application, imported the sample collections from the Sitecore Commerce Engine software development kit (SDK), configured your environment, and retrieved a bearer token to access the Commerce Engine API.

To import catalog inventory data:

  1. If the Sitecore Identity token has expired, execute another GetToken request to get a new token.

  2. In the Collections pane, expand the InventoryAPISamples collection.

  3. Open the Inventory - API folder, and click the Import Inventory Sets request.

  4. In the Environment field, specify the Commerce environment where you need to import data (for example, the HabitatAuthoring environment).

  5. From the Body tab, in the importFile key value, click Choose Files and browse to click  the .zip file that contains the inventory data to import.

  6. To begin the import operation, click Send.

  7. Repeat this procedure for each additional product inventory ZIP file that you want to import into your Sitecore Commerce deployment.

    Caution

    After you have finished importing all your inventory sets, you must set the settings you previously changed in the IIS Manager and in the web.config file back to their original values. 

    Note

    If you have configured logging to collect information during the import operation, set the logging levels back to their original value (for example, from "Information"  back to "Warning").

After you have completed the catalog and inventory data import, you must perform a full indexing of catalog items by manually initiating the Commerce Engine full index minion. This procedure uses the Postman samples provided in the Sitecore Commerce Engine SDK.

To invoke the full index minion: 

  1. In the Postman Collections pane, expand the SitecoreCommerce_DevOps collection.

  2. Open the Minions folder and click the Run FullIndex Minion - Catalogs items request. 

  3. In the Body pane, set the environment to the Authoring Environment (for example, "environmentName":"{{AuthoringEnvironment}}").

  4. To execute the request, click Send.

    Note

    After you have performed a full indexing of catalog items, you must manually rebuild Sitecore XP indexes.