Configure publishing performance

Abstract

This topic describes how you control performance when you publish to Experience Edge.

When you publish to Experience Edge, resource consumption can peak at many of the stages the process goes through. The connector module uses APIs from the Task Parallel Library (TPL) such as Data parallelism and Dataflow to maximize performance. This also provides a simple way to control and scale the performance, based on the available processors.

This topic describes two aspects of performance you can control in the configuration.

Publishing manifest creation involves iteration of all the items identified as being published and searching for dynamic dependencies for each of them, using the Solr index created for this. The manifest is processed in parallel, and the index queries are batched against a group of items to reduce the index round trips.

To control the publishing manifest creation performance, configure the following:

  • The ExperienceEdge.MaxManifestExpandThreadCount setting controls the maximum number of parallel threads used to expand the manifest.

  • The ExperienceEdge.MaxManifestSearchQueryTermsCount setting controls the maximum number of items used in a single search query.

To export the manifest, the connector performs a number of operations on all manifest items. Network latency significantly impacts performance of these operations. Some repeated, local operations (loading item data from the database, generating JSS layout for a page, and generating the GQL schema for the database) also impact performance.

The connector uses a TPL Dataflow for concurrency in all export operations where it is possible.

To control the publishing manifest export performance, configure the following:

  • The ExperienceEdge.ExportEntitiesBufferSize setting controls the size of the export pipeline buffer.

  • The ExperienceEdge.MaxExportEntitiesThreadCount setting controls the maximum number of parallel threads used to export the items in the manifest.

While entities are uploaded individually, entity publish events are grouped and sent in batches. To control the batch size for publishing events:

  • Specify the value of the ExperienceEdge.ExportPublishEventsBatchSize setting.

    Note

    If you specify a value higher than the value of ExperienceEdge.ExportEntitiesBufferSize, the value of the latter is used.