Sitecore Experience Platform

Sitecore Cortex Processing Engine

The Sitecore Cortex™ Processing Engine (Processing Engine) is a worker role that performs data processing tasks. Reasons to use the Processing Engine include:

  • The Processing Engine is built for scale.

  • The Processing Engine allows you to build complex processing jobs by chaining smaller tasks.

  • The Processing Engine allows you to schedule tasks from a Content Management instance using the Task Manager API, which integrates with the Message Bus.

The Processing Engine is made up of the following key components:

  • Message handlers receive tasks registered via the Task Manager API.

  • Agents delegate tasks to workers.

  • Workers perform data processing tasks, such as data projection or model training.

Working with the Processing Engine

Use the Task Manager API to register tasks to be handled by the Processing Engine.

  • Register a task to project data from xConnect into storage

Extending the engine

As a developer, you can extend the Processing Engine in the following ways:

  • Create a custom worker.

  • Create a custom data source that can be used by the projection worker. 

  • Create a custom data projection.

  • Implement custom model training and model evaluation logic.

  • Create a custom agents

Default functionality

In the initial release of 9.1, the Processing Engine role is responsible for the following processing tasks:

  • Build personalization rules and publish to the reporting database. Content Testing does not use the Machine Learning Server integration.

The Processing Engine ships with several default workers that can be chained together to perform data processing tasks.

The role of legacy xDB Processing

The Legacy xDB Processing role is still responsible for the following processing tasks:

  • Aggregation of Experience Analytics data into the xDB Reporting database.

The Processing Engine and machine learning

The Processing Engine provides an integration point for your machine learning platform of choice. Default integrations include:

  • Microsoft Machine Learning Server

Inject IScriptSessionFactory into a model wrapper to use the Microsoft Machine Learning Server integration.