Data exchange and synchronization


Overview of using the Sitecore Data Exchange Framework to synchronize content between endpoints.

Sitecore Data Exchange Framework (DEF) enables you to synchronize data to and from Sitecore and any other system or database. You can define and customize the data exchange process and the extracting, manipulating, and mapping of Sitecore data and external data.

You set up and configure synchronization processes for the DEF through the Sitecore Management interface. Synchronization processes are typically defined by developers or administrators. The synchronization process is essentially a pipeline, or a number of pipeline steps run sequentially, that start by reading data from one endpoint and finish by saving data to another endpoint.

Different providers can serve each endpoint. For example, an endpoint could be the Master database, the xConnect Collection role, or a third-party service such as a CRM or PIM system.

When you trigger a synchronization process, an asynchronously Sitecore task is created and the synchronization process runs in that task. You can check the status of the task, but the task itself is running in the background and does not prevent you from doing other tasks. You can also trigger the synchronization process on a schedule or through other business logic defined by the Sitecore implementation. For example, a synchronization process might be to read marketing campaigns from a CRM system, map them to Sitecore campaigns, and store them in the Master database for use in personalization or marketing automation.

Sitecore Data Exchange Framework synchronization between endpoints can be triggered by you or as a scheduled task.

Because the definition of synchronization processes is very flexible, synchronization can happen in all directions, and the system can use all endpoints for both reading and storing data. Limits are set by the systems involved as well as the DEF providers for those systems.

For scaling reasons, you can scale out the synchronization processes to a separate DEF Remote Client role. The DEF SDK provides an API for developers to create this role freely, for example, as worker roles or web services.

Data synchronization can be performed independently by the Sitecore Data Exchange Framework Remote Client.

The custom business logic in the role reads the pipeline definition from the Content Management role through the DEF SDK. At this point, the DEF Remote Client essentially disconnects from the Content Management role and everything else is handled locally. The DEF Remote Client role is also responsible for determining when to trigger this synchronization process, for example, through a schedule or through events in the system.

When the DEF Remote Client triggers the synchronization process, the logic is run on the client and therefore the client will communicate with the defined endpoints.

Refer to the Architecture and Roles documentation for privacy and security considerations for each role on the processing and aggregation data flow.