Collecting experience data

Abstract

Overview of how Sitecore works with marketing metadata, contacts and interactions, and non-Sitecore websites.

Experience Platform (XP) consists of multiple storage and application roles as well as search indexes. Two of the primary XP components are the xConnect Collection service and the xDB Collection database.

The xConnect Collection service is a front end for the xDB Collection database. It offers the xConnect Client API to any trusted client that wants to get, create, or search experience data. One example of a trusted client is Sitecore Experience Manager (XM). During visitor interactions with websites or mobile channels, XM collects and retrieves experience data to provide personalized experiences.

The xDB Collection database is a repository for marketing data. It collects contacts and interactions and connects them into a comprehensive, unified view of the individual customer.

The Reference Data service is the front end for the xDB Reference Data database that holds common data referenced by XP roles. Examples of reference data are marketing taxonomies such as campaigns, goals, and outcomes as well as IP Geolocation data and device information.

The marketer creates and updates marketing metadata to be associated with websites, pages, or actions on the web pages through the Marketing Control Panel, Campaign Creator, and other Content Management (CM) applications. The marketing metadata is stored in the Master database and includes goals, campaign definitions, marketing automation plan definitions, taxonomies, and more.

The CM role deploys the metadata to the Reference Data service, which in turn stores the data in the xDB Reference Data database. This ensures the data integrity of xDB and that the deployed data can be used by, for example, the aggregation process, the marketing automation service, or in reporting.

The Reference Data service also allows trusted clients outside of XP to submit and retrieve reference data, thereby extending the xDB Reference Data database with outside concepts and data.

Diagram showing the components involved in creating and deploying marketing metadata.

The Content Delivery (CD) role hosts the Web Tracker subsystem. Web Tracker continuously tracks the actions of visitors to the website. It is independent from the xConnect Collection service and the xDB Collection database given that it can track and personalize the experience within a single visit - even if the data is not persisted across visits or sessions.

When a new visitor visits a website or channel, the CD role creates a private session containing a contact and an interaction. The contact contains information about the visitor and is mapped to a unique ID and stored by Web Tracker in the device cookie in the visitor browser. The interaction contains information about the visit such as which pages the visitor has visited and which goals have been triggered.

The contact and the interaction can also contain several facets that represents data a given system wishes to store. A facet could be a customer address, an email address, a contact preference, or the geographical location where the interaction took place or what type of device made the interaction.

The contact and the interaction are submitted to xConnect Collection and stored in the xDB Collection database when the session ends.

The xConnection Collection service can be scaled to accommodate increased levels of read and write operations in scenarios with high visitor load. Likewise, the xDB Collection database can potentially contain very large amounts of visitor information. To handle this, the database can be scaled using a mechanism called sharding that effectively splits the database up into separate databases of reasonable size that can be distributed to better handle the load.

Because of the openness of the xConnect Client API used by Web Tracker, the xConnect Collection service can be accessed by any trusted system through the xConnect Client API. For example, Data Exchange Framework uses this API to synchronize contacts and interactions with 3rd party sources such as CRM systems.

Diagram showing the process of collecting contacts and interactions.

With the Web Tracker subsystem on the CD role, Federated Experience Manager (FXM) can extend the Sitecore experience to non-Sitecore websites that have agreed to host a piece of JavaScript code called a beacon. FXM can interact with these websites in the following ways:

  • Associate metadata like taxonomy, goals and campaigns

  • Track contacts and interactions

  • Analyze tracked activities

  • Push personalized content back

To access the metadata quickly and have the beacon operate as fast as possible, FXM uses the FXM Web index to map the beacon parameters to the metadata in the Web database.

The visitor contacts and interactions on the non-Sitecore websites are stored in the Private Session State Store. When the session ends, the data is submitted to the xConnect Collection services and stored in the xDB Collection database.

Diagram showing the process of extending the Sitecore experience to non-Sitecore websites.

Refer to the Architecture and Roles documentation for privacy and security considerations for each role on the processing and aggregation data flow.