Email marketing

Abstract

Overview of the Sitecore email campaign management and dispatch processes including reputation handling.

Email campaigns are an integrated part of Experience Platform (XP) that relies heavily on the tracking, reporting, and segmentation functionality of Experience Database (xDB).

You use the Email Experience Manager (EXM) application to manage email campaigns that are personal and relevant to your customers, and you use the List Manager application to manage the contacts that you want to be part of your campaign. You can also use marketing automation to target contacts at specific points and different stages of the customer journey.

Both EXM and List Manager are hosted on the Content Management (CM) role.

Campaigns are stored in the Master database, and there are two types available:

  • Regular campaigns that are sent to list of contacts using the dispatch process.

  • Automated campaigns that are sent to individual contacts when they trigger goals or events on your website.

Diagram showing the components involved in managing email campaigns.

You associate regular email campaigns with contact lists or segments that you manage through List Manager. Contact lists are groups of contacts, and segments are contact lists with additional filtering. List Manager reads and writes contact lists and segments in the xDB Collection database using the xDB Collection Search role and the xDB index.

You can associate a regular campaign with multiple contact lists or segments. You can also use lists to exclude contacts in the dispatch process.

Note

Automated campaign emails are handled one at a time and not as part of the dispatch process used for regular campaigns. EXM monitors the Message Bus for Marketing Automation messages stating that a contact has triggered a goal or event. When it receives such a message, EXM renders and sends a campaign email to the contact.

You can use EXM dispatch pipelines and processors to control when campaigns are dispatched, or you can trigger dispatches directly in EXM. In both cases, the dispatch process consists of the following steps:

  • Queuing contacts

  • Rendering emails

  • Sending emails

In the default configuration, the CM role hosts the dispatch process. However, this can be resource intensive and slow down other tasks on the CM role, such as publishing or content editing. If you configure one or more EXM Dedicated Dispatch (DD) roles you can scale out the dispatch process of queuing the contacts, rendering the emails, and sending them.

You configure a DD role in the same way as a CM role, but dedicated to dispatching emails. This does not mean that the CM role itself stops dispatching emails. The dispatch process is shared between the CM role and the additional DD roles, and the CM role monitors and controls the combined process and updates the campaign state in the EXM database.

Diagram showing the dispatch process in general.

The following happens when you dispatch a campaign:

  • EXM changes the campaign state in the EXM database to Queuing.

  • EXM fetches the relevant contact identifiers from the List Manager, which uses the xConnect Collection Search role to query the xDB index and the xDB Collection database.

  • EXM stores the contact identifiers in the EXM database DispatchQueue table.

Contacts are excluded from the queue if they are:

  • On an exclude list associated with the campaign.

  • Part of the global opt-out list.

  • Marked Consent Revoked and Do Not Market.

  • Exceed the bounce count limit established in EXM.

  • On the suppression list.

Diagram showing the process of of queuing contacts.

The dispatch process monitors the DispatchQueue table and reads the contact identifiers in batches. It then fetches campaign data from the Master database and contact details from the xDB Collection database, before using the the Sitecore rendering engine to render the emails.

Simple, personalized emails that use basic token replacement are rendered once and cached for all contacts. Highly personalized emails are rendered separately for each contact.

Campaigns containing highly personalized emails for many contacts therefore puts a significant load on the CM and DD roles which makes scaling considerations important.

For each email rendered the dispatch process stores a message on the Message Bus requesting an update of the contact email address history facet. The CM role picks up this message and updates the contact in the xDB Collection database using the xConnect Collection Search role.

Diagram showing the process of rendering emails.

First, the CM role changes the campaign state in the EXM database to Sending.

Then, when the dispatch process has sent an email to a contact using either the Email Cloud service or a custom SMTP service, it stores an Email Sent message on the Message Bus and removes the contact from the DispatchQueue table.

The xConnect Collection Search role picks up the Email Sent message and registers the campaign on the contact in the xDB Collection database. This enables reporting in EXM and Experience Analytics, and enhances personalization across other channels.

When the dispatch process is finished and all campaign contacts in the EXM database queue have been processed, the CM role changes the campaign state in the EXM database to Sent.

Diagram showing the process of sending emails.

The Email Cloud service maintains a list of email addresses that, for various reasons, it does not send to. This might be, for example, because of complaints, bounces, or invalid email addresses. The list is called the suppression list and you can use it as part of your reputation management and compliance.

The CM role continuously synchronizes the suppression list to the EXM database, and you can use EXM to manually add contacts to suppress. The CM role later synchronizes these contacts back to the Email Cloud service.

If you enable the Right to be forgotten for one or more contacts, the xConnect Collection Search role stores a message on the Message Bus role containing the email addresses that must be forgotten. The CM role picks up this message, removes the email addresses from the suppression list, and synchronizes the list to the Email Cloud service.

Diagram showing the process of maintaining the suppression list.

The CM role handles email bounces and spam complaints to ensure that the email recipients no longer receive emails. This reduces the cost for sending email messages and also prevents Sitecore implementations from being blacklisted due to a poor email reputation.

You can configure reputation management in two ways:

  • If you configure EXM to use the Email Cloud service, the CM role continuously retrieves relevant email bounces and spam complaints from the Email Cloud service.

  • If you configure EXM to use a custom SMTP service for dispatching emails, the CM role does not perform any reputation management, and Sitecore will not respect email bounces and spam complaints. You can, however, configure a custom POP3 service to check for bounced emails and spam complaints.

    Note

    Due to differences in email systems, using a POP3 service to check for bounced emails and spam complains is not a foolproof setup, and some emails might fail to be reported.

In both scenarios, the CM role stores the email bounces and spam complaints in an EXM database queue for later processing.

The CM role continuously process the bounced email and spam complaints queue in EXM database and pass the relevant contact to the xConnect Collection Search role where it is added to the global opt-out list in the xDB Collection database. The xConnect Collection Search role also removes the contact from any marketing lists you have added them to.

Diagram showing the process of reputation management.

When a contact opens a campaign email, the email triggers an identifiable request to the Content Delivery (CD) role. The CD role handles this request by storing an Open message on the Message Bus role and returning an empty 1x1 pixel to the contact.

The CM role picks ups the message and sends an interaction containing an Open event to the xConnect Collection role. Here, an EXM-specific xConnect plugin updates the contact facets in the xDB Collection database for reporting purposes.

All links in campaign emails are associated with a specific EXM tracking page on the CD role. When a contact clicks an email link, the tracking page creates an interaction with a Click page event in the Session State database. The tracking page then redirects the contact to the actual page in the link.

The tracking page uses the normal tracking data flow, so when a session ends, the CD role sends the interaction to the xConnect Collection role for persisting, and an EXM-specific plugin updates the contact facets in the xDB Collection Database for reporting purposes.

Diagram showing the process of tracking emails.

The contact unsubscribes from a campaign by clicking an unsubscribe link in an email. This opens a tracking page on the CD role where an EXM unsubscribe handler does three things:

  • It creates an interaction with an Unsubscribe page event in the Session State database.

  • It stores an Unsubscribe message on the Message Bus.

  • It redirects the contact to a user-friendly unsubscribe page.

When the contact's session ends, the CD role passes the interaction with the Unsubscribe page event to the xConnect Collection Search role that stores it in the xDB Collection database. Sitecore uses this information for analytics and reporting purposes.

Meanwhile, the CM role picks up the Unsubscribe message from the Message Bus and asks List Manager to remove the contact from any the campaign contact list and update the contact itself using the xConnect Collection Search role.

Finally, the CM role orders the Email Cloud service or a custom SMTP service to send an email message confirming that the contact has unsubscribed.

Diagram showing the process of unsubscribing from campaigns.

EXM does not introduce any specific roles for analytics reporting or insights. Campaign analytics are based on xDB, specifically the processing and aggregation functionality in the xDB Processing role, and data served through the xDB Reporting role.

See Processing and Aggregation and Analytics and Reporting for more information on these data flows and roles.

Refer to the Architecture and Roles documentation for privacy and security considerations for each role on the processing and aggregation data flow.