Walkthrough: Preparing to install Data Exchange Framework in a container environment

Abstract

How to prepare installation files and images for installing Data Exchange Framework in a container environment.

To prepare for adding the Sitecore Data Exchange Framework (DEF) module to your installation, regardless of whether you are deploying to Docker or Azure Kubernetes service, follow the steps in this walkthrough. The walkthrough describes how to:

Prepare the installation files
Build the Docker images

Prepare the installation files

To prepare the files required for the installation:

Download the DEF container deployment package from the Sitecore Downloads page. Extract it to your local workstation with the folder structure intact.
In the folder where you extracted the DEF container deployment package, open the folder for the Windows version and topology you are using, for example, def\compose\ltsc2019\xp1.

Open the .env-example file in an editor. The following shows an example of what the contents of the file looks like:

TOPOLOGY=xp1
#Note: Copy this below if you are using the Tenant Service in DEF
TENANT_SERVICE_HOST=xp1ts.localhost

Copy all the required variables in the file to the clipboard.
Go to the Sitecore container deployment folder on your local machine. Go to the folder for the Windows version and topology you are using, for example, compose\ltsc2019\xp1.
Open the .env file in an editor, and paste in the variables from the DEF .env-example file.

Add the following variables to the .env file:

DEF_IMAGE=scr.sitecore.com/sxp/modules/sitecore-def-xp1-assets:7.0.0-1809
TOOLING_IMAGE=scr.sitecore.com/tools/sitecore-docker-tools-assets:10.2.0-1809

Save the .env file.
From the DEF compose\<version>\<topology> folder, copy the docker-compose.override.yml file and paste it to the Sitecore container deployment compose\<version>\<topology> folder (where the docker.compose.yml file is).
If you are not using the Tenant Service in DEF, open the docker-compose.override.yml file, and remove the sections for the xdbautomationworker and id images.

Build the Docker images

When you have prepared the installation files, you must create Docker files for each role, and build the Docker images.

Note

For more information on image assets, see the documentation on how to add Sitecore modules.

To build the images:

Go to the Sitecore container deployment folder on your local machine. Go to the folder for the Windows version and topology you are using, for example, compose/ltsc2019/xp1. Create a subfolder and name it module.
In the module folder, create these subfolders:
- cm
- id
- xdbautomationworker
Note
The xdbautomationworker and id modules are only required if you are using the Tenant Service. If you are not using the Tenant Service, you do not have to create the folders and files for these two modules.
In each subfolder, create a new file and name it Dockerfile.

In the cm folder, in the Dockerfile file, enter the following instructions:

# escape=`

ARG BASE_IMAGE
ARG DEF_IMAGE
ARG TOOLING_IMAGE

FROM ${DEF_IMAGE} as def
FROM ${TOOLING_IMAGE} as tooling
FROM ${BASE_IMAGE}

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

WORKDIR C:\inetpub\wwwroot

# Add DEF module
COPY --from=def \module\cm\content\
COPY --from=def \module\transforms\ C:\transforms\

COPY --from=tooling \tools\ \tools\
RUN C:\tools\scripts\Invoke-XdtTransform.ps1 -Path C:\inetpub\wwwroot -XdtPath \transforms\cm

If you are using the Tenant Service, in the id folder, in the Dockerfile file, enter the following instructions:

# escape=`

ARG BASE_IMAGE
ARG DEF_IMAGE
ARG TOOLING_IMAGE

FROM ${DEF_IMAGE} as def
FROM ${TOOLING_IMAGE} as tooling
FROM ${BASE_IMAGE}

SHELL ["pwsh", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

WORKDIR C:\Identity

# Add DEF module
COPY --from=def \module\transforms\ C:\transforms\

COPY --from=tooling \tools\ \tools\
RUN C:\tools\scripts\Invoke-XdtTransform.ps1 -Path C:\Identity -XdtPath c:\transforms\id

If you are using the Tenant Service, in the xdbautomationworker folder, in the Dockerfile file, enter the following instructions:

# escape=`

ARG BASE_IMAGE
ARG DEF_IMAGE
ARG TOOLING_IMAGE

FROM ${DEF_IMAGE} as def
FROM ${TOOLING_IMAGE} as tooling
FROM ${BASE_IMAGE}

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]

# Add DEF MA module
COPY --from=def \module\xdbautomationworker\content C:\service\
COPY --from=def \module\transforms\C:\transforms\
COPY --from=tooling \tools\\tools\
RUN C:\tools\scripts\Invoke-XdtTransform.ps1 -Path C:\service -XdtPath \transforms\xdbautomationworker

In the compose\<version>\<topology>\docker-compose.override.yml file, add build instructions for each role.

For all topologies, you must add instructions for the cm role, for example:

services:
   cm:
     image: sitecore-def-${TOPOLOGY}-cm:${SITECORE_VERSION}
     build:
       context: ./module/cm
       args:
         BASE_IMAGE: ${SITECORE_DOCKER_REGISTRY}sitecore-${TOPOLOGY}-cm:${SITECORE_VERSION}
         DEF_IMAGE: ${DEF_IMAGE}
         TOOLING_IMAGE: ${TOOLING_IMAGE}

If you are deploying the XP1 topology, add instructions for the cd role as follows:

cd:
  image: sitecore-def-${TOPOLOGY}-cd:${SITECORE_VERSION}
  build:
     context: ./module/cm
     args:
       BASE_IMAGE: ${SITECORE_DOCKER_REGISTRY}sitecore-${TOPOLOGY}-cd:${SITECORE_VERSION}
       DEF_IMAGE: ${DEF_IMAGE}
       TOOLING_IMAGE: ${TOOLING_IMAGE}

If you are deploying XP0 or the XP1 topology, add instructions for the id and the xdbautomationworker roles, for example:

id:
  image: sitecore-def-${TOPOLOGY}-id:${SITECORE_VERSION}
  build:
     context: ./module/id
     args:
       BASE_IMAGE: ${SITECORE_DOCKER_REGISTRY}sitecore-id:${SITECORE_VERSION}
       DEF_IMAGE: ${DEF_IMAGE}
       TOOLING_IMAGE: ${TOOLING_IMAGE}
xdbautomationworker:
  image: sitecore-def-${TOPOLOGY}-xdbautomationworker:${SITECORE_VERSION}
  build:
      context: ./module/xdbautomationworker
     args:
       BASE_IMAGE: ${SITECORE_DOCKER_REGISTRY}sitecore-${TOPOLOGY}-xdbautomationworker:${SITECORE_VERSION}
       DEF_IMAGE: ${DEF_IMAGE}
       TOOLING_IMAGE: ${TOOLING_IMAGE}

In the Windows console, go to the folder containing the docker-compose.override.yml file. Run the following command:
```
docker-compose build
```
If you are deploying to Docker, continue with the instructions in Install Data Exchange Framework in Docker.
If you are deploying to Kubernetes, continue with the instructions in Walkthrough: Installing Data Exchange Framework in Azure Kubernetes Service.

Note

Some modifications to Sitecore deployments, such as adding connection strings or changing the web configuration files, require you to use configuration transforms to change the configuration files. For information on how to apply configuration transforms, see the Sitecore container development documentation container development documentation.