---
title: "Walkthrough: Creating and running a pipeline"
slug: "walkthrough--creating-and-running-a-pipeline"
abstract: "How to set up a pipeline to run a data synchronization from a text file to Sitecure using Sitecore Data Exchange Framework."
---
# Walkthrough: Creating and running a pipeline

This walkthrough is part three of [Synchronizing data from a text file to Sitecore walkthrough series](https://doc.sitecore.com/xp/en/developers/def/80/data-exchange-framework/synchronizing-data-from-a-text-file-to-sitecore-walkthrough-series.html "Synchronizing data from a text file to Sitecore walkthrough series") and describes how to create a pipeline that can run a data synchronization between a text file and Sitecore.

The walkthrough describes how to:

- Add a pipeline that reads the source file.
- Add a pipeline step to read data from a source file.
- Add a pipeline step to iterate the data read from a source file.
- Add a pipeline to sync a single row from the source file.
- Add a pipeline step to resolve the target item.
- Add a pipeline step to apply mappings.
- Add a pipeline step to update the target item.
- Add a pipeline batch
- Test a pipeline batch

<details>
<summary><h2>Add a pipeline to read the source file</h2></summary>

You must implement a pipeline that reads data from the text file, iterates the data, and passes the data to the pipeline that handles a single row of data.

To add a pipeline to read the source file:

1. In the Content Editor, navigate to */sitecore/System/Data Exchange/File System Provider Test Tenant/Pipelines*.

    ![Tenant Pipelines folder in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipelines-single-row-finished.png)

2. In the right-hand pane, click *Pipeline*. Name the new item *Read City Info from File Pipeline*.
3. Drag and drop the new item to the first position in the *Pipelines* folder.

    ![Moving the Read City Info from File pipeline in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-read-data-no-children-ordered.png)

> [!note]
> The order of the pipelines in the folder is not important. However, when a pipeline calls another pipeline, it is helpful to place the pipeline that calls the other pipeline before the pipeline it calls.

</details>

<details>
<summary><h2>Add a pipeline step to read data from a source file</h2></summary>

You must configure a pipeline step that can read the data from the text file specified by the text file endpoint. The pipeline step must also put a pointer to the data in a location where subsequent pipeline steps can access it.

To add a pipeline step to read data from a source file:

1. In the Content Editor, navigate to the pipeline item, for example, *Read City Info from File Pipeline*.

    ![Read City Info from File pipeline in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-read-data-no-children-ordered.png)

2. In the right-hand pane, click *Read Text File Pipeline Step*. Name the new item *Read Text File*.
3. In the content tree, select the new item.

    ![Read Text File pipeline step in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-read-text-file.png)

4. In the right-hand pane, specify the following field values:

    | Field | Value |
    | --- | --- |
    | **Endpoint From** | *File System/City Information File* |

5. Save the item.

</details>

<details>
<summary><h2>Add a pipeline step to iterate the data read from a source file</h2></summary>

The first step in this pipeline adds a pointer to a location where subsequent pipeline steps can access it. You must configure the second step to access the data so it can iterate over the lines read from the text file, and pass the data to the other pipeline that handles the data.

To add a pipeline step to iterate data:

1. In the Content Editor, navigate to the pipeline item *City Info from File to City Info Item Sync Pipeline*.

    ![Read City Info from File pipeline in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-read-data-1-child.png)

2. In the right-hand pane, click *Iterate Data and Run Pipelines Pipeline Step*. Name the new item *Iterate Rows from Text File and Run Pipeline*.
3. In the content tree, select the new item.

    ![Iterate Rows from Text File and Run Pipeline pipeline step in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-iterate.png)

4. In the right-hand pane, in the **Settings** section, specify the following field values:

    | Field | Value |
    | --- | --- |
    | **Pipelines** | *Pipelines/City Info from File to City Info Item Sync Pipeline* |
    | **Iterable Data Location** | *Pipeline Context Iterable Data* |
    | **Data Location** | *Pipeline Context Source* |

5. Save the item.
6. Drag and drop the new item to the second position in the *Pipelines* folder.

    ![Move the Iterate Rows from Text File and Run Pipeline pipeline step in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-iterate-ordered.png)

</details>

<details>
<summary><h2>Add a pipeline to sync a single row from the source file</h2></summary>

The synchronization process you are modeling consists of two pipelines. The first pipeline reads data from a source object, which is a text file. The second pipeline handles a single row from the text file.

Because the first pipeline calls the second pipeline, you must configure the second pipeline before you can configure the first.

To add the pipeline that handles a single row from the text file:

1. In the Content Editor, in the content tree, select the tenant. Navigate to *Pipelines*.

    ![Tenant pipelines in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipelines-empty.png)

2. In the right-hand pane, on the **Folder** tab, click **Pipeline** to add a new item and name it *City Info from File to City Info Item Sync Pipeline*.

</details>

<details>
<summary><h2>Add a pipeline step to resolve the target item</h2></summary>

The first pipeline step must check if there is an existing Sitecore item that matches the value from the text file row.

If the pipeline finds a matching item, it must set that item as the target object in the pipeline context. This means that during the mapping process, the pipeline updates the matching item with the new values.

If the pipeline does not find a matching item, it must create a new item, and set the new item as the target object in the pipeline context.

To set up these actions in the pipeline:

1. In the Content Editor, navigate to the pipeline item you want to create a step for, for example, *City Info from File to City Info Item Sync Pipeline*.

    ![City Info From File to City Info Item Sync pipeline in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-single-row-no-children.png)

2. In the right-hand pane, click *Resolve Sitecore Item Pipeline Step*. Name the new item *Resolve City Info Item*. Select the new item.

    ![Resolve City Info item in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-resolve-city-info-item.png)

3. In the right-hand pane, specify the following field values:

    | Field | Value |
    | --- | --- |
    | **Template for New Item** | *Templates/User Defined/City Information* |
    | **Item Name Value Accessor** | *Data Access/Value Accessor Sets/Providers/File System/City Information File Fields/City from City Information File* |
    | **Endpoint From** | *Sitecore/Sitecore Database Endpoint* |
    | **Identifier Value Accessor** | *Data Access/Value Accessor Sets/Providers/File System/City Information File Fields/Identifier from City Information File* |
    | **Identifier Object Location** | *Pipeline Context Source* |
    | **Resolved Object Location** | *Pipeline Context Target* |
    | **Parent for Item** | *sitecore/content/Cities* |
    | **Matching Field Value Accessor** | *Data Access/Value Accessor Sets/Providers/Sitecore/City Information Item Fields/Identifier on City Information Item* |

4. Save the item.

</details>

<details>
<summary><h2>Add a pipeline step to apply mappings</h2></summary>

The [first step](https://doc.sitecore.com/xp/en/developers/def/80/data-exchange-framework/walkthrough--creating-and-running-a-pipeline.html#UUID-d15188e3-bdf9-00f0-dbe1-9b882d0442cd_N1652779327922) in this pipeline ensures that a target object is available, either by finding an existing item or creating a new one. In the second step, you must use the value mapping set you configured in [Add Value Mappings](https://doc.sitecore.com/xp/en/developers/def/80/data-exchange-framework/walkthrough--creating-components-to-read-and-write-data.html#UUID-797f4b9c-8bf4-0a27-ecbd-231385a144f6_section-idm93305559151838 "Add value mappings") to map values from the source object to the target object.

To add a pipeline step to apply mappings:

1. Navigate to the pipeline item *City Info from File to City Info Item Sync Pipeline*.

    ![City Info from File to City Info Item Sync pipeline in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-single-row-1-step.png)

2. In the right-hand pane, click *Apply Mapping Pipeline Step*. Name the new item *Apply Mapping*. Select the new item.

    ![Apply Mapping item in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-apply-mapping.png)

3. In the right-hand pane, set the following field value:

    | Field | Value |
    | --- | --- |
    | **Mapping Set** | *Value Mapping Sets/Row from City Information File to City Information Item* |

4. Save the item.
5. Drag and drop the item to the second position in pipeline.

    ![Moving the Apply Mapping item in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-apply-mapping-ordered.png)

</details>

<details>
<summary><h2>Add a pipeline step to update the target item</h2></summary>

After mappings are applied, the changes to the target item must be saved to the Sitecore database.

To add a pipeline step to update the target item:

1. In the Content Editor, navigate to the pipeline item you want to add the step to, for example, *City Info from File to City Info Item Sync Pipeline*.
2. In the right-hand pane, click *Update Sitecore Item Pipeline Step*. Name the new item, for example, *Update Sitecore Item*. Select the new item.

    ![Update Sitecore Item pipeline step in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-update-sitecore-item.png)

3. In the right-hand pane, specify the following field values:

    | Field | Value |
    | --- | --- |
    | **Item Location** | *Pipeline Context Target* |
    | **Endpoint To** | *Sitecore/Sitecore Database Endpoint* |

4. Save the item.
5. Drag and drop the new item to the third position in the pipeline.

    ![Move the Update Sitecore Item pipeline step in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-step-update-sitecore-item-ordered.png)

</details>

<details>
<summary><h2>Add a pipeline batch</h2></summary>

To run a pipeline, you must configure a pipeline batch. This is basically the runtime for a pipeline.

To configure a pipeline batch:

1. In the Content Editor, navigate to */sitecore/System/Data Exchange/File System Provder Test Tenant/Pipeline Batches*.

    ![Tenant Pipeline Batches in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-batches-no-children.png)

2. In the right-hand pane, click *Pipeline Batch*. Name the new item, for example, *City Info from File to City Info Item Sync Pipeline Batch*.
3. In the content tree, select the new item.

    ![City Info from File to City Info Item Sync pipeline batch in the content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-batch.png)

4. In the right-hand pane, specify the following field value:

    | Field | Value |
    | --- | --- |
    | **Pipelines** | *Pipelines/Read City Info from File Pipeline* |

5. Save the item.

</details>

<details>
<summary><h2>Test a pipeline batch</h2></summary>

You can run a pipeline batch in a number of ways. The easiest way is to run it manually.

To test a pipeline batch:

1. In the Content Editor, navigate to */sitecore/System/Data Exchange/File System Provder Test Tenant/Pipeline Batches/City Info from File to City Info Item Sync Pipeline Batch*.

    ![Pipeline batches in content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-batch.png)

2. On the ribbon, click **Run Pipeline Batch**.

    ![Run Pipeline Batch option in menu ribbon](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/run-pipeline-batch-button.png)

3. Click **OK**.

    ![Pipeline batch started message](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/pipeline-batch-started.png)

    > [!note]
    > In this example, the text file only contains a few rows. In addition, the logic in the pipelines is very limited. This means the pipeline batch will finish almost instantly.
    >
    > You can tell if the pipeline batch is running by checking if the Stop Pipeline Batch button is enabled. This button does not automatically refresh, so you might have to click the pipeline batch to refresh the UI.

4. In the content tree, navigate to */sitecore/content/Cities*.

    ![Content in content tree](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/cities-populated.png)

    You can see an item for each of the rows in the text files.

5. Click the item you want to review, for example, the *Copenhagen* item. You can see the values from the row in the text file that corresponds to the item.

    ![Data item details](https://doc.sitecore.com/api/markdown-image/xp/en/developers/def/80/data-exchange-framework/images/copenhagen.png)

</details>