---
title: "Create a JavaScript document extractor"
slug: "create-a-javascript-document-extractor"
abstract: "Use JavaScript to extract attribute values when you create a document extractor in a Sitecore Search crawler source. Use tags, rules, and URL matching."
---
# Create a JavaScript document extractor

Create a JavaScript document extractor that uses a function to extract attribute values. This type extractor is typically used this method in complex scenarios where an XPath expression will not suffice.

> [!note]
> You can use a JavaScript document extractor with the [advanced web crawler](https://doc.sitecore.com/search/en/users/search-user-guide/index-items/additional-configurations-and-settings/configuring-document-extractors/create-a-javascript-document-extractor.html "Create a JavaScript document extractor") and the [API crawler](https://doc.sitecore.com/search/en/users/search-user-guide/index-items/additional-configurations-and-settings/configuring-document-extractors/configure-an-api-crawler.html#UUID-35c9f696-feb4-3e49-7b01-dd161da88f58_section-idm4601236874884833662956419681 "Configure a document extractor") sources.

To use JavaScript to extract attribute values from URLs that match the condition of a specific GLOB pattern:

1. On the menu bar, click **Sources**, and select the source you created.
2. On the **Source Settings** page, next to **Document Extractors**, click ![pencil_grey.png](https://doc.sitecore.com/api/markdown-image/search/en/users/search-user-guide/images/pencil_grey.png) **Edit**.
3. To create a [document extractor](urn:resource:component:1008083), on the **Document Extractors** page:
    - In the **Name** field, enter a meaningful name for the extractor.

        For example, enter *Home loans JS extractor*.

    - In the **Extractor Type** drop-down menu, click **JS**.
    - Optionally, to ensure that this extractor's logic only applies to [URLs that match](https://doc.sitecore.com/search/en/users/search-user-guide/index-items/additional-configurations-and-settings/configuring-document-extractors.html#UUID-bf3b8a27-e317-f7e3-9fe5-88d81969a12c_section-idm4597072673881633832915676016 "URLs to match") a certain pattern, configure **URLs to Match**. To do this, in the **URLs To Match** field, click ![Plus_add__grey.png](https://doc.sitecore.com/api/markdown-image/search/en/users/search-user-guide/images/Plus_add__grey.png) **Add Matcher**, select the **TYPE** of expression you want to use, and enter the **VALUE** of that expression.

        For example, to crawl all URLs in this format: *<some text>/homeloans/<some text>*, you can click **Glob Expression** in the **TYPE** drop-down menu and enter the **VALUE** as `**/homeloans/**.*` .

4. In the **Taggers** section, click **Edit** ![pencil_grey.png](https://doc.sitecore.com/api/markdown-image/search/en/users/search-user-guide/images/pencil_grey.png) next to the first [tag](https://doc.sitecore.com/search/en/users/search-user-guide/index-items/additional-configurations-and-settings/crawler-tags.html "Crawler tags"). Usually, it is the *content* tag.

    In the tag editor, you see a sample JavaScript function that returns the *description*, *name*, *type*, *image_url* and *url* attributes. Search provides this sample to help with configuration.

    > [!note]
    > In a document extractor, you can create multiple taggers where each is linked to a unique tag.
    >
    > For example, if you have one tagger with a JavaScript function that defines how to extract five attributes, you get *one* set of index documents where each document has *five* attributes. However, if you have three taggers, and each has a JavaScript function that defines how to extract one, you get *three* sets of index documents where each document has *one* attribute.

5. In the tag editor, edit the sample function or paste a new JavaScript function that returns attribute values.

    > [!note]
    > The function must use [Cheerio](https://cheerio.js.org/) syntax and must return an array of objects.

    For example, paste:

    ```javascript
    function extract(request, response) {
      $ = response.body;
      curr_url = request.url;

      let url = request.url;
      let type, subtype;
      type = 'homes';

      if (url.includes('/moving/')) {
        subtype = 'Moving';
      } else if (url.includes('/plans-products/')) {
        subtype = 'Plans and products';
      } else if (url.includes('/partnerships/')) {
        subtype = 'Partnerships';
      } else if (url.includes('/referrals/')) {
        subtype = 'Referrals';
      } else {
        subtype = 'misc';
      }

      let name, subtitle, description, image, date, year;

      name = $('h1[class="main-title"]').text() !== null ? $('h1[class="main-title"]').text().replace(/\s\s+/g, ' ').trim() : '';
      subtitle = $('h2[class="htmlSubtitle"]').text() !== null ? $('h2[class="htmlSubtitle"]').text().replace(/\s\s+/g, ' ').trim() : '';
      description = $('div[class="bodycopy"]').text() !== null ? $('div[class="bodycopy"]').text().replace(/\s\s+/g, ' ').trim() : '';
      image = $('div[class="hero-img"]').find('img').attr('src') !== null ? 'https://www.txu.com' + $('div[class="hero-img"]').find('img').attr('src') : '';

      return [{
        'name': name,
        'subtitle': subtitle,
        'description': description,
        'image_url': image,
        'type': type,
        'subtype': subtype,
        'url': url
      }];
    }
    ```

    This function uses the following logic to get attributes:

    - `type` - use a fixed value, for example, *homes*.
    - `subtype` - variable that can take one of five values depending on the URL path.
    - `url` - get this from the `request.url`.
    - `name` - if the text from the `<h1 class="main-title">` tag has a value, use a modified form of that text as the *Name*. If it is empty, use an empty string for *Name*.
    - `subtitle` - if the text from the `<h2 class="htmlSubtitle">` tag has a value, use a modified form of that text as the *Name*. If it is empty, use an empty string for *Name*.
    - `description` - if the text from the `<bodycopy>``<div>` class has a value, use a modified form of that text as the *Description*. If it is empty, use an empty string for *Description*.
    - `image_url` - if the `img src` attribute of the `hero-img``div` class has a value, use a modified form of that URL. If it is empty, use an empty string for *Image_url*.
6. In the tag editor, click **Save**.
7. Optionally, to extract attributes for another tag, click ![Plus_add__grey.png](https://doc.sitecore.com/api/markdown-image/search/en/users/search-user-guide/images/Plus_add__grey.png) **Add Tagger**, and in the **Tag** drop-down menu, click a tag and repeat Steps 5 and 6.
8. On the **Document Extractors** page, click **Save**.