Using the Ingestion API to add content to an index

Sitecore Search exposes a RESTful API called the Ingestion API that you can use to create, update, or delete index documents, and check the status of existing ingestion requests.

Note

You can add documents to the index of a source created by an administrator. You cannot use the Ingestion API to create a source.

The primary purpose of the Ingestion API is to push content to either a pull source index or a push source index.

Pushing content to a pull source index

Crawling can affect performance, because a crawler must inspect each item in an index. We recommend you use the Ingestion API to make frequent, small updates to a pull source index, because doing so is less resource heavy. You can add, edit, or delete index documents at any time.

Both manually triggered and scheduled crawls overwrite the index and any changes made using the Ingestion API. To retain your changes, remember to update your items before the next crawl.

Important

Because pull sources such as crawlers do not have an initialized index by default, you need to run an indexing job with them before you use the Ingestion API to make incremental updates.

Pushing content to a push source index

You can use the Ingestion API to add index documents to a push source index. In a push source, an administrator generates an empty index by creating an API push source. Then, you can use the Ingestion API to add index documents to this index.

Tip

You can use the Ingestion API with webhooks to add or modify index documents. Webhooks are automated messages sent from apps when something happens, and they instantly notify the Ingestion API of content changes, ensuring the search index is updated in real-time.

Using the Ingestion API

The Sitecore Search Ingestion API only supports HTTPS. For requests that require a body, you must send it as JSON.

To help you work with the Ingestion API:

  • On the menu bar, in the Developer Resources > API Access section, you can find the base URL.

  • View the Open API or swagger reference to see the detailed data model and descriptions of the objects and keys.

Ingestion API endpoints and when you can use them.

The Ingestion API has the following endpoints that you can use to create, update, and delete index documents. You can also use it to view the status of a request to the Ingestion API.

Note

In the following endpoint examples, text in braces such as {:placeholder-token:}{BASE_URL}{/:placeholder-token:} indicates a placeholder that you need to replace with the correct value for your implementation.

Creating an index document

You can use the following endpoints to create an index document:

  • Create an index document by passing attribute values - POST call to {base-URL}/ingestion/v1/domains/{domain ID}/sources/{sourceID}/entities/{entityID}/documents?locale={locale}

  • Create an index document from a file on your system by using a document extractor - POST call to {base-URL}/ingestion/v1/domains/{domainID}/sources/{sourceID}/entities/{entityID}/file/{documentID}?locale={locale}

  • Create an index document from a URL by using a document extractor - POST call to {base-URL}/ingestion/v1/domains/{domainID}/sources/{sourceID}/entities/{entityID}/url/{documentID}?locale={locale}

Important

If you receive an HTTP 429 response, the rate of incoming requests has exceeded the platform’s rate limits — either due to a sudden spike in traffic or high cumulative volume over time. When this happens, the document isn't ingested or stored and you must try sending the request again.

To reduce the risk of this happening:

  • Implement logic to detect HTTP 429 responses.

  • Use exponential backoff to manage retries.

  • Check whether the response contains the Retry-After header, and if so, send a request after the specified time.

Contact support for further guidance.

Updating an index document

You can use the following endpoints to update an index document:

Deleting an index document

You can use the following endpoint to delete an index document:

  • Delete an index document - DELETE call to {base-URL}/ingestion/v1/domains/{domain ID}/sources/{sourceID}/entities/{entityID}/documents/{documentID}?locale={locale}

Viewing request status

You can use the following endpoint to view the status of an ingestion request:

Authentication

If your implementation doesn't have a subdomain, you must authenticate with an API key that has the ingestion scope in order to access the Ingestion API. You can get the API key in Search , in the Developer Resources > API Access section.

Note

You can't use an access token to authenticate calls to the Ingestion API.

Mandatory parameters

When you send requests to the Ingestion API, there are two types of mandatory parameters:

  • Parameters that are mandated by the data model. All endpoints require the domain, source , and entity parameters. Some endpoints require other mandatory parameters.

  • Parameters for attributes that are marked as required in your domain. For example, if your domain requires the product attribute and you don't pass product:<value> or logic to extract a value for product, Search won't create the index document.

Ensure that you send all the mandatory parameters to get a successful response.

Do you have some feedback for us?

If you have suggestions for improving this article,