Understanding search sources
Feature availability is part of a phased rollout. Your organization may not see this functionality yet. It will become available when your environment is included in the rollout.
A search source determines what content is available in a search experience and how that content is searched.
When a visitor performs a search on your website, the results are returned from the source connected to the search component. A source contains both searchable content and the configuration that controls how that content is indexed, filtered, and ranked.
As well as defining where content is collected from, a source also contains the following configurable behavior settings:
- Fields - that determine how your data is interacted with in a search component.
- Rules - that allow you to reorder and tailor search results.
- Advanced settings - that support a wider range of customer queries.
The content itself is stored in a search index. Search components query this index rather than the original content location, enabling fast and configurable search experiences.
SitecoreAI supports three types of sources:
Content sources
A content source indexes structured content from the Content editor. When creating a content source, you specify a content template and SitecoreAI ingests published items based on that template. In addition to the indexed content, a content source stores field configurations, search rules, and advanced search settings.
Use a content source when your searchable content already exists in the Content editor, such as articles, events, products, or other structured content types.
Site sources
A site source indexes content directly from a website. SitecoreAI discovers pages using sitemaps, crawls those pages, extracts content, and stores the results in a search index. Site sources also include crawl rules, URL filtering policies, extraction settings, and field mappings.
Use a site source when you want to create search experiences from website content, including websites that are not managed in the Content editor.
Some websites use firewalls, bot protection, or web application firewall (WAF) services that can block crawlers. If your website restricts incoming traffic, you might need to allowlist SitecoreAI crawler IP addresses before the site can be indexed successfully.
Push sources
A push source indexes content submitted directly to SitecoreAI using the Ingestion Service API. SitecoreAI stores the submitted documents in the source's index, where connected search experiences can return them.
Use a push source when an external system is the system of record, when the content cannot be crawled, or when you need to submit content for near real-time indexing. Define the source's fields before submitting documents. Push sources do not retain a history of submitted documents, so to rebuild the index, you must submit the full set of documents again.