Using Solr field name resolution

Abstract

Set up global mapping rules to map SItecore field to Solr index fields

Note

This topic is valid for Sitecore 9.0 and later.

When you use Solr as search provider, you must follow some conventions about storing and querying documents stored in Solr. When Sitecore sends a document (Sitecore item) to Solr for indexing, the document must have a specific format:

• Each index field in the document must be a predefined Solr field.

For example, Item Name from Sitecore maps to _name in the Solr document. You must define the _name index field in the Solr schema before you can store the field.

• Alternatively, each index field in the Sitecore document can match the suffix of a predefined dynamic Solr field.

For example, the Created By field in Sitecore maps as parsedcreatedby_s in the Solr document because it matches the *_s dynamic field. You must define the *_s dynamic index field in the Solr schema before you can store the field.

Because you can add a large number of Sitecore item fields anytime, adding a new Solr index field for every single Sitecore item field to the Solr schema is not practical, if not simply impossible.

You can avoid this issue by using the dynamic fields Solr feature to set up global mapping rules for any indexed field. This works for Sitecore item fields as well as for computed index field.

You specify the mapping in the Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config file.

You map Sitecore field types to dynamic field types in the AddFieldByFieldTypeName and AddTypeMatch sections of the configuration file.

For example, if Sitecore item A uses templateA and field1 is a number field, then field1 is indexed as field1_tf by this configuration:

<fieldType fieldTypeName="number" returnType="float" />
<typeMatch typeName="float" type="System.Single"  fieldNameFormat="{0}_tf" settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" />


You map Sitecore computed fields to dynamic field types in the AddComputedIndexField and AddTypeMatch sections of the configuration file.

For example, to index the calculateddimension computed field as calculateddimension_sm:

<field fieldName="calculateddimension" returnType="stringCollection">Sitecore.ContentSearch.ComputedFields.CalculatedDimension,Sitecore.ContentSearch</field>
<typeMatch typeName="stringCollection" type="System.Collections.Generic.List1[System.String]" fieldNameFormat="{0}_sm" multiValued="true" settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" />


You can override the configuration in the AddFieldByFieldTypeName section on a per field basis in the AddFieldByFieldName section.

For example, the title field is indexed as title_t:

<field fieldName="title" returnType="text" />
<typeMatch typeName="text" type="System.String" fieldNameFormat="{0}_t" cultureFormat="_{1}" settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" />


The names of Solr index fields are resolved in an almost identical way when searching as they are when indexing.

There is, however, a minor difference because information about the field type and the type of the value is missing. For example, if you search for title:home, Sitecore does not know what field type the title field is nor does Sitecore know what type title is (string, number, and so on).

Therefore, Sitecore Solr field resolution will find the field type of the title field internally and resolve it. This information is then used to determine the field name format to use when searching the Solr index.

For example, if a user searches an item bucket with a query that includes title:z1000, Sitecore finds the template that has the title field.

Here, the template is Sample Item and the field type of title is Single-Line Text. This is matched against the typeMatch configuration, and this gives title_t as the Solr index field for searching, so the Solr query becomes /select?q=title_t:z1000.

The configuration is the following:

<fieldType fieldTypeName="html|rich text|single-line text|multi-line text|text|memo|image|reference" returnType="text" />
<typeMatch typeName="text" type="System.String" fieldNameFormat="{0}_t" cultureFormat="_{1}" settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" />


You can also use a Sitecore LINQ query:

var queryable = index.CreateSearchContext().GetQueryable< CustomSearchResultItem >();
var q = queryable.Where(i => i.Field1.Equals("search term")).ToList();
public class CustomSearchResultItem : SearchResultItem
{
[IndexField("Field1")]
[DataMember]
public virtual string Field1
{
get;
set;
}
}


When you search using a Sitecore LINQ query and it has a strongly typed property, the runtime type of the property is used to resolve the configuration when the Solr query is constructed.

However, in a case where multiple .Net run time types are found in the AddTypeMatch section, Sitecore takes one more step to locate the field type of the field by querying Solr. Sitecore uses the template resolver internally for this. The template resolver constructs the query as follows, and sends it to Solr to find the template:

/select?q=_name:(field1)&fq=_templatename:("Template+field")&fq=_indexname:(sitecore_master_index)&rows=10&version=2.2}


In the above example, the field1 property has type string. There are two configurations in AddTypeMatch for string. However, Sitecore will pick the configuration with fieldNameFormat {0}_t because the template resolver returns the field type Single-Line Text. The Single-Line Text configuration defined in the AddFieldByFieldTypeName section matches text in the AddTypeMatch section. Therefore, the field name resolves as field1_t.

The final Solr query is /select?q=_field1_t:”search term”&fq=_indexname:(sitecore_master_index).

<fieldType fieldTypeName="html|rich text|single-line text|multi-line text|text|memo|image" returnType="text" />


Finally, the following example shows how you use a Sitecore LINQ query to search for terms with the indexer:

<typeMatch typeName="text" type="System.String" fieldNameFormat="{0}_t" cultureFormat="_{1}" settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" />
var queryable = index.CreateSearchContext().GetQueryable< SearchResultItem >();
var q = queryable.Where(i => i["field1"] == "search term").ToList();


When you search for a term with a Sitecore LINQ query and filter with the indexer, the behavior and resolution is the same as previously described. Because there is no run-time type to map against, Sitecore finds which template has the title field and locates the field type. Sitecore uses the template resolver for this. Finally, Sitecore uses the field type to resolve the configuration, just as described earlier.

For example, if field1 is a Single-Line Text field, the following query will produce /select?q=_field1_t:”search term”&fq=_indexname:(sitecore_master_index):

queryable.Where(i => i["field1"] == "search term").ToList();
<typeMatch typeName = "text" type = "System.String" fieldNameFormat = "{0}_t" cultureFormat = "_{1}" settingType = "Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, Sitecore.ContentSearch.SolrProvider" / >
<fieldType fieldTypeName = "html|rich text|single-line text|multi-line text|text|memo|image|reference" returnType = "text" / >


Note

Sitecore finds the template with required field and resolves it appropriately. If you do not index the templates, Sitecore cannot resolve dynamic fields. You can solve this issue by having two crawlers: one that covers the content you want to index, and another one that covers the templates in /sitecore/Templates.

There are some situations where the Sitecore index resolution cannot be guaranteed to give consistent results.

If, for example, the field1 field is defined in more than one template, the query shown below can give inconsistent results. This is especially so if field1 is defined as having field type Single-Line Text in one template and Droplist in another template:

var queryable = index.CreateSearchContext().GetQueryable < CustomSearchResultItem > ();
var q = queryable.Where(i => i.Field1.Equals("search term")).ToList();
public class CustomSearchResultItem: SearchResultItem {
[IndexField("Field1")]
[DataMember]
public virtual string Field1 {
get;
set;
}
}


A fuller explanation is that the field1 field is defined as string in the CustomSearchResultItem POCO class. Furthermore, both field types resolve to text and stringtypeMatch in the AddTypeMatch section, while both configurations have System.String run-time type. Therefore, there is ambiguity about which configuration to choose.

The following example can also produce inconsistent results:

var queryable = index.CreateSearchContext().GetQueryable< SearchResultItem >();
var q = queryable.Where(i => i["field1"] == "search term").ToList();


If field1` is defined as a Number field in one template and as Single-Line Text in another template, the template resolver returns two results for both templates. Sitecore chooses the first result the template resolver returns, but the order of the results is unpredictable.