Using Solr field name resolution

Abstract

Overview of how Sitecore automatically maps document (item) field names to Solr field names.

Solr as a search provider dictates conventions for indexing and searching documents. When Sitecore sends a document (item) to Solr for indexing, the item fields must map to Solr fields. An example of mapping is when the Sitecore Item Name field maps to the Solr _name field.

Unfortunately, it is not practical to manually match large numbers of Sitecore fields to Solr fields. However, you can use Solr dynamic fields to perform automatic mappings for Sitecore fields.

An example of automatic mapping is when Sitecore maps the Sitecore parsedcreatedby computed string field to the Solr parsedcreatedby_s dynamic string field, because parsedcreatedby is associated with the Sitecore string index field type that is mapped to the *_s field name format.

All mappings are specified in the Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config file.

The current configuration covers every Sitecore field type. However, if a mapping is missing for a field type, Sitecore tries to map the field name by matching it against the fields defined in the Solr schema. If no match is found, Sitecore uses the field value type to determine the mapping.

Consider, for example, a Sitecore field named amount. It holds numbers and is therefore a Sitecore number type field. If the Sitecore field is not manually mapped by name to an index field type, Sitecore automatically maps it by Sitecore field type to a Solr dynamic field, in the following order:

  1. Sitecore maps the Sitecore number type field to the Solr float type field in the AddFieldByFieldTypeName section:

    <fieldTypes hint="raw:AddFieldByFieldTypeName">
      <fieldType fieldTypeName="number" returnType="float" />
      ...
    </fieldType>
  2. Sitecore maps the Solr float type field to the {0}_tf field name format in the AddTypeMatch section:

    <typeMatches hint="raw:AddTypeMatch">
      <typeMatch
        typeName="float"
        type="System.Single"
        fieldNameFormat="{0}_tf"
        settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration,
          Sitecore.ContentSearch.SolrProvider"
      />
      ...
    </typeMatches>
  3. Sitecore replaces the {0} part with the Sitecore amount field name to become the Solr amount_tf field name.

  4. Sitecore indexes the Sitecore number type field named amount as a Solr dynamic float type field named amount_tf.

Overriding mappings by field name

Sitecore can override the mappings in the AddFieldByFieldTypeName section for each field in the AddFieldByFieldName section. You can use this feature, for example, to index the Sitecore title field as a Solr text type field named title_t. Sitecore uses the Sitecore field name and not the field type to match the Solr field type:

<fieldTypes hint="raw:AddFieldByFieldName">
  <field fieldName="title" returnType="text" />
  ...
</fieldTypes>

When Sitecore looks for a field name, it tries to match the exact name. If there is no exact match, Sitecore removes any cultural suffixes and Solr dynamic field suffixes and tries again. For example, the following field names will be mapped to the title field:

  • Title_en - as _en matches the english ISO language code.

    The same applies for _da, _de, and so on.

  • Title_t - as the _t suffix matches the text type format or more specifically the dynamic field *_t.

    The same applies for _s, _sm, and so on.

Indexing computed fields

Sitecore maps Sitecore computed fields to Solr dynamic fields in the AddComputedIndexField and AddTypeMatch sections. For example, Sitecore maps the Sitecore calculateddimension computed field to the Solr stringCollection type field and indexes it as calculateddimension_sm:

<fields hint="raw:AddComputedIndexField">
  <field fieldName="calculateddimension" returnType="stringCollection">
  Sitecore.ContentSearch.ComputedFields.CalculatedDimension,Sitecore.ContentSearch
  </field>
  ...
</fields>

<typeMatches hint="raw:AddTypeMatch">
  <typeMatch
    typeName="stringCollection"
    type="System.Collections.Generic.List`1[System.String]"
    fieldNameFormat="{0}_sm"
    multiValued="true"
    settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration,
      Sitecore.ContentSearch.SolrProvider"
  />
  ...
</typeMatches>

Sitecore resolves fields during searching in a similar way.

For example, when searching for title:Casablanca, Sitecore resolves the title field type to a Solr field name format in the following order:

  1. Resolve by name.

    Sitecore removes any language suffixes (_en, _da, and so on) or dynamic field suffixes (_t, _s, _sm, and so on) from the field name and then looks up the AddFieldByFieldName configurations to find any matches. If no match is found, Sitecore looks for a match in the Solr field names defined in the Solr schema.

  2. Resolve by CLR type of search value.

    If Sitecore cannot resolve the field by name, it uses the CLR type to match it with the configured AddTypeMatch. In this example, the CLR type of Casablanca is System.String and the first AddTypeMatch configuration for System.String is used to resolve the title name. In the following AddTypeMatch configurations, the first entry is lowercaseString and the title field is therefore resolved as title_ls.

    <typeMatches hint="raw:AddTypeMatch">
                  <typeMatch typeName="lowercaseString" 
                    type="System.String"  
                    fieldNameFormat="{0}_ls" 
                    settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, 
                    Sitecore.ContentSearch.SolrProvider" />
                  <typeMatch typeName="string" 
                    type="System.String" 
                    fieldNameFormat="{0}_s" 
                    settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, 
                    Sitecore.ContentSearch.SolrProvider" /> 
                  <typeMatch typeName="text" 
                    type="System.String" 
                    fieldNameFormat="{0}_t" 
                    cultureFormat="_{1}" 
                    settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, 
                    Sitecore.ContentSearch.SolrProvider" 
                  />
                </typeMatches>
    
  3. Resolve by finding the template.

    If Sitecore cannot resolve the field by name or by CLR type, it looks for a template field with the same name. If more than one template field is found, the first one is used. Once Sitecore identifies the template field, the type of the field is then matched against AddFieldByFieldTypeName.

Searching with LINQ

When Sitecore uses LINQ to run search requests with strongly typed properties, Sitecore uses the .NET runtime type of the property to resolve the Solr field name in the AddTypeMatch section:

var queryable = index.CreateSearchContext().GetQueryable< CustomSearchResultItem >();
var q = queryable.Where(i => i.Airport.Equals("search term")).ToList();

public class CustomSearchResultItem : SearchResultItem {
  [IndexField("Airport")]
  [DataMember]
  public virtual string Airport { get; set; }
}

However, in situations when Sitecore can map a field type name to multiple .NET runtime types in the AddTypeMatch section, Sitecore uses the template resolver to inquire Solr for the template and the field type.

For example, the Sitecore airport property is a System.String type that maps to both the string and text .NET runtime types in the AddTypeMatch section. This results in two field name formats ({0}_s and {0}_t) and two Solr dynamic field names (airport_s and airport_t):

<typeMatches hint="raw:AddTypeMatch">
  <typeMatch
    typeName="string"
    type="System.String"
    fieldNameFormat="{0}_s"
    settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, 
      Sitecore.ContentSearch.SolrProvider"
  />
  <typeMatch
    typeName="text"
    type="System.String"
    fieldNameFormat="{0}_t"
    cultureFormat="_{1}"
    settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration, 
      Sitecore.ContentSearch.SolrProvider"
  />
  ...
</typeMatches>

Sitecore uses the template resolver to query Solr for the template and the field type, in the following order:

  1. Sitecore sends the resolver request to Solr, for example:

    /select?q=_name:(airport)&fq=_templatename:("Template+field")&fq=_indexname:(sitecore_master_index)&rows=10&version=2.2}

    Important

    If two templates both define the same field, for example the airport field, the template resolver returns two results in an undetermined order. Sitecore always chooses the first result, making the resolution inconsistent. This is especially a problem if airport is defined as a Single-Line Text field type in one template and a Droplist field type in another template.

  2. Solr returns the single-line text field type that Sitecore maps to the Solr text field type in the AddFieldByFieldTypeName section:

    <fieldTypes hint="raw:AddFieldByFieldTypeName">
      <fieldType
        fieldTypeName="html|rich text|single-line text|multi-line text|text|memo|image" 
        returnType="text"
      />
    </fieldTypes>
  3. Sitecore maps the Solr text field type to the {0}_t field name format in the AddTypeMatch section:

    <typeMatches hint="raw:AddTypeMatch">
      <typeMatch
        typeName="text"
        type="System.String"
        fieldNameFormat="{0}_t"
        cultureFormat="_{1}"
        settingType="Sitecore.ContentSearch.SolrProvider.SolrSearchFieldConfiguration,
          Sitecore.ContentSearch.SolrProvider"
      />
    </typeMatches>
  4. Sitecore replaces the {0} part with the Sitecore airport property name resolving it to the Solr airport_t field.

  5. Sitecore sends the final search query to Solr:

    /select?q=_airport_t:”search term”&fq=_indexname:(sitecore_master_index)

Sitecore Content Search is dependent on templates being indexed to deliver complete and satisfying search results based on correct field type resolutions. Examples of configurations that leaves the templates unindexed are:

  • The Root tag of the crawler, which does not include the /sitecore/templates sub-tree:

    <crawler
      type="Sitecore.ContentSearch.SitecoreItemCrawler,
        Sitecore.ContentSearch">
      <Database>master</Database>
      <Root>/sitecore/content</Root>
    </crawler>
  • The index document options, which are configured to exclude the template field items:

    <exclude hint="list:AddExcludedTemplate">
      <TemplateField>{455A3E98-A627-4B40-8035-E683A0331AC7}</TemplateField>
    </exclude>

If your configuration requires it, add the TemplateIndexName tag to ensure that the templates are indexed. The tag must point to an index whose crawler indexes all Template Field items under the /sitecore/templates root:

<index
  id="sitecore_master_index"
  type="Sitecore.ContentSearch.SolrProvider.SolrSearchIndex, 
    Sitecore.ContentSearch.SolrProvider">
  ...
  <TemplateIndexName>TEMPLATE INDEX NAME</TemplateIndexName>
</index>

With the TemplateIndexName specified, Sitecore Content Search is able to resolve Solr mappings correctly for Sitecore fields that are used in queries to Solr. Failure to index templates can result in failing search requests or empty search results.