Managing the size of your reporting data

Version: 10.4

During data aggregation, Experience Analytics collects facts about each group of each dimension for all the interactions on your websites. These collected facts are stored in fact tables in the reporting database for tracking purposes. The volume of reporting data can become very large depending on the number of visitors to the website. To reduce data volume, the reducer combines the least significant groups that it collects each day into a single group called other. This makes data consumption and read performance more predictable.

Reducer configuration

You configure the reducer using the following configurable parameters in the <webroot>/App_Config/Include/Examples/Sitecore.ExperienceAnalytics.Reduce.config.example file.

Note

The reducer is disabled by default. To activate it, you must copy and rename the configuration file to <webroot>/App_Config/Sitecore/ExperienceAnalytics/Sitecore.ExperienceAnalytics.Reduce.config.

Name	Description	Default Value
`DefaultKeepCountThreshold`	The number of records to keep. For example, if the default of 1,000 is used and there are 3,000 records, the reducer consolidates 2,000 records and leaves 1,000 intact. If the number of records for a single dimension is fewer than this threshold, the reducer does not reduce anything.	`1000`
`DefaultValueThreshold`	The minimum value that the `Value` metric must meet for a record to be considered significant. The reducer consolidates records that do not meet the minimum value.	`-1 (Disabled)`
`DefaultVisitThreshold`	The minimum value that the `Visits` metric must have for a segment record to be considered significant. The reducer consolidates records that do not meet this minimum value.	`10`
`DefaultRetentionPeriod`	The amount of time (`days:hours:minutes:seconds`) that data is kept before the reducer consolidates it.	`90.00:00:00`
`DefaultOrderBy`	The order in which the reducer sorts records to determine relative significance. You can specify any metric that you want to use as an indicator of significance, for example, `Visits`, `Value`, `timeOnSite`, and so on. Note This parameter is only applicable if you are using flexible dimensions.	`Visits, abs(Value)`
`Timeout`	The maximum amount of time the reducer works. After this time, the reducer stops.	`01:00:00`

For example, if 10,000 unique pages are visited in a single day, the reducer by default only keeps facts about the 1,000 most significant pages (DefaultKeepCountThreshold), and reduces the facts for the 9,000 less significant pages. The reducer processes the 10,000 visits after the retention period (DefaultRetentionPeriod), which is 90 days by default.

The reducer and legacy dimensions

If you are using legacy dimensions, the reducer works as follows:

If the visit count for a page is less than the DefaultVisitThreshold, the page is considered insignificant.
If the value for a page is less than the DefaultValueThreshold, the page is considered insignificant.
If you have more than 1,000 pages that are significant, the reducer orders them by Visits, and then by Value, and then consolidates the records for all pages except the top 1,000.

The reducer and flexible dimensions

If you are using flexible dimensions, the reducer works as follows:

The DefaultVisistThreshold and DefaultValueThreshold parameters are not considered when reducing data for flexible dimensions.
If you have more than 1,000 pages that are significant, the reducer orders them by DefaultOrderBy and then consolidates the records for all pages except the top 1,000.

Do you have some feedback for us?

If you have suggestions for improving this article,