Robot detection overview
Overview of the Sitecore robot detection functionality, which filters out unwanted interactions from automated browsers and robots. This ensures that only genuine contacts are registered in the xDB.
Sitecore robot detection functionality filters out unwanted interactions from automated browsers and robots. This ensures that only genuine contacts are registered in the xDB. Robot detection classifies a contact based on an IP address, user agent, or verified human behavior.
There are two main ways to identify robots:
Use the request IP address or user agent.
Use the robot detection functionality to check for human behavior.
The Device Detection component detects user agents including well-known bots, crawlers, spiders, and robots. If a contact visits your website with a user agent that the Device Detection component recognizes as a robot, crawler, or spider, then the request to view the page is not tracked.
TheSitecore.Analytics.ExcludeRobots.config file contains an exclude list of IP addresses and custom user agents to filter unwanted contacts. If a contact visits your website from one of the excluded IP addresses or user agents on the exclude list, then the request to view the page is not tracked. You can manually add IP addresses and custom user agents to this list.
The Sitecore robot detection component is enabled by default. It classifies all visitors on their first visit as robots, using classification code 925.
It next performs a series of tests to determine whether a visitor is a human or a robot. For example, the VisitorIdentification.js JavaScript file tests if a visitor is human by detecting computer mouse movements, or touch screen actions on a mobile phone or tablet, that the visitor performs. Human contacts are classified using classification code 0.
Important
Ensure that you include the robot detection component in your main layout so that it renders with every page view.
The Sitecore.Analytics.Core.ContactClassification class contains helper methods that allow the robot detection component to check the classification of a contact:
IsHuman()IsRobot()IsAutoDetectedRobot()
The Sitecore robot detection component consists of a pipeline processor, an event handler, a JavaScript file, and several robot detection code classes.
The Sitecore.Analytics.Tracking.RobotDetection.config file contains several configuration settings in addition to the following pipeline and event handler:
Sitecore.Analytics.RobotDetection.Pipeline.InitializeTracker.RobotsSitecore.Analytics.RobotDetection.Media.MediaRequestEventHandler
These components are triggered when a contact or robot ibegins to nteract with your website, initiating the process that identifies and prevents robots from further accessing your website.