Data Files
The Cortex Processing Worker extracts data in batches and stores each batch in a separate data file.
The batch size is a configurable setting that is passed via worker options.
You can customize the following data file attributes:
-
File name
-
Data format
File name
The file name generator assigns the file names to the data files. The file name generator is injected into the ExportContactsDistributedWorker and ExportInteractionsDistributedWorker workers. By default it combines the data export ID value, unique batch ID, and file extension to form the name.
You can customize the file name generator. The custom file name generator must inherit the IFileNameGenerator interface and you must specify it in the sc.XConnect.DataTools.Workers.xml configuration file.
Data format
The interaction/contact exporter defines the data format and batch serialization logic.
By default, all the extracted batches are serialized into the json format with the help of JsonEntitiesExporter.
You can customize the exporter. The custom exporter must inherit the IEntitiesExporter interface and you must specify it in the sc.XConnect.DataTools.Workers.xml configuration file.
By default, JsonEntitiesExporter does not serialize values that are the same as the member’s default value, for example null for objects and nullable types, 0 for decimal and floating point numbers, and false for booleans. You can change this behavior with the Sitecore\Processing\Services\StorageProviders.IEntitiesExporter\Options\IgnoreDefaultValues configuration setting.
For example:
<StorageProviders.IEntitiesExporter>
<Type>Sitecore.XConnect.DataTools.CortexWorkers.DataExport.JsonEntitiesExporter, Sitecore.XConnect.DataTools.CortexWorkers</Type>
<As>Sitecore.XConnect.DataTools.CortexWorkers.DataExport.IEntitiesExporter, Sitecore.XConnect.DataTools.CortexWorkers</As>
<LifeTime>Singleton</LifeTime>
<Options>
<!--
Defines whether members, where the member value is the same as the member's default value, should be serialized.
If set to true the exporter will ignore the default values (e.g. null for objects and nullable types;
0 for decimals and floating point numbers; and false for booleans).
Default value: true
-->
<IgnoreDefaultValues>true</IgnoreDefaultValues>
</Options>
</StorageProviders.IEntitiesExporter>