The Data Collector Service manages the collection, aggregation, and trimming of the base reporting data, as well as the calculation of any predefined reports. It is enabled and disabled through TMS. Data collection and aggregation processes run independently of each other.

Note: If the Data Collector Service is disabled, Portal reports are not updated.

Data Collector Service Monitor

The Data Collector Service Monitor is a component of the Tealeaf Data Collector Service. Its primary purpose is to readily provide Data Collector Service processing, alert, and system information in an easily accessible and consolidated format.

The goal for using the Data Collector Service Monitor is to reduce the time between when an error condition or latency state begins and when the Tealeaf administrator or users are made aware of the error. With the Data Collector Service Monitor, the Service Monitor has already begun collecting the information that is necessary for problem diagnosis and resolution.

Data Collector Service Monitor log file processing and configuration

The Data Collector Service Monitor generates a log file that is named DataCollectorServiceMonitor_YYYYMMDD.log to the standard Tealeaf logs directory.

The warning and error level alerts captured (for all Data Collector Service Components) and written to the log file, are configurable. The warning and error level alerts captured by the Service Monitor can be configured to be emailed to a Tealeaf administrator or a Data Collector Service administrator, or both. You can also configure the alerts to display in the Announcement section upon Portal login.

The Data Collector Service Monitor raises the Data Collector Logging Level to ALL when the threshold level of errors or warnings is exceeded within the configured time frame. That is, the logging level of every Data Collector Service component (for example, Fact Aggregation, Data Manager).

The errors and warnings are written to the log file in accordance to the frequency configured (a default of every 30 minutes). Additionally, the following system and processing information is written to the log file every 4 hours.

• Registry settings relevant to the Data Collector Service
• Contents of tl_system database TM_SERVER table
• Contents of tl_system database TM_SERVICE_MAP table
• Contents of tl_system database TM_DATABASE table
• Contents of tl_system database TM_LINKED_SERVER table
• Contents of tl_system database RS_CONTROL table (data collector and data retention control groups)
• Contents of tl_r ports database V_RS_DTS view
• Contents of tl_reports database V_RS_INDEX view
• A summary of the contents of tl_reports database TM_DATA_COLLECTION_SET table

Dimension logs

For any dimension that is flagged for logging, the Canister gathers all observed values for the dimension from the capture stream. Those values are written to a database table once per hour.

By default, the Data Collector polls the Canister ten minutes past the hour to gather new values from the Canister dimension log and inserts newly detected ones into the database. These values can then be used in specifying whitelists, blacklists, and group lists for dimensions that reference them.

The interval and time when the Data Collector values are aggregated can be configured as needed. The following configuration options are available:

• Enable/disable aggregation of values

  You should not need to disable aggregation of values, unless you are attempting to troubleshoot an issue with the Canister or Data Collector.

• Interval of Data Collector aggregation (Dimension Log Aggregation Interval):
  1. Hourly - When log values are aggregated on an hourly basis, the Data Collector process runs at ten minutes after the hour (1:10, 2:10, etc.).
  2. Daily or Weekly - When log values are aggregated on a daily basis, you may configure the time of day when the log values are collected (Dimension Log Aggregation Time).

     Note: When dimension log aggregation is scheduled for a daily or weekly basis, the number of log values that must be collected and aggregated may impact system performance. Tealeaf recommends setting this value to an off-peak hour. By default, it is configured to execute at 3:00 in the morning.

Note: The logging of dimension values must be enabled for each dimension that you create. After these values have been inserted into the database, they can be added to the whitelists, blacklists, or group lists for the referencing dimension through the Tealeaf Event Manager.

The following settings are used to configure the Data Collector settings for dimension log values.