Skip to main content

Integrate through cxConnect

Integrate through cxConnect

CEP session agent schema

The CEP session agent uses a schema to deliver messages to the receiving systems for FILE, TCP (Delimited and XML format), and Coral8 output.

The receiving systems require the data to be organized in a format that can be consumed easily. The raw request text data is in name/value format and needs to be converted into a record format.

Schema output considerations

The schema for delivering messages for TCP (SWA - SAS Web Analytics format) output is different. The schema has only Page level information and no Events. For example: 
IP
<blank>
<blank>
PageTimeStamp
RequestLine
StatusCode
ResponseType
Referrer
UserAgent
Cookies
<blank>
ServerIP
<blank>
RequestMethod
ServerPort
ConnectionID
<blank>
QueryString
WSGenerationTime
<blank>
TLTSID
TLTUID
RequestSize

Finally, HTTP output uses the configurable schema in the CEPCustomFields.cfg file under the [HTTPFields] section.

The following table presents the schema is used by the CEP session agent to deliver messages to receiving systems. There are 33 fixed values and 100 custom fields.

Table 1. Default Event Bus data schema

 
POS Name Tealeaf REQ Field Type Length Description

1

 Canister Canister

String

64

Name of server for the session

2

 TLTSID TLTSID

String

32

Sessionization string used by Tealeaf (typically a 32-character GUID)

3

 TLTHID TLTHID

String

32

Hit ID (32-character GUID)

4

 TLTUID TLTUID

String

32

User ID (usually a cookie)

5

 SessionID SesnIdx

Long

  

Number assigned to session for the in memory database (64-bit integer)

6

 HitNumber HitNumber

Long

  

Page number in the session as observed by Tealeaf

7

 EventID TLEventID

Integer

 

Event ID

8

 FactID TLFID

Integer

 

Fact ID

9

 ReportGroupID TLDimGroupID

Integer

 

Report Group ID (Dimension Group ID)

10

 FactValue TLFactValue

String

256

Fact Value (text found)

11

 FactNumericValue TLFactNumericValue

Num

 

Numeric value of fact value (if applicable, 0.0 default value)

12

 DimValue1 TLDim1

String

256

Dimension value 1 text (TLT$NULL if empty)

13

 DimValue2 TLDim2

String

256

Dimension value 2 text (TLT$NULL if empty)

14

 DimValue3 TLDim3

String

256

Dimension value 4 text (TLT$NULL if empty)

15

 DimValue4 TLDim4

String

256

Dimension value 4 text (TLT$NULL if empty)

16

 PageTimeTicks RequestTimeEx

Long

 

Time of page in milliseconds in UNIX time (GMT, number of milliseconds since 1970)

17

 IP REMOTE_ADDR

String

16

IP address for the session

18

 IPNumber

Long

 

IP address (REMOTE_ADDR ) that is translated and converted to an integer (for easy lookup against GEO IP database).

For example, if the IP address is w.x.y.z, then the IP Number is translated as: 16777216*w + 65536*x + 256*y + z.

19

 URL URL

String

256

URL for page excluding the query string

20

 TLT_URL TLT_URL

String

256

Normalized Tealeaf URL for reporting

21

 QueryString HTTP_QUERY

String

1024

The query string for the URL

22

 Referrer HTTP_REFERER

String

1024

Referrer for the page

23

 UserAgent HTTP_USER_AGENT

String

256

Browser for the page

24

 RequestMethod REQUEST_METHOD

String

16

Request Method (GET/POST)

25

 StatusCode StatusCode

Integer

 

HTTP Status Code for page

26

 ReqCancelled ReqCancelled

Boolean

 

Request Cancelled flag from client or server (True/False)

27

 ReqDiscarded ReqDiscarded

Boolean

 

Request Discarded flag from client or server (True/False)

28

 RequestSize RequestSize

Long

 

Size in bytes of the HTTP Request

29

 ResponseSize ResponseSize

Long

 

Size in bytes of the HTTP Response

30

 WSGenerationTime WS_Generation

Long

 

Generation time for the page on the server in microseconds

31

 NTTotalTime NT_Total

Long

 

Network time for the page in microseconds (last ACK by client)

32

 RTTotalTime RT_Total

Long

 

Total time for page in microseconds

33

 ConnSpeed ConnSpeed

Long

 

Connection speed in bits/second

34/53

 CustomField0 Custom Field String 256

The custom fields can be set using CEPCustomFields.cfg.

Event IDs are listed in the tooltip for an event in the Event Manager.

Fact data, including dimension values, is available only through the request, which can be viewed through BBR or RTV.

Custom fields

Custom fields are populated with the configuration in the CEPCustomFields.cfg file, which is located in the Tealeaf installation directory.

Note: The CEPCustomFields.cfg file should be installed in the install directory on the server where cxConnect for Data Analysis is installed.
The CEP session agent looks for changes in CEPCustomFields.cfg  and automatically updates the rules for processing the events without requiring a restart of the pipeline.
Note: The changes may take up to one minute to update in the CEP session agent.

There are 100 custom fields in each record. The custom fields can be populated with any value from the request.

The custom fields cannot be populated with data from the response during the event processing. If you require data from the response, you must create custom request variables in the main pipeline with the RTA or PrivacyEx session agent(s).

The CEPCustomFields.cfg file is used for the following configuration:
  • Event Include
  • Event Omit
  • Default Event Custom Fields
  • Specific Event Custom Fields
  • HTTP Message Configuration

The following table provides a description of each section.

Table 2. Sections in CEPCustomFields.cfg 

 
Configuration file section Description
[EventInclude]

Used to only send events with the correct EventID (UniqueID in the CEP section). This value overrides any event IDs set in the [EventOmit] section.
[EventOmit]

Used to remove events from the event bus data stream. Any event in the EventID list is not sent to the downstream system.
[DefaultEvent]

Every event bus hit that does not have a specific event entry in the configuration file inherits the custom field settings and status code filters specified in this section.
[EventX] 

(where X is a sequential number)

Each section contains specific custom field configuration for a specific event ID, which is helpful when sending data in form fields only present under certain conditions (such as new account sign-up form).
[DefaultPage]

Every hit that does not have a specific page entry in the configuration file inherits the custom field settings and status code filters specified in this section.
[PageX]

(where X s a sequential number)

Each section contains specific custom field configuration for a specific page identifier, as indicated by the reqField and Filter values.

Sample CEP output

The following is a sample of the CEP file output.

In order to fit the sample on the page, it has been broken into sections.
Note: The actual CEP output includes data only (it does not include headers). The headings are included in the samples for illustrative purposes only.
Figure 1. Sample data for Canister, TLTSID, and TLTHID output fields
Canister         TLTSID                            TLTHID
ADMINIB-0MV8KCT  2A8F2E159C798E2868179C711F49BEC2  C5DC226E31C41031006BED4DEA3ED43
ADMINIB-0MV8KCT  2A8F2E159C798E2868179C711F49BEC2  C5DC226E31C41031006BED4DEA3ED43
ADMINIB-0MV8KCT  2A8F2E159C798E2868179C711F49BEC2  C5DC226E31C41031006BED4DEA3ED43
ADMINIB-0MV8KCT  2A8F2E159C798E2868179C711F49BEC2  C5DC226E31C41031006BED4DEA3ED43
ADMINIB-0MV8KCT  2A8F2E159C798E2868179C711F49BEC2  C5DC226E31C41031006BED4DEA3ED43
Figure 2. Sample data for TLTUID, SessionId, HitNumber, EventID, FactID, ReportGroupID, and FactValue output fields
TLTUID   SessionId   HitNumber   EventID   FactID   ReportGroupID   FactValue
               213          48         0      380               0   § %5Cn   CompXYZ.co.uk Sign In%5Cn
               213          48         0      415               0   § %5Cn   CompXYZ.co.uk Sign In%5Cn
               213          48         0      416               0   § %5Cn   CompXYZ.co.uk Sign In%5Cn
               213          48         0      552               0   Hans-Jⁿrgen
               213          48         0      580               0   /ap/signin
Figure 3. Sample data for FactNumericValue, DimValue1, DimValue2, DimValue3, DimValue4, and PageTimeStamp output fields
FactNumericValue   DimValue1    DimValue2    DimValue3    DimValue4    PageTimeStamp
               0   TLT$NULL     TLT$NULL     TLT$NULL     TLT$NULL     1410898825023
               0   BROWSER      TLT$NULL     TLT$NULL     TLT$NULL     1410898825023
               0   BROWSER      TLT$NULL     TLT$NULL     TLT$NULL     1410898825023
               0   TLT$NULL     TLT$NULL     TLT$NULL     TLT$NULL     1410898825023
               0   TLT$NULL     TLT$NULL     TLT$NULL     TLT$NULL     1410898825023
Figure 4. Sample data for IP, IPNumber, URL,and TLT_URL output fields
IP               IPNumber      URL          TLT_URL
192.168.76.102   3232255078    /ap/signin   /ap/signin
192.168.76.102   3232255078    /ap/signin   /ap/signin
192.168.76.102   3232255078    /ap/signin   /ap/signin
192.168.76.102   3232255078    /ap/signin   /ap/signin
192.168.76.102   3232255078    /ap/signin   /ap/signin
Figure 5. Sample data for QueryString output field
Note: Because of its length, the sample includes forced line-breaks.
QueryString
 _encoding=XXXX&openid.assoc_handle=XXXXXXXXXXXXXXXXXX&openid.claimed_
id=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.identity=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.mode=XXXXXXXXXXXXX&openid.ns=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.ns.pape=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.pape.max_auth_age=X&openid.return_
to=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&pageId=XXXXXXXXXXXXXXXXXX
&showRmrMe=X&siteState=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

 _encoding=XXXX&openid.assoc_handle=XXXXXXXXXXXXXXXXXX&openid.claimed_
id=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.identity=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.mode=XXXXXXXXXXXXX&openid.ns=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.ns.pape=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.pape.max_auth_age=X&openid.return_
to=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&pageId=XXXXXXXXXXXXXXXXXX
&showRmrMe=X&siteState=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

 _encoding=XXXX&openid.assoc_handle=XXXXXXXXXXXXXXXXXX&openid.claimed_
id=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.identity=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.mode=XXXXXXXXXXXXX&openid.ns=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.ns.pape=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.pape.max_auth_age=X&openid.return_
to=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&pageId=XXXXXXXXXXXXXXXXXX
&showRmrMe=X&siteState=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

 _encoding=XXXX&openid.assoc_handle=XXXXXXXXXXXXXXXXXX&openid.claimed_
id=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.identity=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.mode=XXXXXXXXXXXXX&openid.ns=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.ns.pape=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.pape.max_auth_age=X&openid.return_
to=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&pageId=XXXXXXXXXXXXXXXXXX
&showRmrMe=X&siteState=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

 _encoding=XXXX&openid.assoc_handle=XXXXXXXXXXXXXXXXXX&openid.claimed_
id=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.identity=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.mode=XXXXXXXXXXXXX&openid.ns=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.ns.pape=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&
openid.pape.max_auth_age=X&openid.return_
to=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX&pageId=XXXXXXXXXXXXXXXXXX
&showRmrMe=X&siteState=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Figure 6. Sample data for Referrer output field
Referrer
http://www.CompXYZ.co.uk/gp/huc/view.html?ie=XXXX&newItems=XXXXXXXXXXXXXXXXXX
http://www.CompXYZ.co.uk/gp/huc/view.html?ie=XXXX&newItems=XXXXXXXXXXXXXXXXXX
http://www.CompXYZ.co.uk/gp/huc/view.html?ie=XXXX&newItems=XXXXXXXXXXXXXXXXXX
http://www.CompXYZ.co.uk/gp/huc/view.html?ie=XXXX&newItems=XXXXXXXXXXXXXXXXXX
http://www.CompXYZ.co.uk/gp/huc/view.html?ie=XXXX&newItems=XXXXXXXXXXXXXXXXXX
Figure 7. Sample data for UserAgent output field
UserAgent
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36
Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143 Safari/537.36
Figure 8. Sample data for RequestMethod, StatusCode, ReqCancelled, ReqDiscarded, RequestSize, and ResponseSize output fields
RequestMethod    StatusCode    ReqCancelled    ReqDiscarded    RequestSize    ResponseSize
GET                    200     false           false                  1710           30885
GET                    200     false           false                  1710           30885
GET                    200     false           false                  1710           30885
GET                    200     false           false                  1710           30885
GET                    200     false           false                  1710           30885
Figure 9. Sample data for WSGenerationTime, NTTotalTime, RTTotalTime, ConnSpeed, and CustomField0 output fields
WSGenerationTime       NTTotalTime      RTTotalTime      ConnSpeed 
          177873               260           178133       331323076
          177873               260           178133       331323076
          177873               260           178133       331323076
          177873               260           178133       331323076
          177873               260           178133       331323076
Figure 10. Sample data for CustomField1,CustomField2, and CustomField3 output fields
CustomField0                     CustomField1    CustomField2    CustomField3
A8F2E159C798E2868179C711F49BEC2                  Win7            BROWSER
A8F2E159C798E2868179C711F49BEC2                  Win7            BROWSER
A8F2E159C798E2868179C711F49BEC2                  Win7            BROWSER
A8F2E159C798E2868179C711F49BEC2                  Win7            BROWSER
A8F2E159C798E2868179C711F49BEC2                  Win7            BROWSER
community icon

Join the community

Ask questions, share best practices, and connect with fellow marketers.

Ask a question

Academy

Read how-to articles, guides, and more to help you on the path to success.

Learn more