Use the information in this section to troubleshoot problems with Experience Analytics processing.
Canister Services continue to restart
If you disabled Canister Services through the Windows™ Services Control Panel and the service continues to restart, you should verify that the network monitoring and infrastructure diagnostic tools such as Big Brother are not performing these restarts.
Canister spacer file does not exist
The Windows Application Event Log may contain a warning message from the TeaLeaf Pipeline indicating the Canister spacer file does not exist.
SADecoupleEx (TeaLeafCSS_1966): Canister spacer file does not exist (C:\Program Files (x86)\TeaLeaf\Canister\Canister.spacer).
The Canister space file is used to reserve disk space for storage of sessions in the Canister. The above message indicates that this file does not exist. Without it, the Canister cannot save the processed sessions when disk space is low.
The Extended Decoupler session agent normally does not interact with this spacer file. However, when a low disk space condition is detected, the session agent deletes it and frees up disk space. When the message appears, DecoupleEx was unable to find the file.
The spacer file is created when the TLTMaint utility is run. To correct this issue, you can run this utility to create the spacer file.
The causes of session fragmentation
Experience Analytics sessions can become fragmented. Fragmented sessions can be caused by the following conditions:
- A visitor's session is fragmented when the Experience Analytics inactivity timeout period is exceeded. For example, if a visitor leaves a web page open during lunchtime, browsing again after lunch in the same browser window may cause a fragmented session. Because of the long period of inactivity, the first few pages are moved out of the Short-Term Canister (STC) into the Long-Term Canister (LTC). The next pages after lunch appear in the STC and are given a unique session fragment ID. However, since the new session is the same browser window, the TLTSID session cookie value does not change.
- A visitor session that exceeds any of the three 'safety' limits (time duration, number of hits, or bytes) is moved out of STC into LTC to prevent unnecessary consumption of STC space. After the session is moved to LTC, the next hit with the same TLTSID generates a new session fragment in the STC.
- A session opened anytime the Canister services are shut down for maintenance is moved to the LTC. Subsequent hits with the same TLTSID create a new session fragment in the STC.
Index error recovery
When an index job creates a new index or performs a merge, a new index is created and added to
TLL file immediately to ensure that the index name is reserved, but the index
is marked as not valid. After the index or merge operation is complete, the index is marked as
If the index or merge crashes, then the index remains marked as not valid and remains locked. A
lock is broken if it is in place for more than two hours. If during the
Index Check finds an index with no lock that is marked as not valid, the index is deleted.
If an error is returned during indexing, the index is marked as requiring an index check. This
index is not used for indexing until the flag is removed. If the index check fails, the index is
marked as corrupt and removed from the
TLL. If the check succeeds, then the check
required flag is cleared and the index is returned to service.
Corrupted index directories are renamed with a
.CORRUPT file extension for later
Rebuilding base canister files so that Experience Analytics services can connect to canister as TLUSER or ADMIN
If the TLUSER and/or ADMIN userids are not present in the Canister datastore, the other Experience Analytics services are prevented from connecting to the Canister. In such a scenario you must rebuild the base Canister files, saving the existing Canister session and search index data.
The following steps rebuild the base Canister files, saving the existing Canister session and search index data:
- Stop all Experience Analytics services.
CanRebuild.exe, choosing to rebuild only the full Canister and nothing else.
Indexes.olddirectory back to
- Move (do not copy) the following files from
CANISTER.dbs.oldto the new
LSSN * PEVT * NDLY.dat LDLY.dat PATH.dat SRVR.dat EVNT.dat SEVT.dat
- Start all Experience Analytics services.
Changing the logging level to troubleshoot exceptions
When an exception is detected in the Indexer log, raise the logging level for the indexer to 4 or 5, which enables the capture of a wider set of debugging data.
When reporting an error to Experience Analytics, submit a log file that is set to level 4 or higher.
Complete the following steps to change the logging level.
- Log in to the Portal as an administrator.
- From the Portal menu, navigate to TMS.
- From the View drop-down, select
- Click the Session Indexer node.
- Click Index Service configuration. In the Config Actions panel, click View/Edit.
- In Index configuration, click the Scheduling/Diagnostic tab.
- In the Diagnostics group, increase the level to 4 or 5.
- Click Save.
- Push the configuration to each server. A service restart should not be performed currently.
Using "Check and Fix" feature to add missing sessions to the index
If the Session Indexer service is in the stopped state, sessions are not indexed when they move to the Long-Term Canister. If you suspect some sessions were not indexed or that the Indexes may not be up to date, you can use the Search Server "Check and Fix Indexes" feature.
<install_directory>\IndexCheck.exewas upgraded or patched.
To run the Check and Fix feature:
- In a web browser, enter the following URL: http://<hostname or IP address of server>:19000/.
- Click the Canister/Indexer Check link. The table on the resulting page shows how many sessions exist in each day's data and how many of those sessions were indexed.
- Click Check and Fix.
The system indicates the number of sessions that are needed to be indexed and begins the process. The process continues in the background even if you close the browser window.
- To check the status of the process after the browser window is closed, open a new browser window and enter the following URL: http://<hostname/IP addr of server>:19000/CanisterIndexCheckerStatus .