Passive Capture consists of software that is running on a host, which directly connects to the collection device, a network tap, or switch spanning port. The data flow from the collection device to the host workstation is unidirectional; the host only receives data passively.
From the host, the Passive Capture software transports the data in real time to the CX Server environment. Data can be transported over TCP/IP or through a network crossover cable that is connected directly between the Passive Capture host and the receiver workstation in the CX environment. Passive Capture performs the following functions:
- Reconstruct the HTTP(S) request and response bodies from the captured TCP/IP packet data
- Decrypt SSL (if applicable)
- (optional) Sessionize (or sequence) the HTTP request and response pages by a session ID into visitor sessions
- (optional) Privacy blocking can be defined for sensitive data
- Transport the data to the CX Server environment
On-premises deployment
The on-premises deployment architecture represents a common Tealeaf environment that is deployed within your local network infrastructure. In this scenario, the CX Passive Capture Application can be hosted on a physical server or it can be hosted from a compatible virtual server within the same network environment.
The capture device must have access to all traffic sent to the load balancing router or a network segment that is containing the group of application/web servers that are supported by the Tealeaf CX solution.
Because the Tealeaf Passive Capture host is connected directly to the collection device, opening firewall ports is not required to collect data.
The following diagrams illustrate typical deployment architectures for switch spanning or network tap methods. From the Passive Capture host, data is transported (through TCP/IP or SSL) to the CX Server environment where it is analyzed, aggregated, and archived.
Deployment in the Cloud
The cloud deployment architecture represents a common Tealeaf environment that is deployed within a supported cloud-based infrastructure.
Tealeaf can be deployed to one of the following cloud-based infrastructures:
- IBM® SoftLayer®
- Amazon Web Services (AWS)
- Microsoft™ Azure
The following diagram illustrates the deployment architecture for cloud-based installations using a reverse proxy server that also has a packet forwarder installed on it. In this deployment, the packet forwarder captures web traffic from the virtual network to the reverse proxy server and sends the captured data to the PCA which is hosted on a separate virtual machine.
The following diagram illustrates the deployment architecture for cloud-based installations where the packet forwarder is deployed on the web server. In this scenario, each web server instance has a packet forwarder instance that is deployed to the web server. Each packet forwarder instance captures the web traffic between the web server and the client. The packet forwarder sends the captured web traffic to the packet forwarder which is hosted on a separate virtual machine.
PCA Throughput
The following components can affect how Tealeaf CX Passive Capture Application processes hit data that is forwarded to the CX PCA.
Note: Review the recommended CX PCA requirements to optimize the performance of the CX PCA.
| Component | Affect on performance |
|---|---|
| Network interface cards (NICs) |
The network interface card represents the upper limit of what a specific instance of the CX PCA server can capture and process. For example, using NICs that are only capable of 100 megabits per second, limits the maximum throughput for a CX PCA server. If 1 gigabit per second NICs are used, you can achieve up to 10 times more throughput. |
| CPU cores | The CX PCA benefits when installed on a server with eight or more CPUs. With extra available cores, you can install more instances of the Tealeaf CX Passive Capture Application. |
| RAM | More RAM on the CX PCA server enables more resources for processing of captured data. |
| SSL | Secure traffic is CPU-intensive and can have a large impact on overall throughput. For example, if a CX PCA can handle 700 megabits per second of non-SSL traffic throughput, processing the same traffic over SSL might result in achieving only 70 megabits per second throughput. |
| Virtual environments |
Your VMware virtual machine settings must be configured to meet the same operating system and hardware requirements as a physical server that is hosting Tealeaf CX PCA. If the virtual machine does not meet the same requirements as a physical server, you might experience performance-related issues. Limit throughput to no more than 500 Mbps. The CX Passive Capture Application supports throughput for up to 500 Mbps. Environments with throughput rates greater than 500 Mbps can experience packet loss at the CX Passive Capture Application. |
Cloud Packet Capture Overview
The Cloud Packet Capture is used to capture and forward hits to a cloud-based CX PCA that is operating on a virtual machine.
The CX PCA processes the hits that are forwarded by the Cloud Packet Capture.
The Cloud Packet Capture software is included with the CX PCA and consists of a transmitter and a receiver component. The transmitter captures TCP packets and forwards them to the designated receiver. The receiver can be configured to capture data that is submitted from a specified transmitter. You can configure multiple transmitter instances to send data to a centralized CX PCA. Each transmitter instance must connect to an individual receiver instance on the CX PCA.
Note: Transmitter instances and a receiver instances cannot share the same listening port.
The Cloud Packet Capture provides the following functionality:
- Replaces the default TCP packet sniffer component of the PCA with a network socket listener
- Directs traffic into an internal PCA instance. The transmitter points to a centralized PCA instance in the cloud and delivers packets to the receiver through a network connection.
- TCP packets are captured by sniffing the designated port.
The Cloud Packet Capture components can be deployed in a public or private cloud to manage capture and forwarding of TCP packets for processing by a cloud-based Tealeaf installation.
Software architecture
The CX Passive Capture Application uses the following services to perform the capture process.
The core capture processes captures, reassembles, post-processes, and delivers the reassembled HTTP/HTTPS hits to the Tealeaf Transport Service, which is hosted on another server. The five core processes in order of processing during capture are named captured, listend, reassd, pipelined, and deliverd.
| Process | Description |
|---|---|
Captured |
|
Listend |
Captures network traffic packets from the configured primary and secondary interfaces and send them to the reassembly process, |
Reassd |
Reassembles TCP packets, decrypt SSL traffic, and initially parse the resulting HTTP requests and responses. |
Pipelined |
Retrieves the reassembled HTTP request and response from Note: The CX PCA supports the creation of multiple instances of the |
Routerd |
Transparently load balances (TLB) incoming network packets and connections to the multiple Reassd process instances. By distributing network traffic more evenly across all Reassd instances, it increases the efficiency of the system's CPU cores to improve overall performance. This process module is present only if TLB mode is enabled. |
Tcld |
Provides TCL-based script processing to handle the management of the hits for specialized delivery with the deliverd process. This process can accept hits from one or more pipelined source processes. |
Deliverd |
Delivers the formatted hits to one or more Transport Services on remote workstations as instructed by tcld. Tcld is responsible for deciding whether a hit must be sent and to whom it must be sent. Establishes the network connection and sending the hits over the network to the Transport Service. It can optionally communicate with the Transport Service using an SSL connection to provide a secure channel. |
Failoverd |
This optional process is present if failover is enabled and running on an instance of the CX Passive Capture Application.
|
Memcached |
The Memcached process provides a global in-memory caching system to the CX PCA. Memcached is primarily used to store SSL session information for later access by all Reassd instances in processing SSL decryption (resumed SSL sessions). This process module is present only if TLB mode is enabled. |
Multiple Instances
In both TLB mode and non-TLB mode, the CX PCA can be configured to initiate multiple instances of listend and reassd processes to use multiple CPU cores to handle high capture traffic loads.
The instances can be configured to capture different TCP/IP addresses and ports to distribute the traffic load among the capture instances. The instances can share NICs for capturing packets or can capture packets by using multiple NICs available on the Tealeaf CX Passive Capture Application server.
The CX PCA can also create multiple instances of pipelined process to distribute its processing load requirements.
In TLB mode, a single instance of listend is used to feed multiple reassd processes through the routerd process. Multiple instances are provided through the reassd processes where the effective work is needed and eliminates the manual workload of segmenting and distributing the capture traffic load.
Multi-Instance Pipeline Processes
The pipelined process runs multiple CPU-intensive operations, such as privacy blocking activities, which can cause performance bottlenecks in single-threaded configurations.
You can create additional instances of the pipelined process to distribute the processing load for all PCA instances across available CPU resources.
For example, suppose that a single PCA instance is generating 500 pageviews/second and is configured for intensive pipeline privacy processing, which is limiting its throughput to 200 page views per second. Adding two more pipeline instances (for a total of three pipelines) enables the handling of the overall page-view throughput.
One or more reassd processes (multi-PCA instances) can feed its resulting HTTP hits to a single, shared memory (SHM) queue, which manages distribution to the available instances of the pipelined processes in round-robin fashion.
In an example of multi-PCA instances, suppose that you have created four PCA instances, which are generating 1000 pageviews/second. If a single pipeline can process 400 page views per second in your environment, two more pipelines can be added to manage processing the entire volume.
PCA master and slave failover also supports multi-instance pipelines.
CX PCA Transparent Load Balancing Overview
The CX PCA can be configured for transparent load balancing (TLB) which provides the ability to transparently segment and distribute network capture traffic.
New CX PCA installations come with TLB enabled by default. Prior to this feature (non-TLB mode), segmentation of the capture traffic required assigning blocks of traffic to specific CX PCA instances for load balanced processing.
By configuring the CX PCA for transparent load balancing, you can:
- Reduce customer support issues that are caused by uneven traffic loads or changes to the traffic profile across multiple CX PCA instances where a sudden increase in network traffic can overload a CX PCA instance. If a CX PCA instance is overloaded, it can cause the instance to restart and lose of captured traffic. By enabling transparent load balancing, network traffic is distributed to the CX PCA instances by using a round robin method of distribution. Distributing the network traffic to the CX PCA instances prevents an instance from overloading.
- Simplify your CX PCA installation and configuration. By enabling transparent load balancing, you do not need to provide extra configuration for each additional CX PCA instance. You can specify the number of instances that you want to use and Tealeaf automatically distributes network traffic to the instances.
- Capture traffic from a single virtual IP (VIP) for web servers that are configured to work under a single VIP.
Multiple Listend-Routerd Pairs
You can enable multiple listend instances using multiple Listend-Routerd pairs (MLRP).
MLRP provides the capability of using multiple NICs to capture data in a load-balanced environment. You can configure your CX PCA to use MLRP with multiple NICs to improve packet capturing performance and increase scalability to meet the demands of your network traffic. MLRP takes advantage of multiple NICs by creating multiple instances of the routerd process.
Each routerd process requires one CPU core to operate. Each routerd process actively routes the flow of incoming network traffic from a listend process to the reassd processes. Each listend process is paired to one routerd process and can process 1 Gigabit per second of traffic.
The ability to route the network traffic to multiple reassd processes removes the need to manually segment and distribute the captured traffic to prevent queuing. Balancing the network traffic to multiple reassd processors enables the CX PCA to take advantage of using multiple NICs to receive and process large amounts of network traffic.