The documentation set for this product strives to use bias-free language. For the purposes of this documentation set, bias-free is defined as language that does not imply discrimination based on age, disability, gender, racial identity, ethnic identity, sexual orientation, socioeconomic status, and intersectionality. Exceptions may be present in the documentation due to language that is hardcoded in the user interfaces of the product software, language used based on RFP documentation, or language that is used by a referenced third-party product. Learn more about how Cisco is using Inclusive Language.
This document describes the steps required to configure and troubleshoot Live Data issues in a Unified Contact Center Enterprise (UCCE) environment.
Cisco recommends that you have knowledge of these topics:
Cisco Unified Contact Center Enterprise (UCCE)
The information in this document is based on these software and hardware versions:
ICM Version: 12.6
Finesse Version: 12.6
CUIC/Live Data Version: 12.6
The information in this document was created from the devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If your network is live, ensure that you understand the potential impact of any command.
Live Data is a data framework that processes real-time events with high availability for Live Data reports. Live Data continuously processes agent and call events from the peripheral gateway and the router. As events occur, Live Data continuously pushes real-time updates to Unified Intelligence Center reporting clients. The PG and the Router push agent and call events to Live Data as the events occur. Live Data then continuously aggregates and processes the events in-stream and publishes the information. CUIC subscribes to the message stream to receive the events in real-time and continuously update Live Data reports. Individual state values, such as agent states, refresh as they happen. Other values, such as calls in queue, refresh approximately every 3 seconds.
In Unified CCE, Live Data resides on a Cisco Voice Operating System Virtual Machine (VM). You can embed Live Data reports in Finesse agent desktops.
For 2000 Agent deployments
For 4000 and higher Agent deployments
The active Live Data server must establish a TIP and TOS connections to Routers and all Agent PGs. The in-active (standby) Live Data server must establish only TOS connection to Routers and all Agent PGs.
Note:
The Live Data servers work in cold-active or standby mode. Only one Live Data server is active at any time. The other Live Data server is standby. The standby Live Data server constantly monitors the status of the active server. When the active server fails, the standby server takes over and becomes active. The failing server becomes the standby server when it is ready to serve.
Live Data uses the TIP transport protocol to communicate with the Router and PG servers. The active Live Data server establishes TIP connections to both sides of the Router and PGs. The standby Live Data server does not establish any TIP connections. Only one TIP connection is active at a time, either to Side A or to Side B. When the active TIP connection fails, the active Live Data server recovers to the idle TIP connection.
A SocketIO client connects to either side of the Live Data server to receive the Live Data report event stream (SocketIO stream). Unified Intelligence Center clients are an example of a SocketIO client. The standby Live Data server also produces the SocketIO stream by proxy from the active server. SocketIO client heartbeat losses result in a SocketIO connection failure. The SocketIO client then fails over to the other Live Data server.
Prior to deploying Live Data, perform these checks:
Check 1
From the ICM servers, verify the Forward and Reverse DNS Lookup for the Live Data Publisher and Subscriber, using the nslookup command.
nslookup <Live-Data-Server-FQDN>
nslookup <Live-Data-Server-IP>
Check 2
From the Live Data Server CLI, verify the Forward and Reverse DNS lookup for the Routers and PGs (perform checks for A side and B side).
utils network host <FQDN>
Replace <FQDN> with the public FQDNs of the Routers/PGs
utils network host <IP>
Replace <IP> with the public IP address of the Routers/PGs (perform checks for both A side and B side)
Check 3
Verify the NTP configurations and requirements for a CCE environment. Refer to the NTP and Time Synchronization section in the CCE Solution Design Guide.
Check 4
Ensure configuration limits are being followed as per the CCE Solution Design Guide
Note:
Check 5
Certificate exchange for Live Data in a UCCE deployment
Note: If CA-signed certificates are being used across the CCE deployment - ICM, Finesse, CUIC, LD and IdS - (signed by the same Certificate Authority), this check can be skipped.
When using self-signed certificates
For 2000 Agent Deployments
For 4000 and 12000 Agent Deployments
Note:
Check 6
You MUST use fully qualified domain name (FQDN) for all Live Data configuration commands. Using IP address when configuring Live Data can cause issues with the Live Data service.
Check 7
a. For all of the set Live Data commands, manually type in the passwords instead of a copy-and-paste action.
b. Supported Character Set for Live Data Installation CLI Commands
When working with the CLI (and not exclusively for Live Data), you can use plain alphanumeric characters [0-9] [A-Z] [a-z] and these additional characters:
Step 1
Ensure the correct deployment type is set on CCE Administration under Infrastructure > Deployment Settings (https://<AW-Server>/cceadmin).
Step 2
Add server to the CCE Inventory.
For 2000 Agent deployments, add the co-resident CUIC-LD-IdS cluster by selecting the CUIC-LD-IdS Publisher option.
For 4000 and higher Agent deployments, add the standalone CUIC cluster by selecting the Unified Intelligence Center Publisher option.
Note: For 4000 and higher agent deployments, the Live Data servers are added to Inventory using the set live-data machine-services command.
Step 3
Use the set live-data reporting-interval <reporting-interval-in-minutes> command to set the Live Data reporting interval in minutes.
Use the show live-data reporting-interval command to view the current reporting interval setting.
Step 4
Configure a SQL user on the AW DB to work with Live Data
On the primary and the secondary AW DB server > start SQL Server Management Studio (SSMS):
Step 5
Execute the SQL query for the SQL user configured to work with Live Data.
USE master
GO
GRANT CONTROL ON CERTIFICATE :: UCCESymmetricKeyCertificate TO "<user>"
GRANT VIEW DEFINITION ON SYMMETRIC KEY :: UCCESymmetricKey TO "<user>"
Replace <user> with the username of the user created in Step 4.
Step 6
Configure Live Data with AW
set live-data aw-access primary <aw1-server-fqdn> <port> <aw-database> <sql-user>
set live-data aw-access secondary <aw2-server-fqdn> <port> <aw-database> <sql-user>
Where
This command tells Live Data how to access the primary AW DB and the secondary AW DB. The command also automatically tests the connection from Live Data to the primary or secondary AW, checks to see if the configured user has appropriate AW DB access, and reports the results. (Test status must show 'Succeeded'). You do not need to configure the AW DB on both the Live Data Publisher and the Subscriber servers. The configuration is replicated between the Live Data Publisher and Subscriber node.
To view the configured primary and secondary AW DBs, use the command:
show live-data aw-access
Step 7
Connect Live data servers to Machine Service records (only for 4000 and higher Agent deployments)
set live-data machine-services <user@domain>
This command allows the LD servers to register themselves to the machines services table so that LD servers are discoverable. This also registers the credentials for calling Live Data API.
Requirements that the user must have to execute the machine services command successfully:
Note:
Step 8
Set up the Live Data datasource in CUIC.
set live-data cuic-datasource <cuic-fqdn> <cuic-port> CUIC\<cuic-user>
Where
Once this command is run successfully, the primary and secondary Live Data datasources can be seen under the Datasources tab on the CUIC GUI.
To view Live Data datasource configuration, use the command:
show live-data cuic-datasource <cuic-fqdn> <cuic-port> CUIC\<cuic-user>
Step 9
Download the Live Data reports from cisco.com and import the Live Data reports onto the CUIC server.
Note: The Live Data reports MUST match the version of your central controller.
Step 10
In the Cisco Finesse Administration page (https://<Finesse>/cfadmin), navigate to the Finesse Desktop Layout tab and replace the default 'my-cuic-server' with the correct CUIC Server FQDN.
Step 11
Configure Cross Origin Resource Sharing (CORS) for Live Data
a. On Finesse Publisher and Subscriber, ensure CORS is enabled using the command utils finesse cors status.
If this is disabled, you can enable it using the command utils finesse cors enable.
b. Execute the CORS commands on all CUIC servers:
utils cuic cors enable
utils cuic cors allowed_origin add https://<finesse-publisher>
utils cuic cors allowed_origin add https://<finesse-subscriber>
utils cuic cors allowed_origin add https://<finesse-publisher>:8445
utils cuic cors allowed_origin add https://<finesse-subscriber>:8445
c. Execute the CORS commands on the Live Data Publisher and Subscriber servers:
utils live-data cors enable
utils live-data cors allowed_origin add https://<finesse-publisher>
utils live-data cors allowed_origin add https://<finesse-subscriber>
utils live-data cors allowed_origin add https://<finesse-publisher>:8445
utils live-data cors allowed_origin add https://<finesse-subscriber>:8445
d. To verify the CORS configuration:
On all CUIC Servers:
utils cuic cors status
utils cuic cors allowed_origin list
On Live Data Publisher and Subscriber:
utils live-data cors status
utils live-data cors allowed_origin list
Note:
Step 12
Restart all CUIC/LD and Finesse servers using the CLI command:
utils system restart
Ensure all Live Data services are STARTED using the command 'utils service list'.
CCE Live Data ActiveMQ Service[STARTED]
CCE Live Data Cassandra Service[STARTED]
CCE Live Data Event Store Service[STARTED]
CCE Live Data SocketIO Service[STARTED]
CCE Live Data Storm DRPC Service[STARTED]
CCE Live Data Storm Nimbus Service[STARTED]
CCE Live Data Storm Supervisor Service[STARTED]
CCE Live Data Web Service[STARTED]
CCE Live Data Zookeeper Service[STARTED]
Ensure connectivity to the AW servers using the command:
show live-data aw-access
Test status must show 'Succeeded'.
Verify the Live Data Cluster state using the command
show live-data failover
Cluster State | Description |
PAIRED-ACTIVE |
The Live Data server is in the active state and is communicating with the remote side. |
PAIRED-STANDBY |
The Live Data server is in the standby state and is communicating with the remote side. |
ISOLATED-ACTIVE |
The Live Data server is in the active state, but is unable to communicate with the remote side. |
ISOLATED-STANDBY |
The Live Data server is in the standby state, but is unable to communicate with the remote side. |
SIMPLEXED-MODE |
The Live Data server is working in simplex mode. |
OUT-OF-SERVICE |
The Live Data server is out of service. |
CONNECTING |
The Live Data server is attempting to do a handshake with the remote side. |
TESTING |
The Live Data server is unable to communicate with the remote side and is using the Test-Other-Side procedure to determine whether to be in the ISOLATED-ACTIVE or ISOLATED-STANDBY state. |
UNAVAILABLE |
Live Data is not deployed. |
Note: ISOLATED active/standby status indicates a communication between Live Data servers. This does not cause Live Data datasource on CUIC to be offline.
a. If the 'show live-data failover' command shows cluster state as UNAVAILABLE.
Action Items:
b. If the 'show live-data failover' command shows cluster state as OUT-OF-SERVICE.
Action Items
c. If the 'show live-data failover' command shows cluster state as ISOLATED.
Ensure the Streaming Live Data Datasource reflects the correct hosts using the command:
show live-data cuic-datasource <cuic-fqdn> <cuic-port> CUIC\<cuic-user>
If the 'show live-data cuic-datasource' shows the correct configurations, but Live Data Datasource shows offline on CUIC:
Note: Starting Live Data version 12.6.2, the ports 12005 and 12008 are deprecated and removed in future releases. The port 443 is now used for Live Data Web Service and Live Data SocketIO Service.
Verify port connectivity from the Live Data Server CLI using the show open ports command.
show open ports regexp 4[0-5]03[45] (For Instance-0)
show open ports regexp 4[0-5]07[45] (For Instance-1)
Verify port connectivity from the command prompt on the Routers and PGs using the netstat command.
netstat -an | findstr "<LD-SideA-IP> <LD-SideB-IP>"
OR
netstat -an | findstr 4[0-5]03[45] (For Instance-0)
netstat -an | findstr 4[0-5]07[45] (For Instance-1)
a. If the ports do not even show to be in the LISTENING state:
b. If the ports are not in the ESTABLISHED state and continue to be in the LISTENING state:
a. SQL query to check for number of Agents configured per team:
Run query against the awdb (No production impact)
Select TeamName = AT.EnterpriseName, NumAgentsOnTeam = COUNT(ATM.SkillTargetID), SupervisorName = Person.LastName + ', ' + Person.FirstName
FROM Agent_Team AT LEFT OUTER JOIN
(Select * from Agent ) Agent ON AT.PriSupervisorSkillTargetID = Agent.SkillTargetID LEFT OUTER JOIN Person ON Person.PersonID = Agent.PersonID,
Agent_Team_Member ATM
WHERE ATM.AgentTeamID = AT.AgentTeamID
GROUP BY AT.EnterpriseName, Person.LastName + ', ' + Person.FirstName
ORDER BY AT.EnterpriseName
b. SQL query to check for number of Agents configured per skill group:
Run query against the awdb (No production impact)
Select Skill_Group.EnterpriseName, NumAgentsInSG = COUNT(Skill_Group_Member.AgentSkillTargetID)
FROM Skill_Group, Skill_Group_Member
WHERE Deleted = 'N' AND Skill_Group.SkillTargetID = Skill_Group_Member.SkillGroupSkillTargetID
GROUP BY EnterpriseName;
c.
If Live Data issues are seen after an upgrade, check the 'DBMaintenance' configuration value.
If it is disabled, enabling the configuration changes by setting DBMaintenance to 0 and restart the Apache Tomcat service on the AW Server.
Registry path: HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc.\ICM\<instance name>\Router<A/B>\Router\CurrentVersion\Configuration\Global\DBMaintenance.
file get activelog livedata/logs recurs compress
This command collects logs for all of the Live Data components
Alternatively, you can collect logs for the individual Live Data components as well.
CCE Live Data ActiveMQ
file get activelog livedata/logs/livedata-activemq
CCE Live Data Cassandra Service
file get activelog livedata/logs/livedata-cassandra
CCE Live Data Event Store Service
file get activelog livedata/logs/livedata-event-store
CCE Live Data SocketIO Service
file get activelog livedata/logs/socketio-service
CCE Live Data Storm Services
file get activelog livedata/logs/livedata-storm
CCE Live Data Web Service
file get activelog livedata/logs/livedata-web
CCE Live Data Zookeeper Service
file get activelog livedata/logs/livedata-zookeeper
Intelligence Center Reporting Service
file get activelog cuic/logs/ recurs compress
Collect Browser Console & Network logs for the duration issue. Begin with clearing the Cache, restart the browser and capture the logs from the login time onwards, covering the attempt made to reproduce the issue
For the Chrome/Edge browser:
For the Firefox browser:
Command to run Packet Captures on VOS Servers:
Collect Packet Captures using RTMT:
Collect Packet Captures using CLI:
Packet captures remains on disk until deleted. It is recommended to delete these captures periodically.
12.6(1)
Solution Design Guide
Install and Upgrade Guide
Revision | Publish Date | Comments |
---|---|---|
1.0 |
14-Feb-2024 |
Initial Release |