Viewing the Zoom Virtual Agent Chatbot Performance dashboard

The Zoom Virtual Agent Chatbot Performance dashboard provides admins with a visual overview of their Zoom Virtual Agent solution's performance. This allows you to identify areas where your chatbot can be improved and take action to optimize it. The dashboard includes metrics such as self-service rate (SSR), count of engagements, and average response time. It also allows you to explore how your bot flows are performing in terms of resolving customers' issues, resulting in fewer support contacts.

You also have the option to download the chatbot data for in-depth analysis and subscribe to a weekly email report to regularly gain insights into the chatbot's performance and trends.

This article covers:

• How is performance measured for Zoom Virtual Agent chatbot
• How to view the Zoom Virtual Agent Chatbot Performance dashboard
  • Overview tab
  • Bot Flows tab
  • Intents tab
  • Knowledge Base tab
  • Query Insights tab
• How to access the individual Chatbot Performance dashboard

Prerequisites for viewing the Zoom Virtual Agent Chatbot Performance dashboard

• Any account owner or admin role with Virtual Agent permission
• Basic, Pro, Business, Education, or Enterprise account
• Zoom Virtual Agent license

How is performance measured for Zoom Virtual Agent chatbot

A chatbot is considered to be performing well when it allows customers to resolve their issues on their own without having to contact a human agent. However, there are some cases where it is necessary for customers to contact an agent, such as when the issue can only be resolved by a human. In general, businesses want to minimize the number of support contacts, as long as their customers are satisfied.

Every time a customer interacts with a chatbot, even if it is about multiple issues, it is called an engagement. An engagement begins when the user types a message, clicks a quick-reply button, or is automatically routed to a support channel. The performance of the chatbot for that engagement is determined by all of the elements of the chatbot that contributed to the customer's experience. The outcome of every engagement can be one of the following:

• Assistance requested: The customer requested assistance from a human agent.
• Dropped off: The customer started to request assistance but did not complete the process.
• Assisted by design: The customer requested assistance, but it was by design.
• Self-serviced: The customer did not request assistance at all.

The key metric, self-service rate (SSR), is calculated as the percentage of all engagements for which the outcome was self-serviced.

How to view the Zoom Virtual Agent Chatbot Performance dashboard

1. Sign in to the Zoom web portal.
2. In the navigation menu, under Personal, click Analytics & Reports.
3. Click the Virtual Agent Analytics tab.
4. In the Historical Reports section, click Chatbot Performance.
   You will be taken to the Chatbot Performance dashboard.

Overview tab

This tab shows reports on how your chatbot is performing in terms of resolving customers' issues versus generating support contacts.

• Overview: This tab gives an overall view of how the virtual agent is performing. The report initially shows data for one week, but you can adjust it to display any date range you prefer. You can also filter the report by Bot, Campaign, Outcome, Global Variable, and Language.
  • Chatbot Overview: Displays the following metrics:
    • Self-service Rate (SSR): This represents the percentage of engagements successfully resolved by your Virtual Agent. The SSR is calculated by dividing self-serviced engagements by total engagements.
    • Engagements: This is the total number of engagements during the date range of the report. An engagement begins when the user asks a question or interacts purposefully with the chatbot, and ends when they close the chatbot. If the user interacts with the chatbot after an hour of inactivity, a new engagement begins.
    • Self-serviced Engagements: This is the total number of self-serviced engagements during the period. Immediately below is cost savings – the number of self-serviced engagements times the cost savings per engagement. You can adjust your cost savings in the Settings menu on the upper right of the page by specifying either the general Engagement cost savings for each engagement, or setting a bot flow-specific cost savings. Both default to $5 USD. Each self-serviced engagement has one cost savings, no matter how many bot flows were used.
    • Self-service Rate and Engagements: A line graph showing the total number of engagements and the percentage of self-service rate over the selected period of time.
  • Engagement Details: A pie chart that displays the engagement distribution breakdown and a line chart that displays the engagement trends. These are specific information about an engagement that can be used to track the progress of an engagement. Click the drop-down menu and choose from the following options:
    • Outcome: This refers to the result of the engagement. Refer to the engagement outcomes discussed in the previous section.
    • Campaign: This refers to the campaigns that you’ve set up in your Virtual Agent solution.
    • Launch URL: This refers to the URL of the website or landing page that the virtual assistant was working on.
    • Browser: This refers to the web browsers your users were using when they used the Virtual Agent.
    • Operating System: This refers to the operating system your users were using when they used the Virtual Agent.
    • Country: This refers to the country from which your users interacted with your Virtual Agent.
  • Bot: A list of your bots, with total number of engagements, self-service rate, and assistance requested. Click on a bot’s name to see its individual performance.
• Detail: This tab lists the engagements which occurred during the report period. Some engagements occurred in multiple sessions, which are each listed separately. Click on the session ID to view the actual transcript of the interaction between the end user and the chatbot. You can search it by engagement ID and filter by Bot, Campaign, Engagement outcome, Global Variable and Language.
• Data Download: Allows admins to download their data for a more detailed analysis.

Bot Flows tab

This tab shows reports on how your bot flows are performing. The report shows data for one week, but you can adjust it to display any date range you prefer. You can also filter the report by Bot, Campaign, and Bot Flow.

• Bot Flow Outcomes: This pie chart shows the breakdown of engagement outcomes within a specific time frame.
• Bot Flow Outcome Trends: Shows the trend of engagement outcomes. Hover over the chart to view details for a given date.
• Bot Flow Average Duration: A trend chart showing how much time your users spent in your Bot Flows. This includes the shortest time, average time, and longest time spent.
• Bot Flow Performance: This shows the self-service rate (SSR) of each bot flow, over the date range. The SSR of a bot flow is determined by the outcome of the engagement in which it was used.
• Cost Savings Trend: A trend chart showing the support costs saved by handling issues through the Virtual Agent. The cost savings per self-serviced engagement can be configured in your Settings, at the upper right of the page.

Intents tab

This tab provides valuable insights into the chatbot's performance and its ability to understand and respond to user queries effectively. The report shows data for one week, but you can adjust it to display any date range you prefer. You can also filter the report by Intent, Engagement Outcome, and Language.

• Matched Intent Breakdown: A pie chart that shows the number and breakdown of intents that the chatbot successfully identified in the user's messages.
• Intent Distribution: This shows how frequently the top or bottom 5 performing intents occured, based on the SSR of the engagements in which they were triggered.
• Intent Trend: This shows the trend of top or bottom 5 used intents.

Knowledge Base tab

This tab covers the knowledge base articles that were used and displays the most frequently used knowledge base articles.

• Overview: This tab gives an overall view of the knowledge base articles used. The report shows data for one week, but you can adjust it to display any date range you prefer. You can also filter the report by Language.
  • Article Usage: This shows the top or bottom 5 viewed articles.
  • Article Usage Trends: This shows the usage trends of your KB articles over time.
  • Article Performance: This shows the top or bottom 5 performing articles, based on the self-service rate (SSR) of the engagements in which they were viewed
  • Article Performance Trends: This shows the performance trends of your KB articles over time.
• Articles: A list of your knowledge base articles, including the self-service rate (SSR) and total views for each article, during the period of the report. Drill down to see the individual user queries which led to viewing the article, grouped by the outcome of the engagement. You can also search for articles by URL.

Query Insights tab

This tab offers valuable insights into the queries your customers are entering into the chatbot. The report shows data for one week, but you can adjust it to display any date range you prefer. You can filter the report by Bot.

• Query Matching: A pie chart breaking down how queries were matched by Virtual Agent. Queries can match an intent (normally tied to a bot flow) or knowledge base article, and sometimes neither.
• Query Trend: A bar graph that shows the trends of query matching and engagement outcomes.
• Query Handling: This sankey diagram and table help you analyze customer queries in detail. On the left of the diagram, you will see whether the query matched an intent, an article, or if there was no match at all. On the right, you will see the outcome of that engagement. Click any portion of the diagram to filter the table of queries below. You may also search manually for individual queries, by entering matching text.

How to access individual Chatbot Performance dashboard

1. Sign in to the Zoom web portal.
2. In the navigation menu, under Admin, click AI Management and then Virtual Agents.
3. In the Chatbots tab, click the display name of an existing chatbot.
4. Click the Performance tab.