Insight
Last updated:2025-01-01

Insight

Overview

For published Agents, you can track and analyze their performance through the insights dashboard. The Agent insights module provides usage data including Agent effectiveness, usage volume, user statistics, and health metrics. You can use this data to optimize the Agent in targeted ways to better meet user needs.

image-20250120143748471

Enter [Agent] and access the Insights dashboard through [Insights] in the left menu.

Data Definition

All metrics in the insight dashboard are composed of the following core data:

  • User: The number of anonymous IDs generated in the AI Agent. An anonymous ID is a unique identifier used to locate users.

    • New Users: The number of users created by the AI Agent within a specified time period.

    • Active Users: The number of users who have received at least 1 message from the AI Agent within a specified time period.

  • Conversation: The number of "conversation_id"s in the AI Agent. Multiple interactions constitute one conversation.

  • Interaction: The number of "qa_id"s in the AI Agent. One interaction (Q&A) consists of 1 user question and 1 AI Agent response.

  • Credit: The number of credits consumed by the AI Agent.

  • Token: The number of tokens consumed by the AI Agent.

  • Response Time (seconds): The time taken for the AI Agent's message request to receive a response.

  • Error: The number of error responses generated from the AI Agent's message requests.

  • Average Requests Per Minute: The average number of message requests initiated by the AI Agent per minute.

image-20250120143838602

For the meaning of each specific indicator, please refer to the tooltip information on the page for details.

Metric Categories

In the Insights Dashboard, all metrics are categorized into six major groups, allowing you to view data in a targeted way.

Note: While [Real-time] shows today's data, the other 5 categories display data up through the previous day at the latest.

  • Real-time: Monitor the AI Agent's various data metrics throughout the current day, with real-time updates providing timely insights.

  • Effectiveness: Track the AI Agent's performance in resolving user issues, including user feedback, issue resolution rate, and the percentage of cases escalated to human customer service.

  • Usage: Monitor the AI Agent's credit consumption, including trends in credits and Token usage, and their distribution across different dimensions.

  • Users: Track the AI Agent's user metrics, including cumulative user count, new user growth, and active user numbers.

  • Behavior: Analyze the AI Agent's conversation data, including trends in conversation volume, interaction counts, and their distribution across various dimensions.

  • Health: Monitor the AI Agent's performance-related metrics, such as response time and error rates.