OpenTelemetry in Microsoft Agent Framework Apps

Observability has become a critical component for ensuring the performance and reliability of applications. OpenTelemetry, an open-source observability framework, provides developers with the tools necessary to collect and export telemetry data from their applications. When integrated with the Microsoft Agent Framework (MAF), OpenTelemetry offers profound insights into the performance and behavior of AI agents. This post will explore how to effectively use OpenTelemetry in a Microsoft Agent Framework application, highlighting key trends, real-world use cases, challenges, and providing practical code examples.

What is OpenTelemetry?

OpenTelemetry is a set of APIs, libraries, agents, and instrumentation that enables developers to collect telemetry data from their applications. This data can include traces, metrics, and logs, which are essential for monitoring application performance and diagnosing issues. OpenTelemetry is designed to be vendor-agnostic, allowing developers to send their telemetry data to various backends for analysis.

Key Trends and Innovations in OpenTelemetry and MAF

1. Built-in Support for OpenTelemetry

One of the most significant advancements in the Microsoft Agent Framework is its built-in support for OpenTelemetry. This integration allows developers to automatically emit spans and metrics without the need for manual wrapping of every agent call. This feature simplifies the process of adding observability to applications, enabling developers to focus on building functionality rather than instrumentation.

2. Semantic Conventions

OpenTelemetry employs semantic conventions that help structure telemetry data in a meaningful way. In the context of AI agents, the OpenTelemetry GenAI semantic conventions ensure that the telemetry data collected is relevant and useful for analysis. This structured approach aids developers in understanding the performance and behavior of their agents more effectively.

3. Support for Multi-Agent Systems

The Microsoft Agent Framework is designed to support multi-agent architectures, allowing for complex interactions and behaviors to be monitored effectively. This capability is particularly beneficial in scenarios where multiple agents work together to complete tasks, as it provides a comprehensive view of the system’s performance.

By integrating OpenTelemetry, developers can monitor agent performance and interactions, gaining insights into usage patterns, error rates, and overall system efficiency. This data can be invaluable for optimizing the application and enhancing user experience.

Challenges in Integration

While the integration of OpenTelemetry with the Microsoft Agent Framework offers numerous benefits, there are challenges that developers may encounter. One notable issue is the missing Activity Events. Reports indicate that while spans and metrics are emitted correctly, some expected Activity Events, such as gen_ai.client.inference.operation.details, are not being generated. This limitation can restrict the granularity of observability, making it difficult to gain a complete understanding of agent interactions and performance.

Code Example: Setting Up OpenTelemetry in a Microsoft Agent Framework Application

To illustrate how to set up OpenTelemetry in a Microsoft Agent Framework application, consider the following code example:

using Microsoft.Extensions.AI;

// Create OpenAI client
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");
var apiKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY");
var deploymentName = "gpt-4o-mini";

using var client = new AzureOpenAIClient(new Uri(endpoint), new AzureKeyCredential(apiKey))
    .GetChatClient(deploymentName)
    .AsIChatClient()
    .AsBuilder()
    .UseOpenTelemetry(sourceName: "MyAgent", configure: (cfg) => cfg.EnableSensitiveData = true)
    .Build();

var thread = client.GetNewThread();
logger.LogInformation("Agent created successfully with ID: {AgentId}", client.Id);

Explanation of the Code

1. Environment Variables: The code begins by retrieving the Azure OpenAI endpoint and API key from environment variables. This approach ensures that sensitive information is not hard-coded into the application.
2. Creating the OpenAI Client: An instance of AzureOpenAIClient is created using the retrieved endpoint and API key. This client is responsible for interacting with the OpenAI service.
3. Using OpenTelemetry: The UseOpenTelemetry method is called on the chat client builder. This method configures OpenTelemetry for the agent, allowing it to emit telemetry data. The sourceName parameter is set to “MyAgent,” and sensitive data emission is enabled.
4. Creating a New Thread: Finally, a new thread is created for the chat client, and a log message is generated to confirm the successful creation of the agent.

You can use OpenTelemetry on each agent, or you can instrument the model calls. To do the latter you only need to update Program.cs

IChatClient llm = openAIClient
    .GetChatClient(modelName)
    .AsIChatClient()
    .AsBuilder()
    .UseFunctionInvocation()
    .UseOpenTelemetry(sourceName: "BlogWriter.Agents")
    .Use(inner => new TokenCapChatClient(inner, maxTotalTokens: 10000))
    .Build();

Per agent allows you to capture which agent ran and agent-level timing. Per model call allows you to track model name, token usage, tool calls and per-round-trip latency.

Conclusion

Integrating OpenTelemetry with the Microsoft Agent Framework significantly enhances the observability of AI agents, providing developers with valuable insights into their performance and interactions. The built-in support for OpenTelemetry, along with semantic conventions and multi-agent system capabilities, makes it a robust choice for monitoring complex applications.