{"id":60334,"date":"2026-03-18T16:00:00","date_gmt":"2026-03-18T16:00:00","guid":{"rendered":"https:\/\/www.microsoft.com\/en-us\/security\/blog\/?p=145844"},"modified":"2026-03-18T16:00:00","modified_gmt":"2026-03-18T16:00:00","slug":"observability-for-ai-systems-strengthening-visibility-for-proactive-risk-detection","status":"publish","type":"post","link":"https:\/\/www.threatshub.org\/blog\/observability-for-ai-systems-strengthening-visibility-for-proactive-risk-detection\/","title":{"rendered":"Observability for AI Systems: Strengthening visibility for proactive risk detection"},"content":{"rendered":"
<\/div>\n

Adoption of Generative AI (GenAI) and agentic AI has accelerated from experimentation into real enterprise deployments. What began with copilots and chat interfaces has quickly evolved into powerful business systems that autonomously interact with sensitive data, call external APIs, connect to consequential tools, initiate workflows, and collaborate with other agents across enterprise environments. As these AI systems become core infrastructure, establishing clear, continuous visibility into how these systems behave in production can help teams detect risk, validate policy adherence, and maintain operational control.<\/p>\n

Observability<\/a> is one of the foundational security and governance requirements for AI systems operating in production. Yet many organizations don\u2019t understand the critical importance of observability for AI systems or how to implement effective AI observability. That mismatch creates potential blind spots at precisely the moment when visibility matters most.<\/p>\n

In February, Microsoft Corporate Vice President and Deputy Chief Information Security Officer, Yonatan Zunger, blogged about expanding Microsoft\u2019s Secure Development Lifecycle (SDL)<\/a> to address AI-specific security concerns. Today, we continue the discussion with a deep dive into observability as a necessity for the secure development of GenAI and agentic AI systems. <\/p>\n

For additional context, read the Secure Agentic AI for Your Frontier Transformation<\/a> blog that covers how to manage agent sprawl, strengthen identity controls, and improve governance across your tenant.<\/em><\/p>\n

Observability for AI systems<\/h2>\n

In traditional software, client apps make structured API calls and backend services execute predefined logic. Because code paths follow deterministic flows, traditional observability tools can surface straightforward metrics like latency, errors, and throughput to track software performance in production.<\/p>\n

GenAI and agentic AI systems complicate this model. AI systems are probabilistic by design and make complex decisions about what to do next as they run. This makes relying on predictable finite sets of success and failure modes much more difficult. We need to evolve the types of signals and telemetry collected so that we can accurately understand and govern what is happening in an AI system.<\/p>\n

Consider this scenario: an email agent asks a research agent to look up something on the web. The research agent fetches a page containing hidden instructions and passes the poisoned content back to the email agent as trusted input. The email agent, now operating under attacker influence, forwards sensitive documents to unauthorized recipients, resulting in data exfiltration.<\/p>\n

In this example, traditional health metrics stay green: no failures, no errors, no alerts. The system is working exactly as designed\u2026 except a boundary between untrusted external content and trusted agent context has been compromised.<\/p>\n

This illustrates how AI systems require a unique approach to observability. Without insights into how context was assembled at each step\u2014what was retrieved, how it impacted model behavior, and where it propagated across agents\u2014there is no way to detect the compromise or reconstruct what occurred.<\/p>\n

Traditional monitoring, built around uptime, latency, and error rates, can miss the root cause here and provide limited signal for attribution or reconstruction in AI-related scenarios. This is an example of one of the new categories of risk that the SDL must now account for, and it is why Microsoft has incorporated enhanced AI observability practices within our secure development practices.<\/p>\n

Traditional observability versus AI observability<\/h2>\n

Observability of AI systems means the ability to monitor, understand, and troubleshoot what an AI system is doing, end-to-end, from development and evaluation to deployment and operation. Traditional services treat inputs as bounded and schema-defined. In AI systems, input is assembled context. This includes natural language instructions plus whatever the system pulls in and acts on, such as system and developer instructions, conversation history, outputs returned from tools, and retrieved content (web pages, emails, documents, tickets).<\/p>\n

For AI observability, context is key: capture which input components were assembled for each run, including source provenance and trust classification, along with the resulting system outputs.<\/p>\n

Traditional observability is often optimized for request-level correlation, where a single request maps cleanly to a single outcome, with correlation captured inside one trace. In AI systems, dangerous failures can unfold across many turns. Each step looks harmless until the conversation ramps into disallowed output, as we\u2019ve seen in multi-turn jailbreaks like Crescendo<\/a>.<\/p>\n

For AI observability, best practices call for propagating a stable conversation identifier across turns, preserving trace context end-to-end, so outcomes can be understood within the full conversational narrative rather than in isolation. This is \u201cagent lifecycle-level correlation,\u201d where the span of correlation should be the same as the span of persistent memory or state within the system.<\/p>\n

Defining AI system observability<\/h2>\n

Traditional observability is built on logs, metrics, and traces. This model works well for conventional software because it\u2019s optimized around deterministic, quantifiable infrastructure and service behavior such as availability, latency, throughput, and discrete errors.<\/p>\n

AI systems aren\u2019t deterministic. They evaluate natural language inputs and return probabilistic results that can differ subtly (or significantly) from execution to execution. Logs, metrics, and traces still apply here, but what gets captured within them is different. Observability for AI systems updates traditional observability to capture AI-native signals.<\/p>\n

Logs, metrics, and traces indicate what happened in the AI system at runtime.<\/p>\n