AI governance: the missing link in Enterprise security

AI is moving faster than your controls.

Enterprises are rapidly integrating generative AI, copilots, AI coding assistants, retrieval-augmented generation (RAG) systems and autonomous agents into core operations. The productivity gains are undeniable, but so is the widening security gap between deployment speed and control.

Too many organizations still view AI governance as a checkbox exercise: a policy document, a privacy review or a simple warning against pasting sensitive data into public chatbots. That approach is dangerously insufficient.

AI systems are now part of the enterprise attack surface.

They interact with:

• Source code and developer environments
• Customer records and internal knowledge bases
• Cloud storage
• SaaS platforms and identity systems
• Business-critical workflows

Real-world incidents show how AI tools can accelerate data exposure, enable prompt injection, compromise software supply chains and trigger risky agentic behavior.

Not every incident resulted in a complete “full breach,” and it is important not to overstate them. The deeper lesson is that AI acts as a threat multiplier: it makes data movement easier, retrieval boundaries harder to enforce, workflows more autonomous and sensitive context more accessible through new interfaces.

Here's the fix.

Organizations need a paradigm shift: AI governance can no longer live as a passive policy layer managed only by legal or compliance teams. It must evolve into an active, operational security discipline.

Shadow AI: turning the prompt box into a data-loss channel

One of the earliest warnings came from Samsung’s reported ChatGPT data-leak incident in 2023. The company reportedly restricted employee use of ChatGPT and similar tools after reports that engineers pasted sensitive source code and other internal information into the public tool. This was not an external hack of Samsung systems. It was a governance failure: employees turning to unapproved AI to boost productivity. Forbes also covered the incident.

This exemplifies the shadow AI problem. Security teams have long monitored email, USB drives, unmanaged file-sharing services and personal cloud storage. Generative AI added a new vector: the prompt box itself. A prompt can contain source code, customer data, credentials, strategy documents, incident details, HR information or regulated information.

IBM’s "2025 Cost of a Data Breach" research highlights the scale. One in five organizations experienced a breach due to shadow AI; only 37% had policies to manage or detect it. Additionally, high shadow-AI usage correlated with $670,000 higher average breach costs. Broader findings showed that 13% of organizations reported breaches of AI models or applications. 97% of these organizations lacked proper AI access controls.

Governance recommendations

Treat consumer AI tools as potential exfiltration channels.

Implement:

• AI usage visibility
• DLP for prompts and uploads
• Data classification for AI inputs
• Controls that distinguish approved enterprise accounts from personal accounts
• Secure internal alternatives that employees will use

AI data retrieval: 'private later' is not private enough

Enterprise AI shines because it retrieves and reasons over internal data. That same capability creates risk. The key governance question is simple: can the AI surface information the user or current context should not access?

A notable case involved Microsoft Copilot and GitHub repositories. According to TechCrunch, more than 20,000 repositories that had once been public, but were later deleted or made private, still had data accessible through Copilot via Bing’s caching mechanism. This affected more than 16,000 organizations.

This was not a typical case in which an AI assistant ignored a user’s file permissions or used an overprivileged internal account to retrieve private data. It is better described as a zombie-data and AI retrieval problem: data that was exposed at one point can remain discoverable through indexes, caches, model-connected search layers or retrieval systems even after the original repository is made private.

The same pattern applies inside enterprises. If an AI assistant indexes shared drives, Confluence, Slack, email, GitHub, Jira or SharePoint, governance must answer several operational questions:

• Can the AI enforce the user’s current permissions at retrieval time?
• Can it forget data after permissions change and purge cached or indexed content?
• Can it detect secrets in indexed material?
• Can it distinguish public, internal, confidential and restricted content?
• Can administrators audit what data was retrieved, summarized or exposed?

Governance recommendations

Enforce context-aware access controls. Retrieval layers should pass user identity and authorization context to source systems in real time. Indexes should respect deletion, retention, permission changes and secret-scanning policies. If a user cannot access a file directly, the AI should not summarize it indirectly.

Indirect prompt injection: when untrusted content controls the copilot

Prompt injection is one of the defining AI-native security risks. The Open Worldwide Application Security Project (OWASP) describes prompt injection as a vulnerability where user prompts alter an LLM’s behavior or output in unintended ways. The OWASP Top 10 for LLM Applications lists prompt injection as LLM01.

The more dangerous variant is indirect prompt injection. In this scenario, the attacker does not directly prompt the AI system. Instead, they place malicious instructions inside content the AI is likely to process:

• Email, Slack message or calendar invite
• PDF or other document
• Webpage
• Support ticket
• GitHub README

EchoLeak, tracked as CVE-2025-32711, is a major example. NVD classifies the issue as AI command injection in Microsoft 365 Copilot that could allow unauthorized information disclosure over a network. Research on EchoLeak describes a zero-click prompt-injection vulnerability in Microsoft 365 Copilot where a single crafted email could enable remote, unauthenticated data exfiltration without user interaction.

The structural flaw is semantic conflation: to an LLM, there is no native security boundary between system instructions and data payloads. When a copilot reads an email body, it does not inherently know which text is content to summarize and which text is a hidden command attempting to manipulate its behavior.

This is the confused-deputy problem in AI form. The user may be authorized to access sensitive data. The copilot may be authorized to retrieve it. But a malicious external email can influence the copilot into using that access in an unintended way.

Governance recommendations

Treat all external and user-controlled content as untrusted. AI systems need:

• Prompt-scope isolation and provenance tracking
• Output filtering and external-link controls
• Image-fetch restrictions
• Tool-use boundaries
• Continuous adversarial testing

Additionally, retrieved documents should be data only — not executable instructions.

The prevalence of AI-related data exposure

AI leaks go beyond employees pasting information into public tools. It can also occur when historical data remains indexed, when authorization context is lost during retrieval or when untrusted content manipulates a copilot that already has legitimate access to sensitive information.

Governing these systems requires visibility into the complete data path: the source and user identity --> the prompt, the retrieval layer and the model response --> the cache, log, and external destination.

It is only through this deep visibility and thorough investigation that companies can protect their data from unsafe AI practices.