Treat your AI agents like eager but misguided human interns – before you lose control
Follow ZDNET: Add us as a preferred source on Google.
ZDNET’s key takeaways
- Find a balance between AI agent restraint and independence.
- Context and intent must be woven into agent development.
- Consider configurations and the data that agents access.
AI agents are evolving from simple chatbots to full-fledged digital workers authorized to take action on applications and data. And with those capabilities come a raft of security and governance concerns.
Treat your AI agents as eager but misguided interns, requiring the same oversight and guidance as human interns, suggested experts in a panel held at the recent Snowflake Summit in San Francisco. AI agents require specific instructions and careful monitoring by human managers.
Also: How to build better AI agents for your business – without creating trust issues
An agent without restraints can be extremely problematic, the panelists, representing AI security providers, agreed. “You may tell the agent to buy you shoes, and before you know it, it has bought you a car,” said Mayank Agarwal, founder and CTO of Resolve AI.
Restraint, context, and intent
“You have to think very hard about what permissions you’re giving the agent. You can’t just expect an agent to stay on the straight and narrow. You have to put these ironclad constraints around it to limit what it’s able to do.”
Along with restraint, context and intent are the key watchwords for spinning up and managing agents. “It’s not just enough to know what this agent was created to do. You also have to know things like whose authority it is acting under and what it’s going to do, for example, with data it’s accessing,” said Nancy Wang, chief technology officer for 1Password.
Also: What you’ll pay for AI agents will be wildly variable and unpredictable
Professionals should throw out the old software development rulebook, as building and deploying agents today is very different from software practices of the recent past, Agarwal pointed out.
“If you go back just two years, an engineer knew exactly how they were going to connect APIs across different systems,” he said. “The whole thing was very predictable: A is going to call API B, B is going to do this with that data, and call C, and do this with that data. In the agentic world, it’s completely unpredictable. The agent wires the stuff on the fly. Give it a goal, solve this problem, and it goes out and tries all the paths that it has access to.”
This approach can lead to new types of issues for which professionals and managers are not prepared. The agent is “talking to tools which are capable of doing things on your behalf, so you don’t know if these tools are exfiltrating data,” Agarwal said. “The agent may read from a tool and use another tool to write it to someplace it shouldn’t be.”
The specter of shadow AI
This concern raises the specter of shadow AI, operating out of view. “We had a client that had 12 OpenClaw instances within their framework, with access to API feeds, source code, and a contractor using Telegram to communicate,” said Jason Merrick, senior vice president of product at Tenable. “What could go wrong, right?”
Also: AI agents of chaos? New research shows how bots talking to bots can go sideways fast
As a result of these issues, understanding what agents do behind the scenes can be a challenge. Questions will arise, such as “Who actually took an action against this system? Is it a human? Is it a service account? Or is it an agent?” Wang said. “Your team probably doesn’t know, or there’s not 100% certainty to that answer. Because today, agents look like humans, but they also could look like a service account, because they have all your permissions.”
Therefore, a balance needs to be struck between governance and access, as AI is a powerful tool for productivity and innovation that must be able to act independently. “You don’t want to just block everything or firewall everything,” Wang advised.
That need for balance also explains why deep human oversight is essential. “Look at the user pieces the employees are creating — through Copilot, Claude Chat, or Gemini,” Merrick advised. “Look at their configurations. Is AI misconfigured? What type of data is it accessing? And be able to take action on that. Also, look at the prompts themselves. What are the prompts communicating with?”
Bottom line: Specific instructions
This area is where guardrails and traditional identity best practices are crucial, Wang said. The greatest risk will come “from an agent that’s over-permissioned with longstanding credentials.”
Also: Can a newbie really vibe code an app? I tried Cursor and Replit to find out
The challenge is designing security and governance around what are “non-deterministic beings,” Wang continued. “It’s a matter of allowing them to be creative, but also to apply essentially traditional instruction sets in the form of SDKs. You want predictable controls, but also, you don’t want to constrain them so much that it no longer gets you productivity gains.”
The bottom line for professionals to heed is that agents, like interns, need “very, very specific instructions,” Wang said. “Sometimes they still veer off the desired path. Whether you think about governing agents or whether you think about full agent traces comes back to full visibility, remediation, and making sure that you set the right intent from the get-go — and that intent must persist across every step, every action that the agent takes.”
Artificial Intelligence
READ MORE HERE