🩺 The Scalpel Doesn't Sign the Surgery
An Agent With Tools Needs Operational Boundaries
A team connects an agent to their repository, to the database, and to the executive dashboard. The demo takes minutes, corrects a field, opens a change, and leaves everyone with the feeling of having seen the future. One question is missing before applause.
Using an agent is not the same as governing it when it can touch code, data, APIs, infrastructure, reports, and business processes.
The hand still carries responsibility.
An agent stops being a passive assistant when it can execute steps, call tools, modify files, query databases, open requests, update dashboards, or trigger pipelines. At a Colombian company that can sound very attractive for accelerating support, analytics, migrations, or data governance. It can also become an operational risk if given unlimited permissions, traceability, or reversal plan.
NIST published in 2024 the generative AI profile for its AI Risk Management Framework. The document doesn't treat AI as a productivity trick, but as a system that must be designed, evaluated, governed, and monitored throughout its lifecycle. That perspective matters because an agent doesn't just produce text. It produces chained actions with cumulative effects.
OWASP brought the point to applied security. In its Top 10 for applications with LLM of 2025, excessive agency risk appears when a system receives too much functionality, too many permissions, or too much autonomy. The definition is simple and strong. Damage doesn't depend only on the model hallucinating. It can also come from an ambiguous instruction, a prompt injection, a compromised tool, or a chain of agents amplifying the error.
AWS, in its prescriptive guide for agentic AI security, maps that risk to concrete controls like limiting agent scope, threat modeling, applying adaptive authentication, restricting operations against sensitive systems, monitoring behavior, and having emergency shutdown for high-risk scenarios. The matter no longer fits in prompt engineering. It enters architecture, security, and operation.
Let's think about an operating room. A very precise scalpel doesn't decide where to cut, how much to cut, or when to stop. The surgeon answers for diagnosis, consent, sterilization, equipment, monitoring, bleeding, closure, and recovery. In technology, a powerful agent is that scalpel. It can accelerate an intervention, but it doesn't sign the surgery.
Governing agents means defining what it can do, on what systems, with what permissions, under what evidence, with what human oversight, and with what reversal mechanism. Permission is concrete authorization to execute an action. Traceability is being able to reconstruct what happened, who asked for it, what tool was invoked, and what changed. Lineage is knowing where data comes from and how it was transformed before reaching a report or model.
Without those foundations, the agent becomes a practitioner with access to the operating room. It can seem useful because it moves fast, but nobody serious confuses movement with control. The hand still carries responsibility.
The scene shows easily in data. A team wants an agent to automatically correct customer tables before loading an executive report. The rushed version gives it a function that writes straight on the database. The version with judgment separates reading, proposing, validating, and approving.
# Without governance
def corregir_cliente(db, cliente_id, nuevo_email):
db.execute(
"update clientes set email = ? where id = ?",
[nuevo_email, cliente_id]
)
# With governance
def proponer_correccion(db, usuario, cliente_id, nuevo_email):
actual = db.query("select email from clientes where id = ?", [cliente_id])
cambio = {
"tabla": "clientes",
"campo": "email",
"antes": actual[0]["email"],
"despues": nuevo_email,
"solicitado_por": usuario,
"requiere_aprobacion": True
}
db.insert("cambios_pendientes", cambio)
return cambio
The first function seems efficient, but mixes recommendation and execution. If the agent misinterprets an instruction, receives contaminated data, or acts on the wrong customer, the error enters production directly. The second doesn't stop productivity. It channels it. The agent prepares an auditable proposal, leaves evidence of before and after, identifies the requester, and demands approval before touching master data.
That pattern applies equally to code, infrastructure, and BI. An agent can open a pull request, but shouldn't deploy to production without tests. It can suggest pipeline changes, but shouldn't delete history without backup. It can generate an indicator, but shouldn't publish a dashboard without validating business rule. The hand still carries responsibility.
The most useful historical precedent doesn't come from AI, but from algorithmic trading. In 2013, the SEC penalized Knight Capital for failures associated with an incident in August 2012. The investigation found insufficient safeguards, inadequate deployment controls and testing, and millions of erroneous orders sent to the market. The system acted fast. The damage was fast too.
The lesson for agents is direct. When automation has real access to critical systems, it's not enough to ask whether the logic seems intelligent. You have to know what happens when it fails, who stops it, what limit it had, what log it leaves, and how previous state is recovered.
Recommended Resources
- NIST AI Risk Management Framework Generative AI Profile type report
- OWASP Agentic Skills Top 10 type security guide
- AWS Security for agentic AI type technical guide
- OpenAI Agents SDK Guardrails type technical documentation
In Colombia and LATAM the risk becomes very concrete. There are small teams with pressure to reduce times, automate reports, accelerate migrations, or serve internal users without growing payroll. The agent promises to ease that pressure. But if connected to a production database with a shared admin user, if no one registers its calls, if there's no test environment, and if approvals happen through chat, the supposed savings becomes operational debt.
In BI, the risk isn't always deleting a table. Sometimes it's more subtle. An agent can change a sales metric, mix accounting states, duplicate customers, or publish an executive reading without explaining assumptions. In data engineering it can retry a pipeline until doubling loads. In cloud it can create resources without tags or limits. In architecture it can integrate services without reviewing identity, network, or cost.
The human capacity that makes the difference is not writing the prettiest prompt. It's knowing how to design limits. A good professional asks what actions are reversible, which require approval, what data is sensitive, what logs are kept, what tests block change, what minimum permissions the agent needs, and what alert indicates something went wrong.
Talent evaluation also changes. Asking if someone has used agents says little. Better to ask them to design one for a real task and explain permissions, testing, observability, recovery, and responsibility. Observability means being able to see system behavior through logs, metrics, and events. If the person only talks about speed and not control, they're still thinking in demo, not operation.
Agents are already starting to enter data, development, support, security, finance, and operations. Avoiding them is not a technical strategy. Preventing an experimental flow from acting as authority is. In a well-governed operating room, each instrument has purpose, sterilization, responsible party, and protocol. In a well-governed platform, each agent must have scope, permissions, log, testing, and stop button.
First classify your agents by level of possible damage, then separate proposal from execution in critical processes, then review NIST, OWASP, and AWS controls to convert them into a technical checklist, finally practice with a small case where the agent only proposes changes and a person approves before touching production. The scalpel can be ready on the table. The signature is still human.
What minimum control would you demand before allowing an agent to touch data, code, or productive infrastructure in your company? 🧠