The Hidden Risk in AI Delegation

AI delegation is having a moment. But we should be careful with one potentially misleading assumption: that delegating to an AI is “basically like” delegating to a person.

A recent paper on Intelligent AI Delegation by Google DeepMind (see link below) frames delegation with human concepts: roles, responsibility, accountability, trust, reputation, escalation.

I like the direction. At the same time, we should remember what an AI agent actually is: a PROBABILISTIC OPTIMIZER WITH TOOL ACCESS. Meaning such an approach contains risks that should be addressed:

1) “Authority” isn’t social for agents, it’s technical

For humans, authority is enforced by norms and institutions.
For agents, “authority” is permissions, credentials, tool access. If we don’t translate the metaphor into controls (least privilege, revocation, sandboxing), we risk safety-by-storytelling.

2) “Trust” and “reputation” are gameable by default

In an agent ecosystem, identities are cheap, agents are copyable, and “good behavior” can be selectively shown to measurement channels. Trust needs identity + attestation to be operational, not aspirational.

3) Monitoring won’t catch the many small failures

Many failures are silent: plausible-but-wrong outputs, subtle drift, partial tool misuse. Trigger-based “detect → diagnose → re-delegate” can overestimate what’s actually observable.

So what’s the way forward? A hybrid approach:

Human-style governance, machine-style control.

- Use human delegation concepts to assign human responsibility: define who is accountable and who can approve/stop.

- Use security engineering to control agents: scoped permissions, isolation, provenance, rate limits, kill switches, and lean on “verification” only in places where it can genuinely prove something.

If we get this right, delegation becomes scalable. If we get it wrong, we create potential industrial-scale mishaps.

https://lnkd.in/ez5a3STx

#AI #AIAgents #SafetyEngineering #Governance #AgenticSystems