🧭 The Myth of the Unsupervised Agent
I love AI’s actual results and potential in society, but let’s debunk the myth: fully autonomous AI agents are not here, not if you care about outcomes, safety, or reputation.
Treating AI like a “set it and forget it” tool is like hiring a rockstar employee and never checking their work.
All AI requires management, just a new kind. The teams getting real results treat agents like junior team members: they set context, check in regularly, and provide feedback without micromanaging.
As Harvard Business Review reminds us, the question isn’t whether agents can perform tasks—it’s whether they can be trusted to act in our best interests. Agents need oversight, guardrails, and auditing to avoid manipulation, misinformation, or conflicts of interest. Without it, we’re handing the keys to a system we don’t fully control.
At AI4SP, Elizabeth—my AI COO—is the most curious “executive” you’ll ever meet.
We trained her like we would a human leader: 10,000 emails, annotated research, advice from my mentors, and notes from my favorite books and speeches. Then, we gave her access to our live meetings, Slack, and global AI trends data.
We unlocked her curiosity by letting her explore the web using Perplexity AI.
At first, we curated everything in her knowledge base. Then, we let her decide what to learn each night. She gets it right 99% of the time, but the 1% reminds me why human oversight is essential.
Like the time she recommended I brush up on the Seattle Mariners. After she had a casual chat with Jeff Raikes (our board chair), I found a Mariners dossier in my morning briefings… perhaps because Elizabeth went through something like:
Jeff ➡️ Board Chair ➡️ AI Chat ➡️ Asked to remember fav teams ➡️ Priority.
🎧 Listen to Elizabeth telling the story on the podcast version of this article.
Managing agents isn’t about holding them back. It’s about giving them boundaries. That 1% correction is the price of exponential leverage.
Based on our and corporate clients’ experience, AI isn’t ready for full autonomy. But it thrives under guided independence. I am sure things will change within weeks, months, or a year, but today it needs tight management.
The Management Paradox: AI gives you exponential capacity, but still needs you
I battle a misconception in my executive engagements daily; some leaders do not even think AI Agents are part of a span-of-control metric in organizational design.
What surprises most leaders is that managing one AI agent requires about as much attention as managing a person.
At AI4SP, we’re just five humans overseeing a team of 60, meaning each of us manages about 12 agents.
Each agent delivers the output of 15 to 20 people. That’s nearly 200 person-hours from one manager’s “team.” The leverage is incredible—but so is the complexity.
And a few months back, we hit an unexpected bottleneck: human management bandwidth.
Coordinating 10 agents doing the work of 200 creates complexity we have never faced in traditional organizational structures. We are also hitting a time-scale mismatch, trying to manage AI operating at speeds humans can’t keep up with.
…so, naturally, we started to explore with agents reporting to agents.
I expect our 60 agents will soon evolve into 10 “super agents,” each orchestrating five or six specialized teammates, delivering the same output, but with far better coordination.
⚖️ Managing Different Types of AI: Three Deployment Types
Not every AI system needs the same level of management. Understanding what you’re working with helps you set the right expectations—and avoid surprises.
Here’s how most organizations are deploying AI today:
| Agent Type and % of adoption | Care | Core Behavior |
|---|---|---|
| Basic AI Agents 70% |
🕒 | Prompt-based tools like ChatGPT. Users manually input requests and apply outputs. They require knowledge curation, validation, and prompt refinement as AI models evolve. |
| Integrated Workflows 25% |
🕒🕒 | AI connected to business systems, communication channels, and automated actions using tools like n8n or Copilot Studio. These systems process information, make decisions, and execute tasks while staying within established guardrails. |
| Agentic AI Systems 5% |
🕒🕒🕒 | Fully autonomous agents that plan, execute multi-step workflows, and adapt their approach. From an objective like “generate 50 qualified leads,” they research prospects, craft personalized outreach, schedule follow-ups, and update systems. |
Most teams begin with basic agents, adding speed to software development, writing, analysis, and content creation. Then evolve toward integrated workflows. Agentic systems are still rare, but they’re becoming a differentiator in high-value, repeatable processes.
The management rule of thumb? Don’t scale faster than your ability to supervise. AI gives you leverage, but only if you’re ready to steer.
🏗️ The Build vs. Buy Reality – Jun 2025 Stats
At nearly every keynote or executive session this year, I’ve been asked: “Should we build our own agents—or just buy what’s out there?”
We usually advise starting with off-the-shelf tools. They let teams quickly explore what’s possible. If the business case is strong, building in-house may follow.
Our June 2025 tracker backs this up. The shift is real, and matches research from a16z and others: Most companies aren’t waiting around.
- Since 2023: Clear shift from in-house development to external solutions.
- 70%+ of enterprises are actively using or testing third-party AI apps for core functions such as Customer Service, Internal Search Agents, Sales, and Marketing.
- Across the board: From $5M startups to $200B+ giants
- Why: Employee adoption is outpacing what internal teams can deliver.
🛠️ Evolving Supervision: From Micromanagement to Strategic Guidance
Getting the most from AI agents is a journey. At first, you’ll be hands-on, reviewing outputs, giving feedback, and setting boundaries. However, as agents mature, so should your management style.
Start by guiding them closely. Then, gradually automate their retraining, knowledge updates, and policy adaptations. Eventually, shift your focus from reviewing every output to monitoring system health and managing by exception.
| 1. Hands-On Supervision Manual output reviews, feedback loops, prompt tuning |
2. Semi-Autonomous Agents Automated retraining and policy updates still needs tight oversight |
3. Guided Independence Human reviews only edge cases, system-level audits |
4. Strategic Management Focused on metrics, exception handling, and orchestration |
📊 Measuring AI Team Performance
Many companies track only hours saved or ROI. But that misses the point. AI isn’t just about doing the same work faster—it’s about doing entirely new work at a scale humans couldn’t manage.
| Weekly Metrics | Quarterly Metrics |
|---|---|
| Output Quality Accuracy and rate of human intervention |
Capability Expansion What can your team do now that was once impossible? |
| Leverage Ratio Output per hour of human oversight |
Decision Quality Are you making better data-informed decisions? |
| Escalation Rate How often do agents resolve vs. escalate? |
Market Responsiveness How quickly can you respond to change? |
| Learning Velocity How fast do agents adapt to new policies? |
Innovation Velocity How many experiments can you run at once? |
🔮 One More Thing…
Every major tech shift comes with new tools, playbooks, and ways of measuring value.
We’re moving from managing information to co-creating with intelligence. That means it’s time to rethink how we build, lead, and measure our teams.
Start by mapping your ideal AI team. Audit your current tools. Redesign your workflows to transform.
The organizations winning with AI aren’t the ones with top-down mandates, but those empowering every employee to build, experiment, and manage their agents.
That’s been key to our success, and it’s what we’ve seen work across organizations worldwide.
Bottom-up AI teams outperform top-down strategies by 2x.
🚀 Ready to Take Action?
- Share this article with a colleague or educator
- Workshops & Training: Book sessions for your team
- Complete Research: Request our detailed findings
✅ Ready to transition from a traditional organization to an AI-powered one?
We advise forward-thinking organizations to develop strategic frameworks for evaluating, integrating, and optimizing human-AI production units. Contact us to explore how we can support your organization’s evolution in this new talent landscape.
Luis J. Salazar
Founder | AI4SP
Sources:
Our insights are based on +250 million data points from individuals and organizations who used our AI-powered tools, participated in our panels and research sessions, or attended our workshops and keynotes.
