A mixed team of employees reviewing the work of AI agents on shared screens

Managing AI Agents and People: Roles, Accountability and Skills

How to run a team where people and AI agents share the work: redesign roles around tasks, name an accountable owner for each agent, measure outcomes and rebuild skills and career paths.

By Shayan Salehi 8 min read

Image: ColdAI

Managing a team where people and AI agents share the work means treating each agent as a delegated worker with a named human owner. Redesign roles around tasks, not job titles. Make one person accountable for each agent's results. Measure outcomes and quality, not activity. And rebuild skills and career ladders, because the evidence shows AI helps some workers far more than others.

Start from tasks, not job titles

Most role descriptions list a title and a few duties. Agents do not fit that format, because they take on parts of many jobs. A support agent drafts replies, an operations agent reconciles records, and a research agent assembles briefs. Each takes a slice of a person's week.

The World Economic Forum's Future of Jobs Report 2025 gives a sense of the direction. Employers surveyed estimated that today 47% of work tasks are performed mainly by humans alone, 22% mainly by technology and 30% by a combination of both. By 2030 they expect these shares to be nearly evenly split. The report adds that the shift is driven mostly by automation, with expanded human-machine collaboration accounting for about 19% of the change in the human share.

That makes the task the right unit of design. List the tasks in a role, then mark each one as human only, agent only, or shared. For shared tasks, write down who drafts, who checks and who decides. The result is a task map, and it should be reviewed whenever a new agent goes live. A deeper method for exposure analysis exists elsewhere on this site, so here we focus on what you do with the map.

Know where AI helps and where it misleads

Two field studies explain why a uniform rollout goes wrong.

Brynjolfsson, Li and Raymond studied a generative AI assistant introduced in stages to 5,179 customer support agents, in work published in the Quarterly Journal of Economics. Access raised issues resolved per hour by about 14% on average. Novice and low-skilled workers improved by about 34%, while experienced and highly skilled workers saw minimal impact. The authors also report improved customer sentiment and higher employee retention.

Dell'Acqua and colleagues ran a pre-registered experiment with 758 consultants at Boston Consulting Group. On tasks inside what they call the jagged technological frontier, consultants using AI completed 12.2% more tasks, finished them 25.1% faster and produced work rated more than 40% higher in quality. Consultants below the average performance threshold gained 43% and those above gained 17%. On a task outside the frontier, consultants using AI were 19 percentage points less likely to produce correct solutions than those without AI.

Read together, the studies carry three lessons for a manager. Gains are uneven, so you cannot assume the same effect on every team member. The boundary between tasks AI handles well and tasks it handles badly is hard to see from the outside. And the same tool that lifts a junior colleague can lead a confident colleague astray on the wrong task. Your task map needs a column for known weak spots, kept current with real examples from your own team.

Assign accountability before you assign agents

When an agent makes a mistake, the first question is who owns it. An OECD study cited in a 2025 G7 compendium found that 28% of managers reported unclear accountability in cases where algorithmic management tools make a wrong decision, and 27% pointed to a lack of explainability. Ambiguity is the norm unless you remove it.

Three roles settle most of it.

  • Agent owner: a named manager who is accountable for the agent's outputs, cost and incidents, in the same way as for a team member's results.
  • Reviewer: the person who checks defined outputs and who has the authority to reject them. The EU AI Act's Article 26(2) says deployers of high-risk systems must assign human oversight to natural persons with the necessary competence, training and authority. That is a sound test even where the Act does not apply.
  • Process owner: the leader of the end-to-end workflow, who decides what the agent may do and when it is switched off.

NIST's AI Risk Management Framework 1.0 supports this structure. GOVERN 2.1 calls for roles, responsibilities and lines of communication for managing AI risk to be documented and clear throughout the organization, and GOVERN 2.3 says executive leadership takes responsibility for decisions about AI risk. Put the agent owner on the org chart and in the performance review, not only in a register.

Keep one rule above the rest: an agent never holds authority that no human in the organization holds. If a manager cannot approve a payment of a certain size, neither can their agent.

Measure outcomes, not usage

Usage metrics such as prompts sent or hours saved are easy to collect and easy to game. They also tell you nothing about quality. The OECD's Employment Outlook 2023, drawing on its surveys of employers and workers in finance and manufacturing, found that over 80% of workers who work with AI report higher job performance. Self-reported gains are a useful signal, but they are not a management measure.

Build a small scorecard for each mixed team, at the level of the workflow.

  • Outcome: cycle time, cost per completed unit and customer or stakeholder results, compared with the pre-agent baseline.
  • Quality: error rate on a sampled and checked set of outputs, plus rework and escalation rates.
  • Human contribution: share of cases where a reviewer changed the outcome, and what they changed.
  • Load: volume of work per person and after-hours work. The OECD found that AI adoption is associated in some cases with higher work intensity, so watch for faster work that simply means more work.
  • Learning: how many people can perform the task without the agent.

Evaluate individuals on how well they direct, check and improve agent work, and on the outcomes of the team. Do not rank people on volume against an agent, which is a contest nobody should want to win.

Build skills where the work is changing

The same WEF report finds that 63% of employers see skill gaps as a major barrier to business transformation over 2025 to 2030, and 85% plan to prioritize upskilling. It also estimates that if the world's workforce were 100 people, 59 would need training by 2030. Employers foresee that 29 could be upskilled in their current roles and 19 upskilled and redeployed, while 11 would be unlikely to receive the training they need.

The OECD adds a warning for managers. It notes that managers need to be able to critically evaluate and overrule AI-powered recommendations, yet are likely to lack the AI knowledge to do so. Training for reviewers and team leaders is therefore not optional. Cover what the agent does, where it is known to fail, how to read its output skeptically and how to escalate.

Three skills deserve explicit attention: task decomposition, so that people can split work between themselves and agents; verification, so that they can check output against evidence; and exception handling, so that they can resolve the cases agents cannot. These are teachable and they can be assessed.

Rebuild career paths

If agents absorb entry-level tasks, the usual route to expertise narrows. The Brynjolfsson study suggests AI can spread the practices of more capable workers and help newer workers reach higher performance faster, which is an opportunity. It also means that junior staff who lean on the tool without understanding the work may not build the judgment they need later.

Design for that on purpose. Keep structured practice on tasks the agent can do, so people learn what good looks like. Create paths that reward the new work: agent owner, quality reviewer, workflow designer. Tie promotion to demonstrated judgment in review and exception handling, not to throughput.

Talk with employees early. Where the EU AI Act applies, Article 26(7) requires employers to inform workers' representatives and affected workers before putting a high-risk system into use at work. Even where it does not, early involvement is practical: the OECD found that workers using AI were more likely to report improved performance and working conditions when their companies consulted workers or their representatives, and 9 percentage points more likely to say AI improved their health and safety.

Run the team rhythm around the agent

A mixed team needs a few routines that a human-only team can skip. Hold a short weekly review of the agent's exceptions and corrections, led by the agent owner. Ask which errors repeated, which instructions were unclear and which tasks the agent should stop doing. Treat each repeated correction as a defect in the workflow, not as a reviewer failure.

Run a blameless review after any serious incident, in the same way you would for a human error. Record what the agent did, what the reviewer saw, what the process owner had decided in advance and what changes. This keeps accountability clear without turning every mistake into a personal matter, and it gives employees evidence that the agent is managed rather than trusted blindly.

Finally, plan for absence. Agents do not take leave, but their owners and reviewers do. Name deputies, and test that someone else can pause the agent, read its logs and take over its tasks.

Where to start

You do not need a reorganization to begin. Pick one team that already uses an agent and work through these steps. Start with the task map, because the other decisions depend on it.

  1. Map the team's tasks and mark each as human, agent or shared. Note where the agent is known to fail.
  1. Name an agent owner and reviewers, and write down what the agent may and may not do.
  1. Set a baseline for outcome, quality and load before changing anything, then add the scorecard to the team's regular review.
  1. Train reviewers and the team lead on the agent's failure modes and on the escalation path.
  1. Update job descriptions and performance criteria so that directing and checking agents counts as work.
  1. Review after one quarter: which tasks moved, who gained, who struggled and what that means for hiring and development.
Tags:managing-ai-agentshuman-ai-teamsrole-designaccountabilityperformance-managementupskillingcareer-paths

Your starting point

What would you like to move forward?

Start with the outcome that feels closest. We'll help give the next step a useful shape.

Let's find your next move

One conversation. A clearer direction.

Tell us what you want to improve. A few sentences are enough to start.

shayan@coldai.org