You've Installed a Pile of Agents — Which Ones Are Actually Useful?

Updated 2026-10-02: removed specific headcounts and the personal-checkup narrative that lacked original records; kept the self-check method, added a recent acceptance case, and corrected the fixed scores and call thresholds.

How a role list turns into actual work

Topic selection, research, writing, review, layout, publishing — you can write a role definition for each. A complete list is easy to search; whether it can catch the work still has to be seen in a concrete task.

For example, turning an official update into an article: who reads the source, which claims need verifying, who gets the draft, and what proves it's done? Leave these unclear, and no matter how professional the role name, whoever picks it up has to guess again.

Ask four short questions first: when do you call it? What do you give it? What does it hand back? What proves the task is done?

This article organizes a self-check method along those lines. First state the working contract clearly, then see whether the current conditions hold and how far the real delivery covers, and finally judge whether this division of labor is worth keeping.

First state the working contract clearly

Before an Agent takes a task, make six things clear. A small task can be written briefly; what matters is that whoever picks it up can act and accept on that basis.

1. Trigger: which task's appearance should call it.
2. Input contract: which files, fields or context it must receive.
3. Output contract: whether it delivers a suggestion, Markdown, JSON, a screenshot, or a platform status.
4. Acceptance evidence: who judges it done, and by what result.
5. Failure exit: when material is insufficient or a tool fails, when to switch methods, when to hand back, and what completed material to leave.
6. Permission boundary: whether it can read, write, go online, or take external actions.

The six being filled in only means the working contract is complete. If something's missing, judge whether it affects the current task — you can't rate a role's ability by how many boxes are ticked.

The model producing fluent text doesn't mean the flow moved forward a step. You still have to check whether that output meets the current task and whether the next stage can catch it.

How detailed a role card should be

Taking "article source verification" as an example, a workable role card can read like this:

Trigger: the draft contains a verifiable product capability, price, version or data point.

Inputs: the original URL, the claims to verify, the article's target reader, the deadline.

Outputs: a fact table, each entry containing the original source, the page date, the level of support and the still-uncertain parts.

Completion evidence: every public fact traces back to an upstream page; anything that can't be confirmed is marked to-verify, not filled in as a confident statement.

Failure exit: on a read failure, try a suitable alternative or the upstream source; if the full original still can't be obtained, mark the related conclusion unverified and continue work that doesn't depend on it. Don't reconstruct the body from a search summary.

Permission boundary: read-only browsing of public pages; no account login, no payment, no publishing.

Compare "you are a world-class growth expert, give me your most insightful advice" — it has only an identity, no working contract. That description itself says nothing about what the next stage needs.

How to check existing roles

Check in five steps, then decide to keep, adjust or shelve.

Step 1: back-check real calls from recently finished tasks

Find its actual inputs and outputs in recent tasks. With no call record, mark it "not yet verified" first — don't judge it capable from the role's self-introduction.

Here, separate "no value" from "no current position." The latter can be archived, restored when a real need appears; it doesn't need permanent deletion.

Step 2: put duplicate roles on the same table

As long as the trigger, inputs and outputs are highly similar, treat them as one position first. "Article reviewer," "content QA officer" and "senior editor" may just be three tones, not necessarily three jobs.

When merging, keep the clearest input/output contract, not the most impressive name.

Step 3: give the role a shadow run

First confirm this round's conditions hold: can the needed tools be called, are files or accounts accessible, do the version and permissions apply. A role file existing only proves the definition was saved — it can't replace an actual call. A "read-only" behavior clause doesn't automatically prove the run environment forbids writes.

A shadow run means: have the Agent redo a historical task that's already finished with a known result, but don't let it trigger external actions.

In the trial, compare three things:

• Whether it found the real blocker from back then.
• Whether its output can be used directly by the next step.
• Whether, on failure, it honestly states the gap and leaves material that can be continued.

One good performance doesn't prove stability. Choose samples by use and risk, covering full material, insufficient material, tool failure and environment change — don't treat a fixed count as a uniform threshold. Keep the known answer with the acceptor, to avoid mistaking a recitation of the result for ability.

Step 4: decide the handling by the real gap

Based on the real gap, record a concrete handling decision:

• Clear working contract, useful output: keep this division of labor, and keep observing it with real tasks.
• Missing a key source, tool or authorization: fill it in before doing the parts that depend on it.
• Duplicate responsibility, handoffs adding rework: merge the responsibility, or have the main assistant handle it directly.
• No real task right now: keep it as on-demand reference; don't declare it valueless just because it isn't called for now.

Missing acceptance evidence can't be offset by the other five being filled. This is an organizing method; the point is to leave a concrete reason, not to create a new certification tier.

Step 5: strip external permissions out of the position

Research, generation and checking can happen in sequence; login, billing, pushing, deploying and publishing can't gain authorization automatically because the previous step went well.

An Agent can prepare a publish package, but "preparation done" and "already published" are two states. An existing valid authorization keeps applying within its original target, action and task scope; if it has a time limit, it must still be within it. When the scope widens or authorization is missing, ask again, and after executing, read the result back from the target system. When a send or charge result is unclear, reconcile the status first before deciding whether to retry, to avoid duplicate execution.

A division of labor that can add rework

Say the task is "turn one official update into a beginner article."

If you have a researcher, analyst, writer, reviewer and director speak in turn, with no independent questions and no clear handoff, three things can happen:

• Five roles repeatedly summarize the same source.
• The later roles don't know which sentence from earlier is a fact and which is inference.
• A human still has to re-organize it into a publishable file.

One chain where the handoff can be checked:

original source
  -> fact-verification Agent: outputs claims.json
  -> platform Writer: outputs article.md
  -> read-only QA: outputs a problem list
  -> a human decides whether to fix and publish

This is one example division of labor for this scenario; you can't set from it that every task needs three roles. When each step has a checkable intermediate artifact, responsibility and handoff are easy to state clearly; a simple task can also be done by one assistant.

One real acceptance, and how far it proves

On October 1, Zhike ran a round of authorization tests on a physical Mac and a Windows Server cloud host — all 19 checks, including activation, device limit, re-issue and revocation, passed.

The same acceptance report also recorded a problem: a fresh Windows environment lacked a runtime library, an ordinary user's first install didn't finish directly, and the test only continued after an admin handled the dependency.

So what this round proves is the authorization function's result in the specified environment. It doesn't prove an ordinary user can double-click-install and use it on Win10 or Win11, and it doesn't cover the full review business. This revision cites that record; it didn't re-run the product test.

This makes the old article's "acceptance evidence" more concrete: when handing back a result, state which part is done, which isn't covered, and who takes the next step. Another Agent replying "passed" can't replace this evidence.

The checkup also looks at total cost

Call cost is just the most visible item. Several roles discussing each other adds call fees, and there are these other costs too.

Cognitive cost: having to think about who to call before every task starts.

Maintenance cost: the same rule copied into several roles means one update has to change many places.

Evidence cost: output with no structure can only be re-read and judged by a human.

Permission cost: handing an Agent unnecessary account and system permissions to save confirmations.

In a concrete postmortem, you can also note waiting, re-reading and human patching: did the new role reduce omissions, or bring more organizing work? Fill in time and cost only if there's a record; otherwise write untested.

An extra role may also be there to cover a necessary check, worth keeping even with added cost. Weigh the benefit and the cost together. This round has no single-vs-multi-Agent comparison record, so I can't claim the revised method is already faster or cheaper.

When you should add a second Agent

Whether to add a second Agent, first look for a clear "handoff surface":

1. A subtask can be answered independently, or an important conclusion needs separate verification.
2. The intermediate artifact can be saved independently.
3. The two steps need different judgment criteria.
4. After splitting, failures can be located faster.
5. The new role doesn't need a larger default permission.

A one-off task can meet these conditions too; you don't have to wait for it to repeat three times. Repetition just makes it easier to keep observing whether the division is worth keeping.

For example, "write the article" and "check mobile layout" suit splitting, because their inputs and outputs differ and the latter can stay read-only. Conversely, "senior writer" and "chief writer" usually have no verifiable handoff surface.

Finally, look at how the work connects

To judge whether this arrangement is useful, look at:

• Where a task enters.
• What file each step leaves.
• Who can change the content, who can only check.
• When evidence is missing, whether it can mark the gap, stop an unfounded judgment, and continue the other doable parts.
• Which status comes from local, which from an external platform.

Role definitions you don't need for now can stay as reference; a role in the current task has to state which step it owns and what it handed back.

If you also have a long list of Agents, pick one real task, write out the trigger, inputs, outputs, evidence, failure exit and permissions, then check this round's conditions against the real result. When a trial is needed, choose a representative scenario. Finally ask: given this delivery, how far can I confidently proceed?