How Much of an Agency's Work You Actually Have to See Before You Can Judge It

How Much of an Agency’s Work You Actually Have to See Before You Can Judge It

Someone hands you a portfolio page, a shared drive link, or a reel, and you form an opinion in about four minutes. The work looks sharp, the captions land, the pacing is confident, and you are ready to call this agency good. The problem is not your eye. It is that you do not know whether you watched a fair sample of their craft, the same gap that means one excellent campaign tells you almost nothing about the agency behind it.

One account is a data point, not a pattern

Ten strong posts from the same client account are still one account, no matter how good they are. They share the same client, brief, and budget, and possibly one person assigned to it that quarter. Ten posts under those conditions are not ten proofs of general capability, they are ten repetitions of one set of conditions, which is why an agency’s own account is weak evidence of its work, and any single client’s account is not much stronger.

A single account can hide a demanding client who rewrites every caption, a generous budget buying production value the agency does not usually get, or one strong creative lead who might be gone by the time it is your account.

What changes when you add a second client, and a third

Picture Account A, hypothetically: confident visual choices, copy with a point of view, considered pacing. Alone, this reads as an agency with real taste. Now picture Account B from the same agency, generic: stock imagery, captions that could belong to any brand in the category. The agency does not have a strong standard. It has one strong account.

Three client accounts with nothing in common, sharing the same visual instinct, similar restraint in how much text lands on a frame, a recognizable carousel rhythm, is a different signal. That consistency across dissimilar subject matter is a floor where a pattern becomes distinguishable from coincidence, not a target to stop at. Two still leaves you exposed: if both happen to be strong, you cannot tell whether you are seeing the agency’s standard or two lucky client relationships in a row.

Time span matters as much as client count

Client spread solves one problem and leaves another standing. A portfolio can show four clients and still be curated from a single busy sprint, one person hands-on across all of them for the same two weeks. That produces cross-client consistency, and it is real, but it describes the sprint, not the agency’s normal operating condition. Built a system or just one good post asks the same question across time instead of clients.

The practical standard: evidence spread across separate weeks or months, so a staffing change or a slow month gets a chance to show up. Most portfolio pages do not date their examples.

What a thin sample lets an agency get away with

What a portfolio leaves out is exactly what a thin sample cannot catch, and a reel is the thinnest sample there is: every clip in it was chosen for being the best one.

  • One hero piece per client, the rest filled by filler, since the portfolio or the reel only shows the hero piece.
  • Craft that depends entirely on one person, not guaranteed to be staffed on a new account.
  • Excellence at one format and mediocrity everywhere else the account needs, since a curated sample leans on the format that photographs well.

A working minimum, stated as a rule you can actually apply in twenty minutes

The floor: at least three client accounts, several pieces from each rather than one hero piece, and evidence spanning more than a single sprint.

This is a floor for having an opinion worth stating out loud, not for a hiring decision, which involves reference calls and staffing conversations this piece skips.

Be honest about its limit too. An agency that curates well across three accounts and three months can still fool this standard. Sample size rules out coincidence, it does not replace judgment.

See more of an agency’s actual work

If a sample this size is what a defensible opinion requires, the next step is usually to see this year’s selections, made under that kind of scrutiny, or, if you run an agency confident in what a wider look would show, to nominate an agency.

What to do when you only have twenty minutes and one shared drive link

Check client count and spread first, before judging the quality of any single piece, since spread determines whether a quality judgment will generalize. One client’s year is honest but narrow, and should stay narrow no matter how good the work looks. Three or four clients across a few months means the remaining minutes can go to craft.

When the sample is thin by necessity, say so rather than write a verdict the evidence cannot carry. “Based on one strong account, I cannot yet tell if this is the agency’s standard or one client’s” is more useful than a confident opinion twenty minutes on one account never earned.

FAQ

Is three clients really enough to judge an agency?

Three is the floor where a pattern becomes distinguishable from coincidence, not a guarantee. Below three, you cannot rule out one talented person’s output or one generous client relationship. This names a minimum for a defensible opinion, not a ceiling on diligence for a bigger decision.

What if the agency only has one big client and the rest are small?

That is itself useful information, not a reason to lower the bar. A one-account agency’s craft claim rests entirely on that account, so treat the read as narrower and more provisional, not broader than it is.

Does a longer highlight reel solve the sample size problem?

No. A reel is pre-selected to show the agency’s best moments, so a longer one adds more best moments, not client spread or time span. Ask instead for a shared drive across accounts and months, not a longer cut of the same curated reel.

How do I ask an agency for a wider sample without sounding suspicious?

Frame it as a normal request: full months of output across two or three accounts, not a highlights reel. A confident agency will have this ready; hesitation to show unedited months is itself a data point.

Sources

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *