Judging three agencies’ work side by side changes what you notice
You have three agency portfolios open in three tabs, reviewing them one at a time: forming an opinion, then closing the tab. By the third portfolio you can tell you liked something about the first one better, but you cannot say what. The comparison you opened the tabs to make has not actually happened.
The tab-by-tab review quietly resets your standard each time
Reviewing a shortlist sequentially feels thorough, since each portfolio gets your full attention in turn. The problem is what happens to your sense of “good” between tabs. It drifts. This is recency bias: whatever you looked at most recently becomes the reference point for what counts as strong, overwriting the reference point the previous portfolio set, regardless of how the two actually compared.
Here is what that looks like on screen. You open tab one and something about the carousel sticks, though you do not stop to name it. Tab two is competent but unremarkable. Tab three leans harder into bold typography and reads as the most confident of the three simply because it is freshest in your head. Ask what specifically made tab one’s carousel work. Most reviewers cannot answer.
What side by side surfaces that sequential review does not
Put two pieces of comparable work in front of you at once and vague differences become specific. Picture two agencies each producing a product launch carousel for a similar client. Reviewed side by side, the difference in how each resolves the final slide becomes obvious: one closes with a single clear call to action that lands like a final beat, the other trails off into a fourth slide restating features with nowhere to land.
Side by side also surfaces convergence. If two portfolios lean on the same visual template or caption structure, you will not notice reviewing them a day apart, since you have forgotten the first one’s specifics by the time you reach the second. Next to each other, the resemblance is immediate, which says something about how much of what looked like a distinct point of view was actually a shared template.
Fix the comparison unit before you start: same format, same rough brief
A comparison is only a comparison if the things compared are the same kind of thing. Before judging any portfolio, decide the comparable unit across all candidates and hold every one to it.
A bad comparison unit: agency A’s flagship campaign film against agency B’s routine feed post. That compares which agency chose to show its ceiling versus its floor, not craft.
A good comparison unit: each agency’s most recent static carousel made for a client of roughly similar size and category. No showreel highlight is in the mix, so what gets judged is the work each agency produces on an ordinary week, the version of their craft a future client is most likely to receive.
A simple method for running the comparison itself
- Step one: view once, no notes. Look at all candidates’ comparison unit, one after another, without writing anything down, for an honest first reaction rather than analysis that anchors on whatever came first.
- Step two: name one distinguishing thing per candidate. Go back through in the same order and write, for each, the single thing it did that the others did not: how a transition lands, how a caption sets up a payoff, a compositional choice the others missed.
- Step three: rank only after every candidate has that note. Ranking before noting the distinguishing detail collapses the exercise back into a vague overall impression. The note comes first, and the ranking follows from the notes, not a gut call justified afterward.
Where comparative judging goes wrong
Comparing candidates against each other introduces failure modes judging one portfolio alone does not have. The first is the halo effect: a portfolio that impresses early earns the benefit of the doubt on everything after, so a weaker piece later in that same portfolio gets read generously.
The second is the contrast effect, the halo effect’s mirror image. A genuinely average piece can look good simply because it followed something worse. If a candidate’s work strikes you as strong, ask whether you would think so if it were the first thing you saw rather than the third.
The third is subtler: ranking on production polish rather than the craft decisions the review was meant to test. A slicker render can read as better work when it just demonstrates a bigger budget. Anchor the review to what the brief asked for, pacing, structure, the strength of an idea, how a piece resolves, not how expensive it looks.
Put the method to use
Before comparing anything, it helps to know what a portfolio leaves out and why one excellent campaign tells you almost nothing on its own. Two more are worth having in hand: what actually separates good social work from average, and what strategy looks like from the outside.
If you have run this comparison and think an agency’s work holds up, you can nominate an agency with that comparable unit in mind.
What this method cannot tell you
This is a method for judging the quality of the work itself, nothing more. It says nothing about running a hiring process, reference calls, contracts, or pricing. Those are real parts of choosing an agency, and a different kind of decision entirely. A portfolio that wins a comparative review on craft can still be the wrong agency for reasons that have nothing to do with craft.
It is also worth being honest about scale. A shortlist of three is a small sample. Comparative review sharpens what you notice within that sample, it does not make the sample bigger or more representative. Treat what you learn as a read on craft under the conditions you observed, not a guarantee about every piece of work that agency has made or will make.
FAQ
Is it better to review all three portfolios in one sitting or spread it out?
One sitting, for the comparison itself, so the same standard applies to all three at once. Spreading sessions across separate days reintroduces the drift this method exists to avoid, since your sense of good on Monday is not your sense of good on Thursday.
What if the three agencies don’t have any directly comparable work?
Then side by side review cannot do its job yet. Rather than forcing a comparison across mismatched formats, ask each agency for a piece of work made against a similar brief before comparing them. A fair comparison needs a fair comparison unit first.
Sources
- https://thesocialagencyawards.com/blog/what-a-portfolio-leaves-out/, checked 2026-09-09
- https://thesocialagencyawards.com/blog/one-campaign-is-not-a-body-of-work/, checked 2026-09-09
- https://thesocialagencyawards.com/blog/reading-strategy-from-public-output/, checked 2026-09-09
- https://thesocialagencyawards.com/blog/what-makes-social-media-work-good/, checked 2026-09-09
- https://thesocialagencyawards.com/nominate-your-agency/, checked 2026-09-09