Pay

A score with reasoning behind it is a different product

The same observations, organised into defined axes, sell for more than the same observations written as prose.

By Updated 5 min readPay

Guides on Pay: Pricing, from first principles to the annual review, The ceiling is hours, and it arrives sooner than people expect, What rating work actually pays

Buyers pay more for a scored assessment - defined axes with a written reason under each - than for the same observations delivered as prose, and the scored version is usually quicker to produce. Two earners can notice the same four things about a submission and be paid very differently for saying so.

That is the whole argument, and the reason it holds has almost nothing to do with the scored version being better assessment.

Why the structured version prices higher

A buyer commissioning an assessment is buying a reduction in uncertainty, and they cannot check the quality of the judgement before they pay for it. So they check the things they can check. A listing that says "written feedback" leaves them guessing at length, coverage and whether the person will actually address the thing they cared about. A listing that names six axes tells them exactly what arrives.

This is the standard result about markets where quality is unobservable before purchase: buyers substitute whatever observable signal is available. Akerlof's 1970 paper on the market for lemons is the canonical statement of the problem - the 2001 Nobel economics prize announcement credits it with showing how such a market "can contract into an adverse selection of low-quality products" - and the practical consequence here is unglamorous. Naming your axes is not a quality improvement. It is the removal of a reason to hesitate.

There is a second effect on top of it. A prose review is compared to other prose reviews on length, which is the worst axis you can be compared on, because length is the one thing a buyer can measure and the one thing you should not be selling more of. A scored review is compared on coverage, and coverage is cheap for you to widen and expensive for a competitor to fake.

The time argument, which matters more

The pricing effect is real and modest. The production effect is larger and almost nobody mentions it.

An unstructured assessment requires you to decide, every single time, what to talk about and in what order. That decision is genuinely the slow part. It is not the writing; it is the two or three minutes of staring before the writing, repeated at every job, forever.

Defined axes remove that decision permanently. You are no longer answering "what should I say" but "what do I say about axis three", which is a smaller question with a faster answer.

Unstructured prose Scored to defined axes
Deciding what to cover Every job Once, when you built it
Time per job Higher, and variable Lower, and stable
Consistency across jobs Depends on your day Structural
What the buyer can verify before booking Almost nothing Coverage and format
Risk of scope drift High Low, the axes are the scope

Stable time per job is worth more than the average being low. Per-job pricing means variance is your problem rather than the buyer's, which is the argument set out in the piece on choosing between an hourly rate and a price per job, and structure is the cheapest variance reduction available.

The axes themselves, how many to have and how to word them so they do not collapse into each other, are a separate build job covered in the post on constructing a reusable rubric. This post is only about which of the two products to sell.

What the score is not

A number on its own is worth very little, and the trap is thinking the score is the product.

Automated scoring already produces a number instantly and free, so a bare number from a person competes with a bare number from a model and loses on both price and speed. The technical account of how automated scoring reaches its output is the clearest description of what you are competing with, and the honest reading of it is that the number is the commodity part.

What a person sells is the sentence under the number. Why this axis scored where it did, on this specific submission, in terms that could not apply to anything else. Buyers reading a score with no reasoning attached do the sensible thing and treat it as noise, which is roughly the conclusion of the buyer-side literature on how scores are actually interpreted.

So the sellable product is not "scored" and not "written". It is scored with the written part carrying the weight, and priced as the composite.

Where the two formats actually belong on a card

Most earners should not choose. They should sell the structured version as the standard tier and keep prose as a deliberate, higher-priced option for the small number of buyers who want a considered essay rather than a grid.

That ordering surprises people, because prose feels like the cheap default and the grid feels like the premium product. On production time it is the other way round: the essay is slower, less repeatable and harder to scope, and if you are going to carry that you should be paid for it.

Do not offer the same content at two prices with different formatting, though. That is a discount wearing a costume, and it does the same damage to a card as any other discount, which the piece on protecting your rate card while still discounting works through in detail.

The tiers have to differ in what is actually delivered. Six axes with two sentences each is one product. Six axes plus a written overall reading that connects them is a second, genuinely more expensive product, and buyers can see the difference on the page.

One structural warning about specificity: the more precise your axes sound, the more some buyers will read them as measurements rather than judgements. Measurement is a separate discipline with its own conventions, documented in full elsewhere, and borrowing its vocabulary without its method is how an assessment starts sounding like a claim you cannot support. Keep the axes evaluative, keep the language yours, and the line stays clear.

If you want to see how a live marketplace presents both formats side by side and what each is listed at, the judges directory on Rate Cock shows current listings with their tiers visible.

Read next

Full archive