Methodology

The Pragma Method

How Pragma identifies the problems worth working on, and how it develops policy to the point where a government could implement it.

Founding draft for discussion - version 0.4, June 2026. This method is versioned and open to challenge under section 9; see the version history at the end.

Why a method matters

Pragma's credibility does not rest on the cleverness of any individual proposal. It rests on the method that produced it. A transparent, repeatable method is what allows a stranger to trust the conclusion without trusting the author - and what allows the public to extend the work themselves rather than waiting for an institute to do it for them.

The method has two halves. The first identifies which problems Pragma should work on: specifically, problems where collective action through government can achieve something that markets and private capital, left to themselves, will not. The second develops a registered problem into implementation-ready policy. Supporting both are shared standards for evidence, non-partisanship, costing and independence.

A guiding distinction runs through everything below. Empirical questions - how large is a problem, what causes it, what would a given intervention do - can be resolved, or at least narrowed, by evidence. Value questions - how much weight to give the next generation, how to trade liberty against security, how much inequality is acceptable - cannot. Pragma's job is to settle the first kind as far as the evidence allows, and to surface the second kind honestly so that the public and their representatives can decide. Confusing the two - presenting a value choice as a technical necessity - is the most common way policy work misleads, and the method is built to prevent it.

Part A - Identifying the problems worth working on

A1. The principle

Pragma does not work on every problem in Britain. It works on the subset where government policy can make a tangible difference that markets and private capital alone cannot. This is a deliberate, disciplined narrowing. Many real problems are best solved by markets, by civil society, or by individuals, and Pragma should stay out of them. Some real problems have no credible government solution at all, and honesty requires saying so. The method's first job is to find the problems that are genuinely for government - and to be candid about the ones that are not.

A distinction must be drawn carefully here. Whether government can materially improve an outcome is settled on the evidence (Test 3). Whether it should use an available lever - where that bears on the proper scope of the state, and most sharply where a problem touches personal liberty, autonomy or conscience (for example fertility, religious observance, or dress) - is a value question, and the boundary of legitimate state action is itself politically contested. Pragma does not resolve that boundary in-house, and does not use it to screen problems out: to reject a real, market-insufficient, tractable problem on the ground that "the state should not intervene" would be to settle a contested value judgement by fiat - the mirror image of the error the Method exists to prevent, presenting a value choice as a technical necessity - and would quietly remove exactly the hard, contested problems Pragma is meant to confront. Such a problem is admitted; whether, and how far, the state should act is surfaced as a value question and routed to the public during development.

A2. The Four Gateway Tests

A candidate problem enters the Problem Register only if it passes all four tests. Each is recorded explicitly in the Register entry, with evidence, so the judgement can be challenged.

Test 1 - Materiality

Is the harm real, measurable and significant?

The problem must affect a large number of people, or impose severe harm on a smaller number, and it must be demonstrable with evidence rather than asserted. A candidate fails this test if the harm is speculative, trivial in scale, or already declining on its own. Materiality is assessed on two axes - the number of people affected and the depth of harm to each - because a problem can qualify on either.

Test 2 - Market insufficiency

Will markets and private capital, left to themselves, fail to resolve this - and exactly why?

This is the defining test. It is not enough to observe that a problem exists; the entry must identify why ordinary market forces will not clear it. If a competitive market, given time, would resolve the problem, government should usually let it. The entry must name the specific reason the market falls short, using the taxonomy in section A3. A candidate that cannot be placed in that taxonomy has probably not been diagnosed properly yet - and may not be a problem for Pragma at all.

Test 3 - Government tractability

Is there a credible lever by which government policy can materially improve the outcome?

A problem can be real, and beyond the market, and still not be solvable by government - because no policy instrument exists, because the state lacks the capacity to wield it, or because the cure would predictably be worse than the disease. This test forces honesty. If the best available answer is "government should care about this but cannot fix it", that belongs in the entry, and the problem does not proceed to policy development. Tractability is provisional: a problem may be parked here and revisited if a credible lever later emerges.

This test asks whether government can act, not whether it should. That second question - where it touches the proper scope of the state - is a value question, settled neither here nor in-house but surfaced and routed to the public during development (the principle is set out in A1; the machinery is Part B). A material, market-insufficient, tractable problem is admitted even where intervention is contested, with doing nothing always carried forward as a fully costed comparator (B2 Stage 3, B6) and the case tested from each political perspective (B5).

Test 4 - Persistence or neglect

Has this genuinely resisted resolution - recurring, or repeatedly attempted and failed, or left unaddressed?

Pragma's comparative advantage is the hard, stuck problems, not the ones already moving. A candidate passes if it has persisted across governments, been attempted repeatedly without success, or been allowed to drift with no serious attempt at all. This test also disciplines the institute against chasing headlines: a problem that is new, salient and already being acted on is usually better left to the existing process.

A candidate that passes all four tests is admitted to the Register. A candidate that fails any one is recorded as considered and not admitted, with the failing test and the reason stated - because the rejections are part of the evidence of the method working, and a rejected candidate may qualify later if circumstances change.

A3. The market-insufficiency taxonomy

Test 2 requires every registered problem to be classified by why the market will not resolve it. The categories below are the recognised ways in which private markets and private capital, acting rationally, still produce an outcome worse than the public could have. A problem often involves more than one; the entry names all that apply.

Public goods. The benefit is non-excludable and non-rival, so private actors cannot capture enough of the return to justify providing it, and it is under-provided. Examples: flood defences, basic research, public-health surveillance, clean air.
Externalities. A cost or benefit falls on people other than the decision-maker, so the decision-maker over-produces the harm or under-produces the good. Examples: pollution, congestion, the wider social value of a well-educated population, the herd benefit of vaccination.
Information asymmetry and failure. One side of a transaction cannot assess quality or risk, so good provision is not rewarded and bad provision is not punished. Examples: care-home quality, complex financial products, professional services bought once in a lifetime.
Market power and natural monopoly. High fixed costs or network effects mean competition cannot realistically discipline the provider, so price, quality or investment drift from the public interest. Examples: water and sewerage, the electricity grid, rail track, rural broadband.
Coordination failure. Everyone would be better off if all actors moved together, but no individual actor gains by moving first, so no one moves. Examples: employers under-investing in training because rivals can poach the trained worker; town-centre regeneration; technical standards.
Missing or incomplete markets. A market that would be socially valuable simply does not exist, because the risk cannot be priced, the contract cannot be written, or the horizon is too long. Examples: insurance against needing long-term care; patient capital for decades-long research; cover for catastrophic correlated risks.
Capital-horizon mismatch. The returns to an investment accrue over a horizon longer than private capital will wait, so a socially profitable investment is privately unattractive. Examples: preventive healthcare, early-years support, climate adaptation, long-lived infrastructure.
Behavioural failure. Systematic and predictable features of human decision-making - present bias, inertia, over-optimism - lead individuals to act against their own considered long-term interests. Examples: under-saving for retirement, under-insuring, under-investing in preventive health.
Distributional failure. The market may be efficient and still produce an outcome the public judges unjust, or may deny people the means to participate in it at all. This is not a market "failure" in the textbook sense; it is a legitimate, openly value-laden reason for collective action, and the method requires it to be labelled as such rather than disguised as technical necessity. Examples: child poverty, access to justice, the basic capabilities a person needs to take part in economic life.
Government-created distortion. Sometimes the market is not failing on its own - past government policy is the cause: a rule, a tax, a structure or a fragmentation that produces the bad outcome. Fixing the state's own mistakes is a legitimate and often high-return target. Examples: a planning system that suppresses housing supply; a tax structure with damaging cliff edges; fragmented institutions that cannot coordinate.

Categories 1-8 are the classic forms of market failure. Category 9 is an explicit, honest acknowledgement that some of the public's most important goals are about fairness rather than efficiency, and that this is a proper subject for democratic policy rather than something to be smuggled in under technical language. Category 10 keeps the method honest in the other direction: not every problem calls for more government - some call for government to undo its own past error.

A4. What Pragma deliberately does not take on

The method is as much about exclusion as inclusion. Pragma does not work on:

problems a competitive market would resolve on its own, given reasonable time;
problems with no credible government lever, however serious - though it may still publish the diagnosis, so the limits of policy are on the record;
questions that are purely matters of value, presented as though evidence could decide them;
issues framed primarily as cultural or identity conflict rather than as a tractable problem with measurable harm - Pragma will work on the measurable harm beneath such a framing, but not the framing itself;
party-political advantage, in any form.

A5. The Priority Score

Passing the gateway tests admits a problem to the Register. It does not say how urgent it is. Pragma ranks the Register with a transparent Priority Score so that scarce effort goes where it does most good - and so that anyone can see, and dispute, why one problem is ranked above another.

Each registered problem is scored 1-5 on seven dimensions:

Scale - how many people are affected.
Severity - how deep the harm is to those affected.
Cost of inaction - whether the problem compounds, entrenches or grows more expensive the longer it is left.
Tractability - how confident we are that a credible policy lever exists.
Deliverability - whether the state realistically has the capacity to execute a solution.
Cross-partisan viability - whether a solution could plausibly survive a change of government, rather than being unwound.
Time-criticality - whether there is a window that is closing.

The seven scores are summed to a Priority Score out of 35. The score is a communication and triage instrument, not a measurement of truth: it makes a set of judgements explicit and contestable, and it should never be cited as if it were precise. Two reasonable people will score a problem differently; the value is that they can see exactly where they differ. Scores and the reasoning behind each dimension are published with every entry, and may be revised as evidence improves.

A6. Systemic centrality: sequencing by leverage

The Priority Score measures how much a problem matters in isolation. It does not capture how much other problems depend on it. Some problems are largely self-contained; others sit upstream - resolving them relieves, unblocks or de-risks several other entries at once. This upstream leverage is systemic centrality, and it is a distinct consideration from the Priority Score.

Centrality matters for sequencing - which qualified problems Pragma develops first. A highly central problem can warrant earlier development than its raw score alone would suggest, because the benefit compounds across the Register: fixing a keystone eases the problems downstream of it. Adult social-care funding, for instance, is a principal driver of both local-government insolvency and of NHS delayed discharges, so it sits upstream of two higher-scored entries; the state's capacity to deliver is upstream of every proposal. A high-scoring but self-contained problem carries no such multiplier.

Two disciplines keep this honest:

Centrality is a sequencing overlay, not a re-scoring of importance. The Priority Score is left to measure each problem on its own terms; centrality is weighed alongside it when deciding the order of work, never folded into the score. The two are reported separately, so each can be disputed separately.
Sequencing is not implementation order. Pragma chooses what to develop first, by importance and leverage. The order in which solutions are actually enacted is for the public and their representatives to decide; the Method's job is to place the most consequential, highest-leverage options before them, costed and ready.

A7. Where candidate problems come from

The Register is built from, and continuously fed by, several streams:

Public nominations - anyone may nominate a problem, and nominations from people with direct experience of a problem are especially valued.
Unimplemented official recommendations - the recommendations of public inquiries, select committees, the National Audit Office and royal commissions that government has never acted on. These are, almost by definition, problems that have been diagnosed by serious people and then neglected. The sister project auto-wonk tracks exactly this - which recommendations were made and which were never implemented - and is a primary feed for the Register.
International outliers - areas where the UK performs markedly worse than comparable countries, which is prima facie evidence that better is possible.
Official statistics and independent research - the body of evidence published by the ONS, the IFS, the Resolution Foundation, the Institute for Government, the health think tanks and others, much of it discoverable through the open-government data catalogues.

No source is privileged. Every candidate, wherever it came from, faces the same four gateway tests.

Part B - From a problem to implementation-ready policy

B1. The principle

A good idea is not a policy. A policy is not finished when it is persuasive; it is finished when a competent government department could pick it up and execute it. The gap between those two states - the unglamorous work of mechanism, legislation, funding, delivery, sequencing and failure-planning - is exactly the work that interested funders rarely pay for, that a government consumed by day-to-day events has no room to do, and that Pragma therefore treats as its core product.

The aim of every product is to be shovel-ready: developed far enough that a government with no spare capacity to develop it itself can adopt it and move at speed. And because the prerequisite due diligence is done outside the pressure of the electoral cycle, the method is free to design for the horizon on which structural problems actually resolve - longer than a single Parliament - rather than the horizon on which they are usually managed.

B2. The development pipeline

Every Pragma policy product moves through nine stages. Each stage has a defined output and a gate that must be cleared before the next begins. Work can be sent back a stage; that is the method working, not failing.

Stage 0 - Nomination. A problem is proposed, from any of the streams in A7.

Stage 1 - Triage. The four gateway tests are applied and a Priority Score assigned. Output: a Register entry, or a recorded rejection with reasons.

Stage 2 - Diagnosis. The problem is fully worked up: its scale and evidence; its root causes, distinguished from its symptoms; its market-insufficiency classification; an honest account of why previous attempts failed; who bears the cost, and who currently benefits from the status quo; and a clear statement of what success would look like, expressed as measurable outcomes. Output: a problem diagnosis. Gate: the Evidence and Methodology Panel confirms the diagnosis is sound before any solution is designed - because a solution to a misdiagnosed problem is worse than none.

Stage 3 - Options generation. A slate of distinct options is generated, deliberately including the honest "do nothing" baseline and any relevant approach used in another country. No option is excluded because it is politically awkward for one side; suppressing options is how policy work becomes partisan. Output: an options paper with each option's mechanism, cost and likely effect.

Stage 4 - Design. One option, or a synthesis, is selected and fully specified: the mechanism; the legal instrument required; the delivery body; the funding source; the data and technology needed; the timeline; and how it interacts with existing institutions. Output: a policy design specification.

Stage 5 - Implementation-ready drafting. The design is developed into the full artefact set - the standard Pragma policy product - to the standard defined in B3. Output: the implementation-ready policy product.

Stage 6 - Adversarial review. The product is attacked, on purpose, by people who did not write it (section B5). Output: a published review, the author's response, and any required revisions.

Stage 7 - Publication and public deliberation. The product is published in full, with its evidence, its uncertainties and its dissent, and put to the public - through auto-wonk's deliberation layer - to scrutinise, support or reject. Output: a public proposal, versioned.

Stage 8 - Iteration. The proposal is revised in light of public feedback, expert challenge, pilot results, real-world evidence, and new government publications, inquiries and legislation (B8). Every version is retained and the changes between versions are documented, so the proposal's history can always be traced.

B3. What "implementation-ready" means

A Pragma policy product is not implementation-ready until it can answer, specifically, all of the following:

Legislation - what new law or amendment is required, or confirmation that none is.
Delivery body - which existing or new institution will run it, and whether it has the capacity.
Funding - the whole-life cost, the funding source, and who ultimately pays.
First hundred days - the concrete actions that would begin on day one.
Measures of success - the specific, published metrics by which it would be judged, and the independent evaluator who would judge it.
Sequencing and dependencies - what must happen first, and what it relies on.
Failure modes and exit conditions - how it could go wrong, the safeguards against each, and the conditions under which it should be stopped or reversed.

The National Employment Service documents - the worked product for Problem Register entry 12 - illustrate this artefact set: a white paper, an implementation and delivery plan, a public summary, a communications and political strategy, a one-page infographic, an evidence annex, a cost-benefit appraisal, a separate Mission-1 costing, a pilot design and a ministerial deck. Begun before this method existed, they have since been developed up to the standard above and are the Register's first complete worked example of Stages 2-6.

B4. Evidence standards

Every factual claim in a Pragma product carries a citation and an evidence grade. The grade describes the strength of the evidence, not whether Pragma agrees with it.

Grade A - robust causal evidence: well-conducted randomised trials, natural experiments, or findings replicated across several comparable countries.
Grade B - strong observational evidence: official statistics, large representative datasets, consistent findings without clean causal identification.
Grade C - weak or indirect evidence: expert judgement, economic theory, a single case, or modelling resting on contestable assumptions.
Grade D - contested or absent: claims where the evidence is genuinely disputed, or where there is none and judgement is doing the work.

A proposal may rest on Grade C or D evidence - much necessary policy must - but it must say so, in the open, so the reader can weigh it. Hiding a weak foundation is a failure of the method. Wherever possible Pragma uses official statistics and data published under the Open Government Licence, so that any reader can return to the same source.

B5. Non-partisanship as a method

Pragma does not ask its people to be free of political views; that is impossible and pretending otherwise is itself a deception. It builds non-partisanship into the process instead.

Multi-perspective stress test. At Stage 6, every product is reviewed from at least three distinct political perspectives - a fiscal-conservative lens, a social-democratic lens and a libertarian lens. Findings that survive all three are reported as robust. Findings that one perspective rejects are reported with the disagreement shown, not quietly dropped and not quietly kept. This mirrors the analytic approach of the sister project auto-wonk, so that the proposing institute and the scrutinising platform share one standard of fairness.
Named author, named adversary. Nothing is published without a named author and a named reviewer whose explicit job was to try to break it.
Empirical and value claims kept separate. Every product distinguishes what the evidence shows from what is a value choice - including whether, and how far, the state should act where the scope of its role is contested - and routes the value choices to the public rather than resolving them in-house.
Steelmanning required. Every product states the strongest version of the case against its own proposal. A product that cannot state that case well has not understood its own subject.
The Panel. The independent Evidence and Methodology Panel can hold back any product that has not genuinely met these standards.

B6. Costing discipline

No proposal is published without a costing. The costing states the whole-life cost rather than just the first-year cost; identifies a specific, plausible funding source rather than assuming the money appears; states who ultimately bears the cost, including in ways that do not show up in the budget; and is honest about the range of uncertainty. Where Pragma expects a proposal to save money or raise revenue over time, it grades that claim on the evidence scale in B4 like any other, and does not treat hoped-for savings as banked. Every costing is set against the honest "do nothing" baseline (B2 Stage 3): the risks, harms and benefits of leaving the status quo unchanged are stated as the benchmark, so the case for acting is always a comparison against the cost of inaction - never an assumption that acting is free, or that doing nothing is.

B7. Independence and anti-capture

The method is the institute's main protection against capture, and it is enforced:

No funder sees, shapes or approves a product before publication.
Funding may be earmarked for a problem area, never for a conclusion.
Everyone who works on or reviews a product declares relevant interests, published with the work; a live interest requires recusal.
The method itself is public, so that any reader - supporter or critic - can check whether a given product actually followed it.

These rules are stated in binding form in the Pragma Charter, sections 7 and 8.

B8. Living proposals: monitoring, commentary and revision

A Pragma proposal is not finished when it is published; it is a living document. The world keeps moving - governments publish reports, inquiries and committees recommend, legislation advances - and a proposal that does not move with it becomes irrelevant the moment the first overlapping government publication appears. Keeping proposals current is part of the work, not an afterthought.

The institute therefore continuously monitors the flow of relevant official output. The same auto-wonk feed that surfaces neglected problems (A7) also surfaces developments that bear on existing proposals. Two responses follow:

Commentary. When a development overlaps a Pragma proposal, the institute publishes a short, non-partisan note: how the proposal bears on what is being reported or legislated, where they agree, where they differ, and what the proposal would add. (An interim report on young people and work, for instance, bears directly on the National Employment Service.) This keeps Pragma part of the live debate rather than a library of finished papers.
Revision. The proposal itself is then brought up to date - its evidence, costings and political reading refreshed - under the iteration discipline (Stage 8), with every version retained and each change recorded.

A Register and its proposals kept this way are living instruments, responsive to events - not a one-time snapshot that ages out of relevance.

Section 9 - A living method

This method is version 0.4. It is deliberately published before it is perfect, in the same spirit Pragma asks of government.

It will be wrong in places. The categories may not fit every problem; the Priority Score dimensions may be mis-weighted; the pipeline may have a missing stage. Pragma treats improvements to the method as among the most valuable contributions anyone can make. Proposed changes are themselves assessed against evidence and reasoning, approved by the Evidence and Methodology Panel, and published as a new numbered version, with the change and its rationale recorded. Earlier versions are never deleted, so that any past piece of work can be read against the method that actually produced it.

Version history

0.1 (May 2026) - founding draft for discussion.
0.2 (May 2026) - added A6, Systemic centrality, distinguishing a problem's leverage across the Register (how many other problems depend on it) from its importance in isolation, and making that leverage an explicit factor in the order of development. Rationale: development sequencing was already implicitly weighing inter-problem dependence - adult social-care funding sits upstream of local-government finance and the NHS; state capacity is upstream of every proposal - so naming it makes the judgement explicit and contestable, consistent with the rest of the Method. Centrality is a sequencing overlay only; the seven-dimension Priority Score is unchanged.
0.3 (May 2026) - added B8, Living proposals: Pragma proposals are living documents, continuously monitored against new government publications, inquiries and legislation (via the auto-wonk feed, A7), with published non-partisan commentary on how a proposal bears on what is being reported or legislated, and routine revision so proposals evolve rather than ageing out of relevance after the first overlapping publication. Also refreshed the B3 illustration to record that the NES worked product has been developed to Method standard. Rationale: prompted by the overlap between Alan Milburn's Young people and work: interim report (28 May 2026) and the National Employment Service - a proposal not kept current becomes irrelevant the moment government moves.
0.4 (June 2026) - separated, across Part A, whether government can act from whether it should. The principle is stated in A1: whether the state should use an available lever - sharpest where a problem touches liberty, autonomy or conscience - is a contested value question that Pragma does not resolve in-house and does not use to screen problems out; a material, market-insufficient, tractable problem is admitted even where intervention is contested, and the should-we-act question is routed to the public during development (Part B). Test 3 (Government tractability) is kept as a lean capability gate ("can government act, not should it") with a pointer to A1. The honest "do nothing" baseline, already a required option at B2 Stage 3, is made an explicit costed comparator in B6, and the scope-of-the-state value question is named in the B5 empirical/value separation. Rationale: an external reviewer proposed folding "should government act?" into the Test 3 gate. The instinct is sound - the boundary of state action is political, and inaction must be a costed benchmark - but making it a gate would have had Pragma settle a contested value judgement in-house (the mirror image of the cardinal error the Method prevents) and could screen out exactly the hard, contested problems Pragma exists to confront. This change keeps the capability gate clean while routing the value/scope question through the existing public-deliberation machinery, and makes the do-nothing comparison explicit. The substantive point sits in A1 (a principle governing the whole gateway) with a pointer from Test 3 (where the temptation to gate on "should" actually arises).

A method that cannot be questioned is a dogma. This one is meant to be questioned.