Pragma: Evidence-Based Policy Analysis

Policy synthesis that shows its work

Most policy analysis comes from organizations with positions. Think tanks produce reports that align with their funders’ priorities. Advocacy groups marshal evidence for conclusions they’ve already reached. Government agencies optimize for political feasibility. The evidence is real, but it’s been filtered through institutional incentives before it reaches you.

Pragma is a structured methodology for policy evidence synthesis. You give it a policy question - “What does the evidence say about carbon taxes?” or “Should this jurisdiction adopt Housing First?” - and it tells you what the evidence actually establishes, where the evidence is weak, what value choices are embedded in the question, and what would need to be true for a given recommendation to hold.

It is not advocacy. It is not opinion. It is a documented process for translating verified evidence into rigorous, auditable policy inference.

How it works

Pragma operates at four tiers of depth, depending on the question:

Scan: Quick assessment of what the evidence landscape looks like. Identifies whether a question is empirical, value-based, or both. Minutes, not hours.
Standard: Full evidence synthesis with transferability assessment, trade-off mapping, and implementation constraints. The default for most policy questions.
Full: Deep dive with all 13 gaming countermeasure checks, detailed normative framework analysis, and pathway assessment for implementation.
Forensic: Maximum rigor. For high-stakes questions where the evidence is contested, the political pressure is intense, or the consequences of getting it wrong are severe.

Every Pragma assessment evaluates evidence along three independent dimensions: where it was published (source quality, 4 tiers), what methodology was used (study design, 6 levels with quality modifiers), and how stable the field’s evidence base is (field reliability). Confidence cannot exceed the ceiling imposed by the weakest of the first two dimensions. Ten unreliable studies don’t compensate for one missing rigorous one.

Evidence is then filtered through a 7-dimension transferability assessment: does evidence from Context A actually apply to Context B? A successful policy in a small, high-trust Nordic country does not automatically transfer to a large, federalized, low-trust jurisdiction. Pragma assesses population match, institutional match, economic match, cultural fit, scale, temporal relevance, and constitutional/legal compatibility. Each is rated independently. Low transferability on a mechanism-critical dimension reduces confidence regardless of how strong the underlying study is.

What makes it different from think tanks

Transparent methodology. The entire process is documented and inspectable. You can see exactly how Pragma reached its conclusions and where value choices entered the analysis.

Explicit confidence bands. Pragma uses five confidence levels (High, Moderate-High, Moderate, Low, Speculative) plus three special assessments (Contested, Inadvisable, Not Assessable). Each has defined evidence requirements. “Moderate confidence” means something specific: good evidence, reasonable transferability, plausible mechanism, some implementation precedent.

Gaming countermeasures. Policy evidence is more vulnerable to manipulation than factual claims. Pragma checks for 13 specific attack vectors: cherry-picked jurisdictions, mechanism laundering (treating correlation as usable causal mechanism), implementation idealization (citing pilot results as if they’d hold at scale), value laundering (concealing normative choices inside empirical language), transferability theater (acknowledging context differences then ignoring them), and eight others. Each has a named detection procedure.

Value disputes identified, not hidden. When a policy question involves genuine disagreement about values rather than facts, Pragma says so. It maps the value structure - what each side is actually prioritizing - instead of pretending the disagreement is purely empirical. A Pragma output that reads like pure technocratic advice has failed. A Pragma output that reads like advocacy has also failed.

Where Pragma sits

Pragma is the middle stage of a three-system pipeline:

Veridi answers “Is this true?” - fact-checking with source hierarchy, gaming countermeasures, and structured verdicts.
Pragma answers “What should we do about it?” - policy evidence synthesis with transferability assessment, trade-off mapping, and explicit value identification.
Praxis answers “What can I do?” - individual action planning with leverage matching, risk assessment, and do-no-harm guardrails.

Each system can be used independently. Together, they form a pipeline where confidence decreases transparently at each stage - from factual certainty through policy uncertainty to action-level speculation.

Who it’s for

Policymakers who want to know what the evidence actually says before deciding what to prioritize. Researchers who want a structured framework for translating findings into policy-relevant synthesis. Journalists covering policy who need to distinguish evidence-grounded claims from advocacy dressed as analysis. Advocates who want their positions grounded in evidence they’ve actually examined rather than evidence they’ve selected.

Current status

Pragma v1.1 has been validated against 55 test claims across multiple policy domains: 53 passed, 2 partial, 0 failures. The partial results were corrected and re-validated. The test suite included contested questions, multi-domain policy interactions, and adversarial claims designed to exploit specific weaknesses.

Pipeline integration testing covered 10 end-to-end scenarios across all three systems (30 stage executions), all passing. Gaming flags propagated correctly across stages, and confidence decreased appropriately from fact-checking through policy analysis to action planning.

The methodology is documented, the test results are available, and we welcome attempts to break it.