Open Methodology
How to inspect Veridi’s methodology
The complete Veridi methodology is published as a set of Markdown documents. These are the same files the AI follows when performing a fact-check; there is no hidden layer or proprietary process. What you see is what runs.
The methodology files
Core system
| File | Purpose |
|---|---|
| Claim_Triage.md | The primary system prompt. Defines claim classification, complexity assessment, and routing logic. |
| System_Flow.md | Architecture diagram showing how a claim moves through the system - from input through triage, specialist invocation, and integrated output. |
| Output_Format_Standard.md | Defines the nine verdict categories, output structure, and annotation requirements. As of v2.5, confidence is presented as verbal bands with structural ceiling context. |
| Verdict_Decision_Trees.md | Explicit decision logic for resolving verdict boundary cases, particularly Misleading vs. Lacks Context and Mixed vs. Mostly False. |
Evidence evaluation
| File | Purpose |
|---|---|
| Source_Hierarchy.md | The four-tier source classification system with confidence ceilings and independence verification procedures. v2.8 adds a (R)/(P)/(S) source-classification gate that requires every EVIDENCE entry to be either Retrieved (R), Primary-in-claim (P), or removed; substrate-knowledge (S) entries cannot pass. |
| Confidence_Calibration_Framework.md | Tier-based confidence caps, field reliability coefficients with sourcing honesty labels, confidence-in-verdict vs. likelihood distinction (ICD 203), and the interaction rules between them. |
| Institutional_Reliability_Index.md | Per-agency, per-function reliability assessments for institutions whose output may have been compromised. Includes degradation levels, observable indicators, and comparison anchors. As of v2.5, covers both US and Canadian federal agencies. |
Gaming countermeasures
| File | Purpose |
|---|---|
| Gaming_Countermeasures.md | Detection procedures for all 12 disinformation attack vectors, including substrate self-reference (the assessor evaluating a claim about itself, its developer, or the operator’s own system; v2.7 added an explicit trigger and a 75% confidence ceiling). Includes the 15-item quick checklist applied before any verdict above 70% confidence (item 15 added in v2.6 detects promotional/advocacy framing — evaluative assertions embedded in apparently factual claims, decomposed and routed to VALUE JUDGMENT treatment). |
Domain specialists
| File | Purpose |
|---|---|
| Scientific_Claims_Specialist.md | Evaluation framework for scientific claims: study quality, methodology assessment, consensus evaluation. |
| Medical_Health_Specialist.md | Medical and public health claim evaluation, including clinical trial assessment and pharmacological claims. |
| Legal_Regulatory_Specialist.md | Legal and regulatory claim evaluation: statute interpretation, court ruling analysis, regulatory procedure. |
| Financial_Economic_Specialist.md | Financial and economic claim evaluation: market data, institutional analysis, statistical claims. |
| Electoral_Voting_Specialist.md | Electoral and voting claim evaluation: election procedures, voter data, policy analysis. |
| Historical_Context_Specialist.md | Historical claim evaluation: contextualization, historiographic assessment, source verification for historical records. |
| Technology_Digital_Specialist.md | Technology and digital claim evaluation: platform analysis, digital forensics, AI-generated content detection. |
| Propaganda_Deconstruction_Specialist.md | Propaganda and narrative analysis: rhetorical technique identification, narrative deconstruction, disinformation pattern recognition. |
| Breaking_Event_Analyst.md | Evaluation framework for claims about events less than 72 hours old: timeline construction, source ecosystem mapping, uncertainty inventory. |
Supporting frameworks
| File | Purpose |
|---|---|
| Statistical_Claims_Checklist.md | Structured evaluation for statistical claims: methodology validation, sample assessment, cherry-picking detection. |
| Infrastructure_Authenticity.md | Digital infrastructure verification: domain registration, hosting analysis, website authenticity assessment. |
Operational
| File | Purpose |
|---|---|
| Regression_Testing_Framework.md | How the methodology is tested for consistency across versions. v1.1 adds §5c StrongREJECT capability-aware judge readiness ladder (4 readiness gates). |
| cross-model-evaluation-protocol.md | Protocol for evaluating Veridi outputs across model families. v1.1 extends with a Multitrait-Multimethod (MTMM) design: trait × method matrix, four pre-registered Campbell-Fiske decision rules, sample-size scaling target, expert-fact-checker as third method. Panels not running this quarter; protocol-doc shipped for readiness. |
| Crisis_Communication_Plan.md | Procedures for when Veridi produces an incorrect assessment. |
| Legal_Escalation_Path.md | Procedures for claims with legal implications. |
| Volunteer_Safety_Framework.md | Safety procedures for volunteers working with sensitive or distressing content. |
| CHANGELOG.md | Version history of methodology changes. |
Validation
| File | Purpose |
|---|---|
| golden_test_set_A.md | 25 cross-domain test claims with documented ground truth (GTS-001 to GTS-025). |
| golden_test_set_B.md | 25 weakness-targeting test claims (GTS-026 to GTS-050). |
| golden_test_set_C.md | 20 gap-filling test claims (GTS-051 to GTS-070). |
| adversarial_test_suite_a.md | 12 single-vector adversarial claims (ADV-001 to ADV-012). |
| adversarial_test_suite_b.md | Multi-vector adversarial claims (ADV-013 onward), including ADV-025 (methodology self-reference, v2.6), ADV-026 (substrate self-reference, v2.7), and ADV-027 to ADV-031 (IPI scenarios across 5 NIST subcategories with ATLAS AML.T#### IDs, template-form with worked examples, v1.1). |
| validation-results/ | Per-claim scorecards with full evidence and reasoning. |
What to look for
If you’re reviewing the methodology, here are the most important things to evaluate:
Decision tree consistency. Do the verdict decision trees produce the same result regardless of which path you take? Are there contradictions between the trees and the output format definitions?
Source hierarchy completeness. Are there source types that don’t fit cleanly into the four tiers? Are there edge cases where the confidence ceiling produces unreasonable results?
Gaming countermeasure coverage. Are there disinformation techniques not covered by the twelve vectors? Can you construct a claim that should be detected but wouldn’t be?
IRI assessment methodology. Are the degradation levels well-defined? Are the comparison anchors genuinely independent? Could the IRI itself be gamed?
Confidence calibration. Are the field reliability coefficients well-sourced? Does the interaction between tier ceilings and field coefficients produce sensible results across edge cases? Is the confidence-in-verdict vs. likelihood distinction clear and consistently applied?
Evidence directness and assumptions. Are the indirectness types (population, context, temporal, metric) well-defined? Are assumptions documented with meaningful consequence-if-wrong statements?
Pragma methodology files
Pragma is the policy evidence synthesis component: given a policy question, it evaluates the evidence base, assesses transferability, identifies value disputes, and produces a structured recommendation with calibrated confidence. Location: Pragma/
Core system
| File | Purpose |
|---|---|
| PRAGMA_METHODOLOGY.md | Primary methodology document. Defines the 4-tier analysis depth (Scan/Standard/Full/Forensic), 9 pathways from question to recommendation, and output format. |
| Pragma_System_Flow.md | Architecture diagram showing how a policy question moves through evidence gathering, quality assessment, transferability analysis, and recommendation synthesis. |
| Pragma_Decision_Trees.md | Decision logic for resolving assessment outcomes: SUPPORTED vs CONTESTED vs NOT ASSESSABLE, evidence strength band assignment, and value dispute identification. |
Evidence evaluation
| File | Purpose |
|---|---|
| Pragma_Evidence_Quality_Framework.md | Three-dimensional evidence assessment: source quality (4 tiers), study design (6 levels with quality modifiers), and field reliability. Includes the Level 3 identification strategy sub-assessment (v2.5) with credibility modifiers for RD, DiD, IV, and Synthetic Control designs. |
| Pragma_Confidence_Calibration.md | Ceiling enforcement, verbal confidence bands (High/Moderate-High/Moderate/Low/Speculative), transferability and implementation gap adjustments, and Brier score tracking against external ground truth. |
| Pragma_Transferability_Rubrics.md | 7-dimension transferability assessment: population, institutional, economic, cultural/social, scale, temporal, and constitutional/legal match. Determines whether evidence from Context A applies to Context B. |
Policy-specific frameworks
| File | Purpose |
|---|---|
| Pragma_Normative_Framework.md | Handles value-laden policy questions: disparity indicators, trajectory assessment, preferential weighting with explicit disclosure, and the distinction between empirical and normative disputes. |
| Pragma_Value_Assessment_Guide.md | Identifies and maps competing values in contested policy questions. Produces the Contested Value Map showing each direction’s evidence strength and underlying value commitments. |
| Pragma_Gaming_Countermeasures.md | 14 gaming vectors specific to policy evidence: mechanism laundering, value laundering, transferability theater, trade-off burial, counterfactual suppression, status quo bias exploitation, and others. |
| Pragma_Veridi_Interface.md | Defines how Veridi fact-check results feed into Pragma analysis: confidence inheritance, gaming flag propagation, and the pipeline handoff protocol. |
Domain specialists (13)
| File | Domain |
|---|---|
| Health_Systems_Specialist.md | Healthcare policy, pharmaceutical regulation, public health |
| Environmental_Climate_Specialist.md | Climate policy, environmental regulation, energy transition |
| Fiscal_Tax_Specialist.md | Tax policy, fiscal policy, public finance |
| Labor_Employment_Specialist.md | Labor markets, employment policy, workplace regulation |
| Housing_Urban_Specialist.md | Housing policy, urban planning, homelessness |
| Education_Specialist.md | Education policy, early childhood, higher education |
| Criminal_Justice_Specialist.md | Criminal justice reform, policing, sentencing |
| Social_Policy_Specialist.md | Social safety nets, welfare, income support |
| Indigenous_Tribal_Specialist.md | Indigenous rights, self-governance, treaty obligations |
| International_Trade_Specialist.md | Trade policy, tariffs, international economic agreements |
| Technology_Digital_Specialist.md | Technology regulation, AI policy, digital rights |
| Political_Economy_Specialist.md | Institutional design, democratic reform, governance |
| Comparative_Specialist.md | Cross-national policy comparison methodology |
Validation
| File | Purpose |
|---|---|
| pragma_golden_test_set.md | 25 policy analysis test claims spanning assessment types. |
| pragma_adversarial_test_suite.md | 15 adversarial claims testing all 14 gaming vectors. |
| pragma_boundary_tests.md | 15 boundary test claims for assessment edges and transferability. |
| pragma_mtmm_protocol.md | Multitrait-Multimethod protocol document (v1.6). Trait × method matrix (3 traits × 4 methods), four pre-registered Campbell-Fiske decision rules, multi-value-frame expert-panel design (4 named normative frames), Cluster D harness dependency. Panels not running this quarter; protocol-doc shipped for readiness. |
Cumulative validation: 53 PASS, 2 PARTIAL, 0 FAIL across 55 test claims.
Praxis methodology files
Praxis is the individual action component: given a policy goal and a person’s profile, it identifies the highest-leverage actions available to that specific person. Location: Praxis/
Core system
| File | Purpose |
|---|---|
| PRAXIS_METHODOLOGY.md | Primary methodology document. Defines the 9 change pathways, the 6-step assessment process (issue landscape, profile matching, leverage scoring, action recommendation, risk screening, sustainability planning), and output format. |
| Praxis_System_Flow.md | Architecture diagram showing the flow from issue assessment through pathway matching to action recommendation with risk gates. |
| Praxis_Leverage_Matching.md | The matching algorithm: pathway identifier table, per-pathway scoring rubrics (profile fields mapped to 0/1/2/-1 scores), threshold logic, and the 9x9 cross-pathway interaction matrix with synergy rationales. |
Evidence and risk
| File | Purpose |
|---|---|
| Praxis_Evidence_Framework.md | Confidence system for action recommendations: pathway evidence ceilings, field reliability coefficients (0.30-0.75 range), leverage confidence bands (Speculative/Weak/Moderate/Strong), and worked examples showing why action-level confidence is lower than policy-level confidence. |
| Praxis_Sustainability_Risk.md | Risk assessment across 6 categories (financial, employment, physical, immigration, social, emotional), vulnerability-adjusted guidance, the do-no-harm guardrail (blocks high-risk actions with only expressive impact), compound vulnerability rules, and portfolio sustainability planning. |
| Praxis_User_Profile.md | 30-field profile schema across 6 sections: demographics, position, skills/resources, constraints, vulnerabilities, and engagement capacity. |
| Praxis_Gaming_Countermeasures.md | 6 gaming vectors specific to individual action: learned helplessness induction, substitution promotion, astroturf organization, urgency manufacture, savior framing, and commitment exploitation. |
Change pathways (9)
Each pathway file defines the mechanism, leverage conditions, evidence base, actions by engagement level, risk profile, sustainability characteristics, and cross-pathway synergies.
| File | Pathway |
|---|---|
| Political_Participation.md | P1: Voting, contacting representatives, running for office, campaign volunteering |
| Collective_Action.md | P2: Organizing, unions, community groups, protest |
| Professional_Leverage.md | P3: Using workplace expertise and position for advocacy |
| Economic_Pressure.md | P4: Strategic spending, shareholder advocacy (individual boycotts are low-leverage) |
| Cultural_Shift.md | P5: Narrative change, education, media, art |
| Mutual_Aid.md | P6: Direct community support, building alternative infrastructure |
| Prefigurative_Action.md | P7: Modeling the world you want (cooperatives, community land trusts) |
| Direct_Action.md | P8: Civil disobedience, disruption (highest risk; nonviolent only) |
| Litigation_Legal_Advocacy.md | P9: Courts as a venue for change; organizational affiliation is the strongest leverage predictor (v2.5) |
Validation
| File | Purpose |
|---|---|
| praxis_golden_test_scenarios.md | 20 test scenarios spanning personas, pathways, and issue types. |
| praxis_adversarial_tests.md | Adversarial scenarios testing the 6 gaming vectors. v1.4 adds PRXA-011 to PRXA-015 IPI scenarios across 5 NIST subcategories with ATLAS AML.T#### IDs (template-form with worked examples). |
| praxis_boundary_tests.md | 10 boundary scenarios testing risk-impact matrix, compound vulnerability, and edge cases. |
| praxis_test_suite_design.md | Test suite design document. v1.4 adds §5b StrongREJECT readiness ladder with composite refusal × competence × specificity × pathway-coherence. |
| praxis_mtmm_protocol.md | Multitrait-Multimethod protocol document (v1.4). Trait × method matrix, four-criteria decision rules, expert-panel design across organizing traditions (Ganz / McAlevey / Han), method-variance disclosure binding, Cluster D dependency note. Panels not running this quarter; protocol-doc shipped for readiness. |
Cumulative validation: 34 PASS, 6 PARTIAL, 0 FAIL across 40 test scenarios.
The pipeline
Veridi (is this true?) feeds into Pragma (what should we do?) feeds into Praxis (what can I do?). Each system can also be used independently. Pipeline integration was validated in v2.5: 10 end-to-end scenarios, 30/30 stage executions PASS. Confidence decreases transparently at each stage as uncertainty accumulates.
Reporting issues
If you find an inconsistency, gap, or vulnerability in the methodology, we want to hear about it. The methodology improves through exactly this kind of external scrutiny.
Contact: veridi [at] nettercap.net