Enterprise AI Truth Governance Harness

Source Excerpt

Author: Julius (Governance Commander)

Date: 2026-05-21

Context: Response to Emerson senior leadership concern about competing truths, epistemic drift, and knowledge poisoning in enterprise AI systems

Related: Karpathy Large Language Model Wiki pattern, RAG data governance, Multi-Agent Orchestration governance

Executive Summary

Enterprise AI systems face a fundamental challenge: how to maintain truth integrity when ingesting from sources that may be stale, contradictory, contextually limited, or deliberately false. This document outlines a company-specific harness architecture built around specialized AI agents, designed to establish and protect truth — not by eliminating competing truths, but by honestly representing what the system knows, what it's uncertain about, and where sources conflict.

The harness is built on six specialized agent roles, a three-layer knowledge architecture (inspired by Karpathy's Large Language Model Wiki pattern), and a source trust framework that weights information by provenance, recency, and domain applicability.

The Problem Space

Epistemic Drift

The overarching phenomenon. An AI system's definition of "truth" gradually shifts away from reality as it ingests conflicting sources, outdated information, or outright falsehoods. The system fails to maintain a stable epistemic baseline over time.

Retrieval Poisoning

The adversarial subset. Research by Amine Raji demonstrates that injecting just 3 fabricated documents into a RAG knowledge base caused the Large Language Model to report false data as fact with full confidence in 19 out of 20 queries — a 95% poisoning success rate with no defenses. Even benign data degradation (stale docs, conflicting policies) produces the same effect.

Context Contamination

When an Large Language Model receives multiple "authoritative" retrieved sources that contradict each other, it either hallucinates a middle ground or picks the wrong one with confidence. The system appears authoritative while being wrong.

Domain-Specific Truth Decay

Not all knowledge decays at the same rate. A 2023 blog post about AI model capabilities is already stale. A 2023 mechanical engineering specification may still be valid. The harness must apply differential decay based on domain volatility.

Core Architecture: Three-Layer Knowledge Model

Adapted from Karpathy's Large Language Model Wiki pattern for enterprise scale.

Layer 1: Raw Sources (Immutable Anchor)

• Original documents, articles, papers, vendor specs, internal test results
• Never modified — only appended to
• Every document carries provenance metadata: source URL, author, publication date, source tier, domain classification
• This is the ground truth layer. The system never claims the raw layer is wrong — it only flags when raw sources contradict each other

Layer 2: Wiki (Curated Synthesis)

• Large Language Model-generated markdown pages: summaries, concept articles, cross-references, synthesis pages
• Explicitly marked as generated interpretation, not authoritative fact
• Every wiki page carries: sources cited, last-updated timestamp, confidence score, flagged contradictions
• When sources conflict, the wiki page surfaces the competition rather than resolving it
• Human reviewers can lock pages, add annotations, or adjust confidence scores

Layer 3: Schema (Governance Rules)

• Defines structure, conventions, and agent workflows
• Source tier definitions and trust weight assignments
• Domain volatility classifications and decay schedules
• Escalation thresholds and human-review triggers
• This is the rulebook. Agents follow it. Humans maintain it.

The Six Agent Roles

Each agent has a narrow mandate, explicit tool contracts, and defined escalation paths. No single agent has authority to establish truth unilaterally.

1. Ingestion Agent

Mandate: Acquire, validate, and archive new sources into the Raw layer.

Responsibilities:

• Fetch documents from approved sources (web, internal repos, APIs)
• Run source-tier classification: what category of trust does this source belong to?
• Extract and validate provenance metadata (author, date, domain, version)
• Perform embedding anomaly detection: does this document's embedding vector fall within the statistical distribution of existing content on the same topic? (This alone reduces poisoning success from 95% to 20% per Raji's testing)
• Archive to Raw layer with immutable hash
• Flag for human review if: anomaly detected, source tier is low, or metadata is incomplete

Tool Contract:

• Input: source URL or document + source tier config
• Output: archived raw document + metadata + anomaly score + review flag
• Error modes: fetch failure, encoding error, metadata extraction failure

Critical Pitfall: The Ingestion Agent must never modify source content. It archives verbatim. Any normalization happens in the Wiki layer, not Raw.

2. Verification Agent

Mandate: Cross-check claims against multiple sources and establish confidence scores.

Responsibilities:

• When a new source is ingested, compare its key claims against existing Raw sources
• Detect contradictions: flag when Source A says X and Source B says Y on the same topic
• Perform multi-source verification: require minimum N independent sources before a claim reaches "established" status
• Calculate confidence scores based on: source tier weighting, recency, number of corroborating sources, domain applicability
• Generate contradiction reports for the Governance Agent

Tool Contract:

• Input: new raw document + existing corpus query results
• Output: claim-level verification report with confidence scores + contradiction flags
• Error modes: insufficient sources for verification, ambiguous claim parsing

Critical Pitfall: The Verification Agent does not resolve contradictions. It surfaces them. Resolution is a human decision or a Governance Agent policy decision.

3. Synthesis Agent

Mandate: Maintain the Wiki layer — create, update, and cross-reference knowledge pages.

Responsibilities:

• Read new Raw sources and create/update Wiki pages
• Build cross-references between related topics
• Surface contradictions explicitly in wiki pages: "Source A claims X, Source B claims Y. Status: unresolved."
• Tag knowledge with domain applicability: "Valid for: Plant A, 2024-2026. Not verified for: Plant B."
• Run periodic lint operations: identify stale pages, orphan pages, missing cross-references, unresolved contradictions
• Never claim certainty where the Verification Agent has flagged uncertainty

Tool Contract:

• Input: raw documents + verification reports + schema rules
• Output: wiki page updates + cross-reference links + lint report
• Error modes: schema violation, contradiction not surfaced, missing provenance citation

Critical Pitfall: The Synthesis Agent is the most likely to drift — it's the one generating natural language. It must always cite sources, always flag uncertainty, and never smooth over contradictions for readability.

4. Governance Agent

Mandate: Enforce the Schema layer — maintain rules, manage source tiers, and handle escalations.

Responsibilities:

• Maintain source tier definitions and trust weights
• Manage domain volatility classifications and knowledge decay schedules
• Review contradiction reports from the Verification Agent
• Make policy decisions: when to escalate to humans, when to auto-decay, when to quarantine sources
• Maintain the agent registry: who is authorized to modify Schema rules
• Generate governance reports: drift metrics, poisoning attempts detected, contradiction resolution rates

Tool Contract:

• Input: schema state + governance reports + escalation queue
• Output: updated schema + policy decisions + governance metrics
• Error modes: conflicting policy rules, unauthorized schema modification attempt

Critical Pitfall: The Governance Agent must never auto-resolve high-stakes contradictions. Its role is to apply rules, not to make truth judgments. Human review is the final authority.

5. Decay Agent

Mandate: Track knowledge half-lives and flag stale information.

Responsibilities:

• Apply domain-specific decay functions to all wiki knowledge
• Volatility classifications:
• High volatility (AI models, cybersecurity, market trends): 30-90 day half-life
• Medium volatility (vendor specs, regulatory updates): 6-12 month half-life
• Low volatility (fundamental engineering principles, historical data): 2-5 year half-life
• Flag pages for re-verification when decay threshold is reached
• Auto-archive knowledge that exceeds maximum age without re-verification
• Generate staleness reports for the Governance Agent

Tool Contract:

• Input: wiki pages + domain classifications + decay schedule
• Output: staleness flags + re-verification queue + archive candidates
• Error modes: missing domain classification, decay schedule conflict

Critical Pitfall: Decay is not deletion. Stale knowledge is flagged and archived, not removed. Historical context matters, especially for engineering and compliance.

6. Query Agent

Mandate: Answer user questions with honest epistemic positioning.

Responsibilities:

• Receive user queries and retrieve from Wiki layer
• Apply semantic consensus checking: do retrieved documents actually agree? If not, surface the disagreement
• Return answers with explicit confidence levels: "Established (high confidence)", "Contested (sources disagree)", "Tentative (limited sources)", "Unknown (no reliable sources)"
• Never fabricate certainty. If the system can't answer with confidence, it says so
• Log all queries for the Governance Agent to detect drift patterns

Tool Contract:

• Input: user query + wiki retrieval results
• Output: answer + confidence level + source citations + contradiction flags
• Error modes: insufficient retrieval, consensus failure, confidence below threshold

Critical Pitfall: The Query Agent is the face of the system. It must resist the Large Language Model's natural tendency to be confidently wrong. When in doubt, it escalates rather than guesses.

Source Trust Framework

Not all sources are equal. The harness implements a tiered trust system:

Tier 1: Authoritative Sources (Weight: 1.0)

• Peer-reviewed research with replication
• Official vendor documentation (e.g., Emerson product specs)
• Regulatory standards and codes
• Internal validated test results with documented methodology

Tier 2: Trusted Sources (Weight: 0.7)

• Industry publications with editorial review
• Vendor technical briefs and bulletins
• Internal engineering reports
• Conference proceedings

Tier 3: Reference Sources (Weight: 0.4)

• Industry blogs with named authors
• Community forums with established reputations
• Unverified internal communications
• News articles about technical topics

Tier 4: Low-Confidence Sources (Weight: 0.1)

• Anonymous or unattributed claims
• Social media posts
• Unverified trend reports
• AI-generated content without human review

Trust Calculation

For any claim, the system calculates:

Trust Score = SUM(source_weight × recency_factor × domain_applicability) / MAX_POSSIBLE_SCORE

Source excerpt truncated at 220 of 367 lines. Open the canonical wiki path above for the full page.