Tag: AI Governance

The Guardian: Human-in-the-Loop AI Governance

The Guardian: Human-in-the-Loop AI Governance

We’ve built a system that is Reliable and Affordable. Our Forensic Team is accurate, and The Accountant ensures we aren’t wasting our cognitive budget.

But in the enterprise, “capable” is not enough. For high-stakes decisions—like a $50k rare book audit or a compliance check—fully autonomous AI is a Liability.

Today, we introduce The Guardian: The final phase of our Production-Grade AI trilogy. We are implementing a standardized Human-in-the-Loop (HITL) checkpoint, moving from “Autonomous Agents” to “Augmented Intelligence.”

1. The Autonomous Trap: Confident Hallucination

In the first post of this series, The Judge proved that even the best models can confidently hallucinate. In a forensic audit, an agent might identify a water damage pattern and declare: “CRITICAL: High probability of modern forgery.” If that finding is wrong, the reputational and financial damage is severe. The problem isn’t the AI’s capability; it’s the lack of authorization. The agent is a worker, not a partner.

2. Implementing the “Governance Gate”

We need a way to “brake” the agent’s flow when it finds a high-severity issue. We’ve added the request_human_signature tool to our Forensic Analyzer MCP server project.

In orchestrator.py, we updated the logic. When the Analyst flags a “HIGH” severity discrepancy, the system performs a specialized handshake:

  1. Stateful Pause: The Python orchestrator interrupts the agent workflow.
  2. Authorization Prompt: It presents the evidence to the user via a CLI prompt.
  3. Cryptographic Signature: The user must authorize the finding before it’s committed to the final report.
# The Guardian's "Nuclear Key" moment in orchestrator.py
def _apply_guardian_handshake(analyst_result: dict) -> tuple[dict, list[dict]]:
    """
    Human-in-the-Loop: if Analyst has HIGH discrepancies, prompt for authorization.
    """
    disputed: list[dict] = []
    data = analyst_result.get("data") or {}
    disc = data.get("discrepancies", [])

    # Filter for the "High Stakes" findings
    high_disc = [d for d in disc if (d.get("severity") or "").upper() == "HIGH"]

    for d in high_disc:
        summary = f"[{d.get('severity')}] {d.get('field')}: {d.get('expected')} vs {d.get('observed')}"
        print(f"\n  Guardian: HIGH severity finding — {summary}")

        # THE STATEFUL PAUSE: The orchestrator stops and waits for a human
        answer = input("  Do you authorize this forensic finding? (yes/no): ").strip().lower()

        if answer != "yes":
            # Escalation: If not authorized, it's flagged as 'DISPUTED_BY_HUMAN'
            disputed.append({**d, "status": "DISPUTED_BY_HUMAN"})

    return analyst_result, disputed

By requiring a human to type ‘yes’, we are moving from Autonomous Assumption to Authorized Augmentation in the following ways:

  1. Severity-Based Intervention: “We don’t interrupt the user for every ‘Low’ or ‘Medium’ variance. We only trigger the Guardian for High-Severity findings—those that carry legal or financial liability. This preserves the ‘UX flow’ while maintaining safety.”
  2. The ‘Disputed’ State: “Notice that a ‘No’ from the human doesn’t just delete the finding. It moves it to a specialized ‘Requires Further Investigation’ section of the report. This ensures that the AI’s observation is preserved but clearly labeled as unauthorized.”
  3. Non-Interactive Fallback: “The code includes a check for EOFError (line 507). If the system is running in a non-interactive environment like a CI/CD pipeline, it defaults to ‘No’ (Dispute) for safety. Never default to ‘Yes’ for a high-risk authorization.”
Architectural diagram of a human-in-the-loop AI governance system called The Guardian. An agent workflow processes a task. When it detects a high-severity finding, it pauses and performs a stateful 'Authorization Handshake' with a Human Guardian. The human must sign or reject the finding before it proceeds to finalize the output report.
The Guardian Architecture—Moving from Autonomous Agents to Stateful, Authorized Human-AI Augmentation.

3. Beyond the CLI: The Enterprise Handshake

This reference implementation uses a CLI input() prompt for simplicity. However, the MCP tool is standardized. In a production environment, this tool wouldn’t pause a Python script; it would:

  • Trigger a Slack/Teams Alert to a senior auditor.
  • Open a Jira Ticket for manual review.
  • Request a Webauthn (Biometric) Signature in a web dashboard.

Summary: Building the Sovereign AI Stack

Across this series, we’ve moved from basic orchestration to a Production-Grade AI Mesh. We’ve proven that we can build systems that are:
1. Reliable: Audited by The Judge.
2. Sustainable: Optimized by The Accountant.
3. Safe: Governed by The Guardian.

The road to autonomous agents isn’t paved with more tokens; it’s paved with better guardrails.

What’s Next?

The code for the entire trilogy is available in the MCP Forensic Analyzer repository.

I’m currently working on Phase 3: The Sovereign Vault, where we will explore Local Multimodal Vision (processing artifact images without cloud egress) and PII Redaction to protect proprietary “Golden Data.”

Have questions about implementing these patterns in your own enterprise? Connect with me on LinkedIn or follow the blog for the next series.

The Production-Grade AI Series (Complete)

Looking for the foundation? Check out my previous series: The Zero-Glue AI Mesh with MCP.

Facebooktwitterredditlinkedinmail

Who Audits the Auditors? Building an LLM-as-a-Judge for Agentic Reliability

We’ve built a powerful Forensic Team. They can find books, analyze metadata, and spot discrepancies using MCP.

But in the enterprise, ‘it seems to work’ isn’t a metric. If an agent misidentifies a $50,000 first edition, the liability is real.

Today, we move from Subjective Trust to Quantitative Reliability. We are building The Judge—a high-reasoning evaluator that audits our Forensic Team against a ‘Golden Dataset’ of ground-truth facts.

Before you Begin

Prerequisites: You should have an existing agentic workflow (see my MCP Forensic Series) and a high-reasoning model (Claude 3.5 Opus/GPT-4o) to act as the Judge.

1. The “Golden Dataset”

Before we can grade the agents, we need an Answer Key. We’re creating tests/golden_dataset.json. This file contains the “Ground Truth”—scenarios where we know there are errors.

Example Entry:

{
"test_id": "TC-001",
"input": "The Great Gatsby, 1925",
"expected_finding": "Page count mismatch: Observed 218, Standard 210",
"severity": "high"
}

Director’s Note: In an enterprise setting, “Reliability” is the precursor to “Permission”. You will not get the budget to scale agents until you can prove they won’t hallucinate $50k errors. This framework provides the data you need for that internal sell.

2. The Judge’s Rubric

A good Judge needs a rubric. We aren’t just looking for “Yes/No.” We want to grade on:

  • Precision: Did it find only the real errors?
  • Recall: Did it find all the real errors?
  • Reasoning: Did it explain why it flagged the record?

3. Refactoring for Resilience

Before building the Judge, we had to address a common “Senior-level” trap: hardcoding agent logic. Based on architectural reviews, we moved our system prompts from the Python client into a dedicated config/prompts.yaml.

This isn’t just about clean code; it’s about Observability. By decoupling the “Instructions” from the “Execution,” we can now A/B test different prompt versions against the Judge to see which one yields the highest accuracy for specific models.

4. The Implementation: The Evaluation Loop

We’ve added evaluator.py to the repo. It doesn’t just run the agents; it monitors their “vital signs.”

  • Error Transparency: We replaced “swallowed” exceptions with structured logging. If a provider fails, the system logs the incident for diagnosis instead of failing silently.
  • The Handshake: The loop runs the Forensic Team, collects their logs, and submits the whole package to a high-reasoning Judge Agent.

The Evaluator-Optimizer Blueprint

This diagram represents our move from “Does the code run?” to Does the intelligence meet the quality bar?” This closed-loop system is required before we can start the fiscal optimization of choosing smaller models to handle simpler tasks.

Architectural diagram of an AI Evaluator-Optimizer loop. It shows a Golden Dataset feeding into an Agent Execution layer, which then passes outputs and logs to a Judge Agent for scoring against a rubric. The final Reliability Report provides a feedback loop for prompt tuning and iterative improvement.
The Evaluator-Optimizer Loop-Moving from manual vibe-checks to automated, quantitative reliability scoring.

Director-Level Insight: The “Accuracy vs. Cost” Curve

As a Director, I don’t just care about “cost per token.” I care about Defensibility. If a forensic audit is challenged, I need to show a historical accuracy rating. By implementing this Evaluator, we move from “Vibe-checking” to a Quantitative Reliability Score. This allows us to set a “Minimum Quality Bar” for deployment. If a model update or a prompt change drops our accuracy by 2%, the Judge blocks the deployment.

The Production-Grade AI Series

  • Post 1: The Judge Agent — You are here
  • Post 2: The Accountant (Cognitive Budgeting & Model Routing) — Coming Soon
  • Post 3: The Guardian (Human-in-the-Loop Handshakes) — Coming Soon

Looking for the foundation? Check out my previous series: The Zero-Glue AI Mesh with MCP.

Facebooktwitterredditlinkedinmail