← Back to TIL
Second Gear / July 26, 2026

Model Routing Needs Governance

Aggregation

Tech

1 critical / 0 happenings

Ideation

Ideation

Idea 1 Model Chain-of-Custody Firewall

Sell regulated companies a runtime control plane that proves which model, host, fine-tune, adapter, tool, and cached trace touched each AI decision before that output reaches a customer, employee, or government contract workflow. The wedge is not cheaper routing. It is making model choice auditable enough that teams can use low-cost open-weight models without accidentally inheriting export, procurement, IP, residency, or vendor-origin risk.

Source Signals

Why Now

Model routers are becoming normal because the price gap is too large to ignore, but the governance layer is still mostly static policy, logging, and trust-center paperwork. Frontier systems are now cheap enough to be selected per task, which means compliance has to move from annual vendor review to per-inference chain-of-custody.

First Wedge

A proxy that plugs into LiteLLM, OpenRouter, Bedrock, Azure, and self-hosted vLLM deployments for defense suppliers, fintechs, and healthcare vendors. It blocks or quarantines calls when a model's origin, hosting region, license, fine-tune lineage, or provider terms violate a customer-defined contract or regulatory policy.

Commercial Model

Security, procurement, or AI platform teams pay per protected AI application plus usage-based audit storage. The budget exists because the alternative is forcing teams onto expensive approved models, losing the cost savings that made routing attractive, or accepting contract risk they cannot explain to auditors.

Defensibility

The compounding asset is a normalized evidence graph of models, derivative weights, hosting providers, licenses, benchmark regressions, policy changes, and customer-specific contract constraints. Incumbent gateways can log traffic, but they do not automatically know that a cheap model was distilled from, hosted by, proxied through, or contractually tied to a newly restricted supplier.

Technical Risk

The hard part is trustworthy provenance inference when model cards are incomplete and derivative models hide their ancestry. The product needs a mix of signed attestations, vendor feeds, weight fingerprinting, benchmark-behavior fingerprints, policy ingestion, and conservative unknown-state handling without crushing developer velocity.

Market Expansion

Start with LLM calls in regulated software, then expand to multimodal models, robotics policies, autonomous-agent toolchains, and AI-enabled supplier due diligence. The same primitive applies anywhere autonomous systems choose external intelligence at runtime.

Self-Critique

This could collapse into a feature inside Palo Alto, Wiz, Cloudflare, OpenRouter, or a procurement-risk platform if the product is only a dashboard. It only deserves to exist if it enforces decisions inline and becomes the trusted evidence layer auditors and platform teams rely on when model supply chains change weekly.

Next Experiment

Interview 12 AI platform or security leads at defense-adjacent SaaS, fintech, and healthcare companies already using more than one model provider. Build a thin proxy that emits an AIBOM per request, flags Chinese-origin or unknown-origin model paths, and tests whether teams will pay to keep low-cost routing while satisfying one real procurement or compliance review.