Jay Malhotra
Back to all work

AsOf

AsOf checks each policy claim in an AI assistant's answer against the policy version that was in force on the customer's date, stops answers that contradict it, and keeps a tamper-evident record of every decision.

0 of 65 wrong drafts released on blind sets
Role
Solo build
Context
IBM Db2Hack 2026, 2nd place of 5,000+ participants, November 2026
Stack
IBM Db2 12.1, SQL PL, LangGraph, LangChain, Python
Outcome
1,331 tests. 0 of 65 wrong drafts released on blind test sets.
Customer's date
10 March 2026
AI draft, the same on every date

“Yes, you can still apply for a bereavement fare after your trip.”

Policy in force on that date: BRV-002 v1

Drag the date. The draft never changes, but the right answer does. Simplified from AsOf's demo data, a fictional airline's customer service plan.

In action

The problem

In 2024 a Canadian tribunal held Air Canada liable after its website chatbot told a grieving customer he could claim a bereavement fare after travelling. The airline's own policy said the opposite (Moffatt v. Air Canada, 2024 BCCRT 149).

Every company running an AI assistant has the same exposure, and it gets worse with time. Policies change, and when a complaint arrives months later, nobody can show which version of the policy the assistant should have been answering from.

Checking an answer against today's policy is not enough. It has to be right for the customer's date.

What I built

  • Policy clauses live in a bitemporal Db2 table. Business time records when a version applies. System time records when the database learned about it, so any past answer can be replayed against what was known at that moment.
  • One SQL statement finds the clauses in force on the customer's date and ranks them by meaning, using Db2's native vector search in the same query as the date filter.
  • A LangGraph pipeline with fixed edges checks every claim in the draft. A contradicted answer is rewritten from the policy text or held for a person. If any step fails, the answer is not released.
  • Every decision is appended to a ledger whose rows are hash-chained with SHA-256 inside Db2. Triggers block edits and deletes, even by the table owner.
  • It plugs in wherever answers are written: an API, a LangChain runnable, a chat page and an MCP tool.
SELECT c.CLAUSE_ID, c.VERSION_LABEL, c.CLAUSE_TEXT, c.RULE_JSON,
       VECTOR_DISTANCE(c.EMBEDDING, VECTOR(?, 384, FLOAT32), COSINE) AS DIST
FROM POLICY_CLAUSE FOR BUSINESS_TIME AS OF ? AS c
WHERE c.DOC_ID = ?
ORDER BY DIST FETCH EXACT FIRST ? ROWS ONLY
The retrieval at the centre of the check: clauses in force on the customer's date, ranked by meaning.

Results

  • On the five frozen test sets that never motivated a fix, it released 0 of 65 wrong drafts (95% upper bound about 4.6%).
  • Across all nine frozen sets in the latest run, it released 1 of 121 wrong drafts and stopped 4 of 70 correct ones.
  • Checked against today's policy instead of the customer's date, the same pipeline released 6 of 16 date-sensitive wrong drafts. With the customer's date, none.
  • On a policy written by another AI that it had never seen, it made the right call on 30 of 40 drafts. A plain LLM judge managed 6 of 40.
  • 1,331 tests at 80% line coverage, with CI running against IBM's stock Db2 image.
  • Along the way I documented a Db2 defect with a one-command reproduction: a forced vector-index scan under a date predicate fails with SQL0901N.

Limits

Four of the nine test sets motivated a fix at some point, so they are not blind. The number I stand behind is the 0 of 65 on the five that never did.

Verifying the full hash chain takes 46 minutes at one million decisions. Checkpointed verification reads only the rows since the last checkpoint, and the full scan is kept for audits.

The policies are two demo domains: an airline's customer service plan and a bank's fee schedule. It has not yet run on a real company's policy history.

See it