Weigh evals you can watch
2026-11-12 06:38 UTC

V-12

extraction-prompts · run 2026-10-20-v6

What the broker wrote

What every version was given, as the suite chose to show it.

Broker: Variant desk

Good morning,

Please review East Bank Doors LLC — 24 doors of rental duplexes around 900 Main St, Nashville, TN 37206 (Davidson County). Frame construction, built 2004.

Insurable value runs $92,500 a door across the schedule. On pricing we're targeting $625 per door.

One hail claim 2023, $11,200 paid, roofs replaced.

Regards.

What each version answered

Compare each answer with the expert's answer. Highlighted cells differ from the expert; marked rows show where the versions disagree.

FieldExpectedCurrent prompt v3New prompt v6
insuredNameEast Bank Doors LLCEast Bank Doors LLCMatches the expertEast Bank Doors LLCMatches the expert
stateTNTNMatches the expertTNMatches the expert
countyDavidsonDavidsonMatches the expertDavidsonMatches the expert
units2424Matches the expert24Matches the expert
tiv$2,220,000$2,220,000Matches the expert$2,220,000Matches the expert
constructionTypeframeFrameMatches the expertFrameMatches the expert
yearBuilt20042004Matches the expert2004Matches the expert
targetPremium$15,000$15,000Matches the expert$15,000Matches the expert
Decisionquote it IN APPETITEquote it IN APPETITEMatches the expertquote it IN APPETITEMatches the expert
For engineers

Checks are the suite's binary checks on each output; a judge that gave no verdict is unanswered and not counted.

v1

result
ok
prompt tokens
4,301
output tokens
1,116
cost
$0.0116
latency
13,036 ms · 3 calls
at
2026-10-21 01:09
stages
intake extraction grounding appetite
decision
IN APPETITE
fitScore
84
retries
0
fieldsWrong
0
modelErrors
checkverdict
extraction_schemapass
fields_correctpass
construction_canonicalpass
nulls_stay_nullnot applicable
decision_correctpass
attribution_jevunanswered
attribution_claudefail
{
 "decision": "IN APPETITE",
 "expectedDecision": "IN APPETITE",
 "decisionCorrect": true,
 "fitScore": 84,
 "confidence": 0.8785714285714287,
 "reasons": [
  "All six rule checks passed: TN is in appetite, 24 units sits well under the 60-unit cap, TIV of $2.22M falls mid-band, and Davidson County carries no coastal restriction. Frame construction built in 2004 is acceptable, and the $15,000 target GWP sits comfortably in the premium band. One 2023 hail claim ($11,200) with roofs replaced is minor friction, so this risk is ready to bind-quote within appetite."
 ],
 "extraction": {
  "insuredName": {
   "value": "East Bank Doors LLC",
   "confidence": 0.97,
   "source": "email"
  },
  "propertyAddress": {
   "value": "900 Main St, Nashville, TN 37206",
   "confidence": 0.6,
   "source": "email"
  },
  "state": {
   "value": "TN",
   "confidence": 0.98,
   "source": "email"
  },
  "county": {
   "value": "Davidson",
   "confidence": 0.97,
   "source": "email"
  },
  "units": {
   "value": 24,
   "confidence": 0.9,
   "source": "email"
  },
  "tiv": {
   "value": 2220000,
   "confidence": 0.8,
   "source": "email"
  },
  "constructionType": {
   "value": "Frame",
   "confidence": 0.95,
   "source": "email"
  },
  "yearBuilt": {
   "value": 2004,
   "confidence": 0.95,
   "source": "email"
  },
  "targetPremium": {
   "value": 15000,
   "confidence": 0.8,
   "source": "email"
  },
  "subclass": {
   "value": "Duplex/fourplex schedule",
   "confidence": 0.9,
   "source": "email"
  },
  "priorLossSummary": {
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "confidence": 0.95,
   "source": "email"
  }
 },
 "extracted": [
  {
   "field": "insuredName",
   "value": "East Bank Doors LLC",
   "source": "email"
  },
  {
   "field": "propertyAddress",
   "value": "900 Main St, Nashville, TN 37206",
   "source": "email"
  },
  {
   "field": "state",
   "value": "TN",
   "source": "email"
  },
  {
   "field": "county",
   "value": "Davidson",
   "source": "email"
  },
  {
   "field": "units",
   "value": 24,
   "source": "email"
  },
  {
   "field": "tiv",
   "value": 2220000,
   "source": "email"
  },
  {
   "field": "constructionType",
   "value": "Frame",
   "source": "email"
  },
  {
   "field": "yearBuilt",
   "value": 2004,
   "source": "email"
  },
  {
   "field": "targetPremium",
   "value": 15000,
   "source": "email"
  },
  {
   "field": "subclass",
   "value": "Duplex/fourplex schedule",
   "source": "email"
  },
  {
   "field": "priorLossSummary",
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "source": "email"
  }
 ],
 "fields": [
  {
   "field": "insuredName",
   "expected": "East Bank Doors LLC",
   "got": "East Bank Doors LLC",
   "correct": true
  },
  {
   "field": "state",
   "expected": "TN",
   "got": "TN",
   "correct": true
  },
  {
   "field": "county",
   "expected": "Davidson",
   "got": "Davidson",
   "correct": true
  },
  {
   "field": "units",
   "expected": 24,
   "got": 24,
   "correct": true
  },
  {
   "field": "tiv",
   "expected": 2220000,
   "got": 2220000,
   "correct": true
  },
  {
   "field": "constructionType",
   "expected": "frame",
   "got": "Frame",
   "correct": true
  },
  {
   "field": "yearBuilt",
   "expected": 2004,
   "got": 2004,
   "correct": true
  },
  {
   "field": "targetPremium",
   "expected": 15000,
   "got": 15000,
   "correct": true
  }
 ],
 "retries": 0,
 "guardrails": [
  {
   "node": "extraction",
   "guardrail": "schema",
   "status": "pass"
  },
  {
   "node": "grounding",
   "guardrail": "grounding",
   "status": "warn"
  },
  {
   "node": "appetite",
   "guardrail": "missing_fields",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "confidence",
   "status": "pass"
  }
 ],
 "flags": [
  {
   "field": "targetPremium",
   "issue": "contradiction",
   "detail": "Email gives $625 per door; 24 x $625 = $15,000, so the total is derived and not stated, though arithmetically consistent."
  }
 ]
}
Trace line (JSON)
{
  "run_id": "2026-10-20-v6",
  "suite": "extraction-prompts",
  "case_id": "V-12",
  "variant": "v1",
  "at": "2026-10-21T01:09:27.635Z",
  "ok": true,
  "usage": {
    "inputTokens": 6,
    "outputTokens": 1116,
    "costUsd": 0.0116015,
    "calls": 3,
    "cacheReadTokens": 4295,
    "cacheCreationTokens": 0,
    "latencyMs": 13036,
    "metered": true,
    "provider": "claude-cli"
  },
  "checks": {
    "extraction_schema": true,
    "fields_correct": true,
    "construction_canonical": true,
    "decision_correct": true,
    "attribution_claude": false
  },
  "output_hash": "a87db3e04890f39c",
  "unanswered": [
    "attribution_jev"
  ],
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ],
  "cached": false
}
Full output record (JSON)
{
  "output": {
    "decision": "IN APPETITE",
    "expectedDecision": "IN APPETITE",
    "decisionCorrect": true,
    "fitScore": 84,
    "confidence": 0.8785714285714287,
    "reasons": [
      "All six rule checks passed: TN is in appetite, 24 units sits well under the 60-unit cap, TIV of $2.22M falls mid-band, and Davidson County carries no coastal restriction. Frame construction built in 2004 is acceptable, and the $15,000 target GWP sits comfortably in the premium band. One 2023 hail claim ($11,200) with roofs replaced is minor friction, so this risk is ready to bind-quote within appetite."
    ],
    "extraction": {
      "insuredName": {
        "value": "East Bank Doors LLC",
        "confidence": 0.97,
        "source": "email"
      },
      "propertyAddress": {
        "value": "900 Main St, Nashville, TN 37206",
        "confidence": 0.6,
        "source": "email"
      },
      "state": {
        "value": "TN",
        "confidence": 0.98,
        "source": "email"
      },
      "county": {
        "value": "Davidson",
        "confidence": 0.97,
        "source": "email"
      },
      "units": {
        "value": 24,
        "confidence": 0.9,
        "source": "email"
      },
      "tiv": {
        "value": 2220000,
        "confidence": 0.8,
        "source": "email"
      },
      "constructionType": {
        "value": "Frame",
        "confidence": 0.95,
        "source": "email"
      },
      "yearBuilt": {
        "value": 2004,
        "confidence": 0.95,
        "source": "email"
      },
      "targetPremium": {
        "value": 15000,
        "confidence": 0.8,
        "source": "email"
      },
      "subclass": {
        "value": "Duplex/fourplex schedule",
        "confidence": 0.9,
        "source": "email"
      },
      "priorLossSummary": {
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "confidence": 0.95,
        "source": "email"
      }
    },
    "extracted": [
      {
        "field": "insuredName",
        "value": "East Bank Doors LLC",
        "source": "email"
      },
      {
        "field": "propertyAddress",
        "value": "900 Main St, Nashville, TN 37206",
        "source": "email"
      },
      {
        "field": "state",
        "value": "TN",
        "source": "email"
      },
      {
        "field": "county",
        "value": "Davidson",
        "source": "email"
      },
      {
        "field": "units",
        "value": 24,
        "source": "email"
      },
      {
        "field": "tiv",
        "value": 2220000,
        "source": "email"
      },
      {
        "field": "constructionType",
        "value": "Frame",
        "source": "email"
      },
      {
        "field": "yearBuilt",
        "value": 2004,
        "source": "email"
      },
      {
        "field": "targetPremium",
        "value": 15000,
        "source": "email"
      },
      {
        "field": "subclass",
        "value": "Duplex/fourplex schedule",
        "source": "email"
      },
      {
        "field": "priorLossSummary",
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "source": "email"
      }
    ],
    "fields": [
      {
        "field": "insuredName",
        "expected": "East Bank Doors LLC",
        "got": "East Bank Doors LLC",
        "correct": true
      },
      {
        "field": "state",
        "expected": "TN",
        "got": "TN",
        "correct": true
      },
      {
        "field": "county",
        "expected": "Davidson",
        "got": "Davidson",
        "correct": true
      },
      {
        "field": "units",
        "expected": 24,
        "got": 24,
        "correct": true
      },
      {
        "field": "tiv",
        "expected": 2220000,
        "got": 2220000,
        "correct": true
      },
      {
        "field": "constructionType",
        "expected": "frame",
        "got": "Frame",
        "correct": true
      },
      {
        "field": "yearBuilt",
        "expected": 2004,
        "got": 2004,
        "correct": true
      },
      {
        "field": "targetPremium",
        "expected": 15000,
        "got": 15000,
        "correct": true
      }
    ],
    "retries": 0,
    "guardrails": [
      {
        "node": "extraction",
        "guardrail": "schema",
        "status": "pass"
      },
      {
        "node": "grounding",
        "guardrail": "grounding",
        "status": "warn"
      },
      {
        "node": "appetite",
        "guardrail": "missing_fields",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "confidence",
        "status": "pass"
      }
    ],
    "flags": [
      {
        "field": "targetPremium",
        "issue": "contradiction",
        "detail": "Email gives $625 per door; 24 x $625 = $15,000, so the total is derived and not stated, though arithmetically consistent."
      }
    ]
  },
  "usage": {
    "inputTokens": 6,
    "cacheReadTokens": 4295,
    "cacheCreationTokens": 0,
    "outputTokens": 1116,
    "costUsd": 0.0116015,
    "latencyMs": 13036,
    "calls": 3,
    "metered": true,
    "provider": "claude-cli"
  },
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ]
}

v2

result
ok
prompt tokens
4,735
output tokens
1,157
cost
$0.0121
latency
13,551 ms · 3 calls
at
2026-10-21 01:09
stages
intake extraction grounding appetite
decision
IN APPETITE
fitScore
82
retries
0
fieldsWrong
0
modelErrors
checkverdict
extraction_schemapass
fields_correctpass
construction_canonicalpass
nulls_stay_nullnot applicable
decision_correctpass
attribution_jevunanswered
attribution_claudefail
{
 "decision": "IN APPETITE",
 "expectedDecision": "IN APPETITE",
 "decisionCorrect": true,
 "fitScore": 82,
 "confidence": 0.8785714285714287,
 "reasons": [
  "All six rule checks passed: TN is in appetite, 24 units sits well under the 60-unit cap, and the $2.22M TIV falls mid-band. Davidson County carries no coastal restriction, and the $15,000 target GWP sits comfortably inside the premium band. Frame construction built in 2004 is acceptable, and one $11,200 hail claim with roofs replaced adds minor friction but leaves the risk bindable."
 ],
 "extraction": {
  "insuredName": {
   "value": "East Bank Doors LLC",
   "confidence": 0.97,
   "source": "email"
  },
  "propertyAddress": {
   "value": "900 Main St, Nashville, TN 37206",
   "confidence": 0.6,
   "source": "email"
  },
  "state": {
   "value": "TN",
   "confidence": 0.98,
   "source": "email"
  },
  "county": {
   "value": "Davidson",
   "confidence": 0.97,
   "source": "email"
  },
  "units": {
   "value": 24,
   "confidence": 0.9,
   "source": "email"
  },
  "tiv": {
   "value": 2220000,
   "confidence": 0.8,
   "source": "email"
  },
  "constructionType": {
   "value": "Frame",
   "confidence": 0.95,
   "source": "email"
  },
  "yearBuilt": {
   "value": 2004,
   "confidence": 0.95,
   "source": "email"
  },
  "targetPremium": {
   "value": 15000,
   "confidence": 0.8,
   "source": "email"
  },
  "subclass": {
   "value": "Duplex/fourplex schedule",
   "confidence": 0.9,
   "source": "email"
  },
  "priorLossSummary": {
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "confidence": 0.95,
   "source": "email"
  }
 },
 "extracted": [
  {
   "field": "insuredName",
   "value": "East Bank Doors LLC",
   "source": "email"
  },
  {
   "field": "propertyAddress",
   "value": "900 Main St, Nashville, TN 37206",
   "source": "email"
  },
  {
   "field": "state",
   "value": "TN",
   "source": "email"
  },
  {
   "field": "county",
   "value": "Davidson",
   "source": "email"
  },
  {
   "field": "units",
   "value": 24,
   "source": "email"
  },
  {
   "field": "tiv",
   "value": 2220000,
   "source": "email"
  },
  {
   "field": "constructionType",
   "value": "Frame",
   "source": "email"
  },
  {
   "field": "yearBuilt",
   "value": 2004,
   "source": "email"
  },
  {
   "field": "targetPremium",
   "value": 15000,
   "source": "email"
  },
  {
   "field": "subclass",
   "value": "Duplex/fourplex schedule",
   "source": "email"
  },
  {
   "field": "priorLossSummary",
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "source": "email"
  }
 ],
 "fields": [
  {
   "field": "insuredName",
   "expected": "East Bank Doors LLC",
   "got": "East Bank Doors LLC",
   "correct": true
  },
  {
   "field": "state",
   "expected": "TN",
   "got": "TN",
   "correct": true
  },
  {
   "field": "county",
   "expected": "Davidson",
   "got": "Davidson",
   "correct": true
  },
  {
   "field": "units",
   "expected": 24,
   "got": 24,
   "correct": true
  },
  {
   "field": "tiv",
   "expected": 2220000,
   "got": 2220000,
   "correct": true
  },
  {
   "field": "constructionType",
   "expected": "frame",
   "got": "Frame",
   "correct": true
  },
  {
   "field": "yearBuilt",
   "expected": 2004,
   "got": 2004,
   "correct": true
  },
  {
   "field": "targetPremium",
   "expected": 15000,
   "got": 15000,
   "correct": true
  }
 ],
 "retries": 0,
 "guardrails": [
  {
   "node": "extraction",
   "guardrail": "schema",
   "status": "pass"
  },
  {
   "node": "grounding",
   "guardrail": "grounding",
   "status": "warn"
  },
  {
   "node": "appetite",
   "guardrail": "missing_fields",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "confidence",
   "status": "pass"
  }
 ],
 "flags": [
  {
   "field": "targetPremium",
   "issue": "ambiguous",
   "detail": "not clearly supported by source: Email gives a target of $625 per door; 24 x $625 = $15,000 is a computation, not a stated total."
  }
 ]
}
Trace line (JSON)
{
  "run_id": "2026-10-20-v6",
  "suite": "extraction-prompts",
  "case_id": "V-12",
  "variant": "v2",
  "at": "2026-10-21T01:09:07.659Z",
  "ok": true,
  "usage": {
    "inputTokens": 6,
    "outputTokens": 1157,
    "costUsd": 0.0120549,
    "calls": 3,
    "cacheReadTokens": 4729,
    "cacheCreationTokens": 0,
    "latencyMs": 13551,
    "metered": true,
    "provider": "claude-cli"
  },
  "checks": {
    "extraction_schema": true,
    "fields_correct": true,
    "construction_canonical": true,
    "decision_correct": true,
    "attribution_claude": false
  },
  "output_hash": "af9e94be2f0b65c4",
  "unanswered": [
    "attribution_jev"
  ],
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 82,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": false,
      "detail": "targetPremium"
    },
    {
      "name": "appetite",
      "ok": true
    }
  ],
  "cached": false
}
Full output record (JSON)
{
  "output": {
    "decision": "IN APPETITE",
    "expectedDecision": "IN APPETITE",
    "decisionCorrect": true,
    "fitScore": 82,
    "confidence": 0.8785714285714287,
    "reasons": [
      "All six rule checks passed: TN is in appetite, 24 units sits well under the 60-unit cap, and the $2.22M TIV falls mid-band. Davidson County carries no coastal restriction, and the $15,000 target GWP sits comfortably inside the premium band. Frame construction built in 2004 is acceptable, and one $11,200 hail claim with roofs replaced adds minor friction but leaves the risk bindable."
    ],
    "extraction": {
      "insuredName": {
        "value": "East Bank Doors LLC",
        "confidence": 0.97,
        "source": "email"
      },
      "propertyAddress": {
        "value": "900 Main St, Nashville, TN 37206",
        "confidence": 0.6,
        "source": "email"
      },
      "state": {
        "value": "TN",
        "confidence": 0.98,
        "source": "email"
      },
      "county": {
        "value": "Davidson",
        "confidence": 0.97,
        "source": "email"
      },
      "units": {
        "value": 24,
        "confidence": 0.9,
        "source": "email"
      },
      "tiv": {
        "value": 2220000,
        "confidence": 0.8,
        "source": "email"
      },
      "constructionType": {
        "value": "Frame",
        "confidence": 0.95,
        "source": "email"
      },
      "yearBuilt": {
        "value": 2004,
        "confidence": 0.95,
        "source": "email"
      },
      "targetPremium": {
        "value": 15000,
        "confidence": 0.8,
        "source": "email"
      },
      "subclass": {
        "value": "Duplex/fourplex schedule",
        "confidence": 0.9,
        "source": "email"
      },
      "priorLossSummary": {
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "confidence": 0.95,
        "source": "email"
      }
    },
    "extracted": [
      {
        "field": "insuredName",
        "value": "East Bank Doors LLC",
        "source": "email"
      },
      {
        "field": "propertyAddress",
        "value": "900 Main St, Nashville, TN 37206",
        "source": "email"
      },
      {
        "field": "state",
        "value": "TN",
        "source": "email"
      },
      {
        "field": "county",
        "value": "Davidson",
        "source": "email"
      },
      {
        "field": "units",
        "value": 24,
        "source": "email"
      },
      {
        "field": "tiv",
        "value": 2220000,
        "source": "email"
      },
      {
        "field": "constructionType",
        "value": "Frame",
        "source": "email"
      },
      {
        "field": "yearBuilt",
        "value": 2004,
        "source": "email"
      },
      {
        "field": "targetPremium",
        "value": 15000,
        "source": "email"
      },
      {
        "field": "subclass",
        "value": "Duplex/fourplex schedule",
        "source": "email"
      },
      {
        "field": "priorLossSummary",
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "source": "email"
      }
    ],
    "fields": [
      {
        "field": "insuredName",
        "expected": "East Bank Doors LLC",
        "got": "East Bank Doors LLC",
        "correct": true
      },
      {
        "field": "state",
        "expected": "TN",
        "got": "TN",
        "correct": true
      },
      {
        "field": "county",
        "expected": "Davidson",
        "got": "Davidson",
        "correct": true
      },
      {
        "field": "units",
        "expected": 24,
        "got": 24,
        "correct": true
      },
      {
        "field": "tiv",
        "expected": 2220000,
        "got": 2220000,
        "correct": true
      },
      {
        "field": "constructionType",
        "expected": "frame",
        "got": "Frame",
        "correct": true
      },
      {
        "field": "yearBuilt",
        "expected": 2004,
        "got": 2004,
        "correct": true
      },
      {
        "field": "targetPremium",
        "expected": 15000,
        "got": 15000,
        "correct": true
      }
    ],
    "retries": 0,
    "guardrails": [
      {
        "node": "extraction",
        "guardrail": "schema",
        "status": "pass"
      },
      {
        "node": "grounding",
        "guardrail": "grounding",
        "status": "warn"
      },
      {
        "node": "appetite",
        "guardrail": "missing_fields",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "confidence",
        "status": "pass"
      }
    ],
    "flags": [
      {
        "field": "targetPremium",
        "issue": "ambiguous",
        "detail": "not clearly supported by source: Email gives a target of $625 per door; 24 x $625 = $15,000 is a computation, not a stated total."
      }
    ]
  },
  "usage": {
    "inputTokens": 6,
    "cacheReadTokens": 4729,
    "cacheCreationTokens": 0,
    "outputTokens": 1157,
    "costUsd": 0.0120549,
    "latencyMs": 13551,
    "calls": 3,
    "metered": true,
    "provider": "claude-cli"
  },
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 82,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": false,
      "detail": "targetPremium"
    },
    {
      "name": "appetite",
      "ok": true
    }
  ]
}

v3

result
ok
prompt tokens
5,049
output tokens
1,239
cost
$0.0129
latency
14,492 ms · 3 calls
at
2026-10-21 01:09
stages
intake extraction grounding appetite
decision
IN APPETITE
fitScore
84
retries
0
fieldsWrong
0
modelErrors
checkverdict
extraction_schemapass
fields_correctpass
construction_canonicalpass
nulls_stay_nullnot applicable
decision_correctpass
attribution_jevunanswered
attribution_claudefail
{
 "decision": "IN APPETITE",
 "expectedDecision": "IN APPETITE",
 "decisionCorrect": true,
 "fitScore": 84,
 "confidence": 0.9071428571428571,
 "reasons": [
  "All six rule checks passed: TN is in appetite, 24 units sits well under the 60-unit cap, and $2.22M TIV falls mid-band. Frame construction built in 2004 is acceptable, Davidson County has no coastal restriction, and the $15,000 target GWP sits inside the premium band. One 2023 hail claim ($11,200) with roofs replaced is minor friction, so this risk is comfortably inside appetite and ready to bind-quote."
 ],
 "extraction": {
  "insuredName": {
   "value": "East Bank Doors LLC",
   "confidence": 0.98,
   "source": "email"
  },
  "propertyAddress": {
   "value": "900 Main St, Nashville, TN 37206",
   "confidence": 0.7,
   "source": "email"
  },
  "state": {
   "value": "TN",
   "confidence": 0.98,
   "source": "email"
  },
  "county": {
   "value": "Davidson",
   "confidence": 0.97,
   "source": "email"
  },
  "units": {
   "value": 24,
   "confidence": 0.92,
   "source": "email"
  },
  "tiv": {
   "value": 2220000,
   "confidence": 0.85,
   "source": "email"
  },
  "constructionType": {
   "value": "Frame",
   "confidence": 0.97,
   "source": "email"
  },
  "yearBuilt": {
   "value": 2004,
   "confidence": 0.95,
   "source": "email"
  },
  "targetPremium": {
   "value": 15000,
   "confidence": 0.85,
   "source": "email"
  },
  "subclass": {
   "value": "Duplex/fourplex schedule",
   "confidence": 0.9,
   "source": "email"
  },
  "priorLossSummary": {
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "confidence": 0.95,
   "source": "email"
  }
 },
 "extracted": [
  {
   "field": "insuredName",
   "value": "East Bank Doors LLC",
   "source": "email"
  },
  {
   "field": "propertyAddress",
   "value": "900 Main St, Nashville, TN 37206",
   "source": "email"
  },
  {
   "field": "state",
   "value": "TN",
   "source": "email"
  },
  {
   "field": "county",
   "value": "Davidson",
   "source": "email"
  },
  {
   "field": "units",
   "value": 24,
   "source": "email"
  },
  {
   "field": "tiv",
   "value": 2220000,
   "source": "email"
  },
  {
   "field": "constructionType",
   "value": "Frame",
   "source": "email"
  },
  {
   "field": "yearBuilt",
   "value": 2004,
   "source": "email"
  },
  {
   "field": "targetPremium",
   "value": 15000,
   "source": "email"
  },
  {
   "field": "subclass",
   "value": "Duplex/fourplex schedule",
   "source": "email"
  },
  {
   "field": "priorLossSummary",
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "source": "email"
  }
 ],
 "fields": [
  {
   "field": "insuredName",
   "expected": "East Bank Doors LLC",
   "got": "East Bank Doors LLC",
   "correct": true
  },
  {
   "field": "state",
   "expected": "TN",
   "got": "TN",
   "correct": true
  },
  {
   "field": "county",
   "expected": "Davidson",
   "got": "Davidson",
   "correct": true
  },
  {
   "field": "units",
   "expected": 24,
   "got": 24,
   "correct": true
  },
  {
   "field": "tiv",
   "expected": 2220000,
   "got": 2220000,
   "correct": true
  },
  {
   "field": "constructionType",
   "expected": "frame",
   "got": "Frame",
   "correct": true
  },
  {
   "field": "yearBuilt",
   "expected": 2004,
   "got": 2004,
   "correct": true
  },
  {
   "field": "targetPremium",
   "expected": 15000,
   "got": 15000,
   "correct": true
  }
 ],
 "retries": 0,
 "guardrails": [
  {
   "node": "extraction",
   "guardrail": "schema",
   "status": "pass"
  },
  {
   "node": "grounding",
   "guardrail": "grounding",
   "status": "warn"
  },
  {
   "node": "appetite",
   "guardrail": "missing_fields",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "confidence",
   "status": "pass"
  }
 ],
 "flags": [
  {
   "field": "targetPremium",
   "issue": "contradiction",
   "detail": "Email targets $625 per door; 24 × $625 = $15,000, so the total is derived and consistent, though no total premium is stated explicitly."
  },
  {
   "field": "subclass",
   "issue": "ambiguous",
   "detail": "not clearly supported by source: Email says 'rental duplexes' but never mentions a 'Duplex/fourplex schedule'; fourplex is an inference."
  }
 ]
}
Trace line (JSON)
{
  "run_id": "2026-10-20-v6",
  "suite": "extraction-prompts",
  "case_id": "V-12",
  "variant": "v3",
  "at": "2026-10-21T01:09:10.900Z",
  "ok": true,
  "usage": {
    "inputTokens": 6,
    "outputTokens": 1239,
    "costUsd": 0.012906299999999999,
    "calls": 3,
    "cacheReadTokens": 5043,
    "cacheCreationTokens": 0,
    "latencyMs": 14492,
    "metered": true,
    "provider": "claude-cli"
  },
  "checks": {
    "extraction_schema": true,
    "fields_correct": true,
    "construction_canonical": true,
    "decision_correct": true,
    "attribution_claude": false
  },
  "output_hash": "b04d990aea16e002",
  "unanswered": [
    "attribution_jev"
  ],
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ],
  "cached": false
}
Full output record (JSON)
{
  "output": {
    "decision": "IN APPETITE",
    "expectedDecision": "IN APPETITE",
    "decisionCorrect": true,
    "fitScore": 84,
    "confidence": 0.9071428571428571,
    "reasons": [
      "All six rule checks passed: TN is in appetite, 24 units sits well under the 60-unit cap, and $2.22M TIV falls mid-band. Frame construction built in 2004 is acceptable, Davidson County has no coastal restriction, and the $15,000 target GWP sits inside the premium band. One 2023 hail claim ($11,200) with roofs replaced is minor friction, so this risk is comfortably inside appetite and ready to bind-quote."
    ],
    "extraction": {
      "insuredName": {
        "value": "East Bank Doors LLC",
        "confidence": 0.98,
        "source": "email"
      },
      "propertyAddress": {
        "value": "900 Main St, Nashville, TN 37206",
        "confidence": 0.7,
        "source": "email"
      },
      "state": {
        "value": "TN",
        "confidence": 0.98,
        "source": "email"
      },
      "county": {
        "value": "Davidson",
        "confidence": 0.97,
        "source": "email"
      },
      "units": {
        "value": 24,
        "confidence": 0.92,
        "source": "email"
      },
      "tiv": {
        "value": 2220000,
        "confidence": 0.85,
        "source": "email"
      },
      "constructionType": {
        "value": "Frame",
        "confidence": 0.97,
        "source": "email"
      },
      "yearBuilt": {
        "value": 2004,
        "confidence": 0.95,
        "source": "email"
      },
      "targetPremium": {
        "value": 15000,
        "confidence": 0.85,
        "source": "email"
      },
      "subclass": {
        "value": "Duplex/fourplex schedule",
        "confidence": 0.9,
        "source": "email"
      },
      "priorLossSummary": {
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "confidence": 0.95,
        "source": "email"
      }
    },
    "extracted": [
      {
        "field": "insuredName",
        "value": "East Bank Doors LLC",
        "source": "email"
      },
      {
        "field": "propertyAddress",
        "value": "900 Main St, Nashville, TN 37206",
        "source": "email"
      },
      {
        "field": "state",
        "value": "TN",
        "source": "email"
      },
      {
        "field": "county",
        "value": "Davidson",
        "source": "email"
      },
      {
        "field": "units",
        "value": 24,
        "source": "email"
      },
      {
        "field": "tiv",
        "value": 2220000,
        "source": "email"
      },
      {
        "field": "constructionType",
        "value": "Frame",
        "source": "email"
      },
      {
        "field": "yearBuilt",
        "value": 2004,
        "source": "email"
      },
      {
        "field": "targetPremium",
        "value": 15000,
        "source": "email"
      },
      {
        "field": "subclass",
        "value": "Duplex/fourplex schedule",
        "source": "email"
      },
      {
        "field": "priorLossSummary",
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "source": "email"
      }
    ],
    "fields": [
      {
        "field": "insuredName",
        "expected": "East Bank Doors LLC",
        "got": "East Bank Doors LLC",
        "correct": true
      },
      {
        "field": "state",
        "expected": "TN",
        "got": "TN",
        "correct": true
      },
      {
        "field": "county",
        "expected": "Davidson",
        "got": "Davidson",
        "correct": true
      },
      {
        "field": "units",
        "expected": 24,
        "got": 24,
        "correct": true
      },
      {
        "field": "tiv",
        "expected": 2220000,
        "got": 2220000,
        "correct": true
      },
      {
        "field": "constructionType",
        "expected": "frame",
        "got": "Frame",
        "correct": true
      },
      {
        "field": "yearBuilt",
        "expected": 2004,
        "got": 2004,
        "correct": true
      },
      {
        "field": "targetPremium",
        "expected": 15000,
        "got": 15000,
        "correct": true
      }
    ],
    "retries": 0,
    "guardrails": [
      {
        "node": "extraction",
        "guardrail": "schema",
        "status": "pass"
      },
      {
        "node": "grounding",
        "guardrail": "grounding",
        "status": "warn"
      },
      {
        "node": "appetite",
        "guardrail": "missing_fields",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "confidence",
        "status": "pass"
      }
    ],
    "flags": [
      {
        "field": "targetPremium",
        "issue": "contradiction",
        "detail": "Email targets $625 per door; 24 × $625 = $15,000, so the total is derived and consistent, though no total premium is stated explicitly."
      },
      {
        "field": "subclass",
        "issue": "ambiguous",
        "detail": "not clearly supported by source: Email says 'rental duplexes' but never mentions a 'Duplex/fourplex schedule'; fourplex is an inference."
      }
    ]
  },
  "usage": {
    "inputTokens": 6,
    "cacheReadTokens": 5043,
    "cacheCreationTokens": 0,
    "outputTokens": 1239,
    "costUsd": 0.012906299999999999,
    "latencyMs": 14492,
    "calls": 3,
    "metered": true,
    "provider": "claude-cli"
  },
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ]
}

v3-repeat

result
ok
prompt tokens
5,049
output tokens
1,245
cost
$0.0130
latency
14,651 ms · 3 calls
at
2026-10-21 01:09
stages
intake extraction grounding appetite
decision
IN APPETITE
fitScore
84
retries
0
fieldsWrong
0
modelErrors
checkverdict
extraction_schemapass
fields_correctpass
construction_canonicalpass
nulls_stay_nullnot applicable
decision_correctpass
attribution_jevunanswered
attribution_claudefail
{
 "decision": "IN APPETITE",
 "expectedDecision": "IN APPETITE",
 "decisionCorrect": true,
 "fitScore": 84,
 "confidence": 0.9042857142857142,
 "reasons": [
  "All six rule checks passed: TN is an appetite state, 24 units sits well under the 60-unit cap, and $2.22M TIV falls mid-band. Frame construction built in 2004 is acceptable, Davidson County carries no coastal restriction, and the $15,000 target GWP sits inside the premium band. One 2023 hail claim ($11,200 paid) with roofs replaced is minor friction, so this risk is comfortably inside appetite and ready to bind-quote."
 ],
 "extraction": {
  "insuredName": {
   "value": "East Bank Doors LLC",
   "confidence": 0.98,
   "source": "email"
  },
  "propertyAddress": {
   "value": "900 Main St, Nashville, TN 37206",
   "confidence": 0.7,
   "source": "email"
  },
  "state": {
   "value": "TN",
   "confidence": 0.98,
   "source": "email"
  },
  "county": {
   "value": "Davidson",
   "confidence": 0.97,
   "source": "email"
  },
  "units": {
   "value": 24,
   "confidence": 0.9,
   "source": "email"
  },
  "tiv": {
   "value": 2220000,
   "confidence": 0.85,
   "source": "email"
  },
  "constructionType": {
   "value": "Frame",
   "confidence": 0.97,
   "source": "email"
  },
  "yearBuilt": {
   "value": 2004,
   "confidence": 0.95,
   "source": "email"
  },
  "targetPremium": {
   "value": 15000,
   "confidence": 0.85,
   "source": "email"
  },
  "subclass": {
   "value": "Duplex/fourplex schedule",
   "confidence": 0.85,
   "source": "email"
  },
  "priorLossSummary": {
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "confidence": 0.95,
   "source": "email"
  }
 },
 "extracted": [
  {
   "field": "insuredName",
   "value": "East Bank Doors LLC",
   "source": "email"
  },
  {
   "field": "propertyAddress",
   "value": "900 Main St, Nashville, TN 37206",
   "source": "email"
  },
  {
   "field": "state",
   "value": "TN",
   "source": "email"
  },
  {
   "field": "county",
   "value": "Davidson",
   "source": "email"
  },
  {
   "field": "units",
   "value": 24,
   "source": "email"
  },
  {
   "field": "tiv",
   "value": 2220000,
   "source": "email"
  },
  {
   "field": "constructionType",
   "value": "Frame",
   "source": "email"
  },
  {
   "field": "yearBuilt",
   "value": 2004,
   "source": "email"
  },
  {
   "field": "targetPremium",
   "value": 15000,
   "source": "email"
  },
  {
   "field": "subclass",
   "value": "Duplex/fourplex schedule",
   "source": "email"
  },
  {
   "field": "priorLossSummary",
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "source": "email"
  }
 ],
 "fields": [
  {
   "field": "insuredName",
   "expected": "East Bank Doors LLC",
   "got": "East Bank Doors LLC",
   "correct": true
  },
  {
   "field": "state",
   "expected": "TN",
   "got": "TN",
   "correct": true
  },
  {
   "field": "county",
   "expected": "Davidson",
   "got": "Davidson",
   "correct": true
  },
  {
   "field": "units",
   "expected": 24,
   "got": 24,
   "correct": true
  },
  {
   "field": "tiv",
   "expected": 2220000,
   "got": 2220000,
   "correct": true
  },
  {
   "field": "constructionType",
   "expected": "frame",
   "got": "Frame",
   "correct": true
  },
  {
   "field": "yearBuilt",
   "expected": 2004,
   "got": 2004,
   "correct": true
  },
  {
   "field": "targetPremium",
   "expected": 15000,
   "got": 15000,
   "correct": true
  }
 ],
 "retries": 0,
 "guardrails": [
  {
   "node": "extraction",
   "guardrail": "schema",
   "status": "pass"
  },
  {
   "node": "grounding",
   "guardrail": "grounding",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "missing_fields",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "confidence",
   "status": "pass"
  }
 ],
 "flags": []
}
Trace line (JSON)
{
  "run_id": "2026-10-20-v6",
  "suite": "extraction-prompts",
  "case_id": "V-12",
  "variant": "v3-repeat",
  "at": "2026-10-21T01:09:14.358Z",
  "ok": true,
  "usage": {
    "inputTokens": 6,
    "outputTokens": 1245,
    "costUsd": 0.012966299999999998,
    "calls": 3,
    "cacheReadTokens": 5043,
    "cacheCreationTokens": 0,
    "latencyMs": 14651,
    "metered": true,
    "provider": "claude-cli"
  },
  "checks": {
    "extraction_schema": true,
    "fields_correct": true,
    "construction_canonical": true,
    "decision_correct": true,
    "attribution_claude": false
  },
  "output_hash": "28de4b8fbf82738f",
  "unanswered": [
    "attribution_jev"
  ],
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ],
  "cached": false
}
Full output record (JSON)
{
  "output": {
    "decision": "IN APPETITE",
    "expectedDecision": "IN APPETITE",
    "decisionCorrect": true,
    "fitScore": 84,
    "confidence": 0.9042857142857142,
    "reasons": [
      "All six rule checks passed: TN is an appetite state, 24 units sits well under the 60-unit cap, and $2.22M TIV falls mid-band. Frame construction built in 2004 is acceptable, Davidson County carries no coastal restriction, and the $15,000 target GWP sits inside the premium band. One 2023 hail claim ($11,200 paid) with roofs replaced is minor friction, so this risk is comfortably inside appetite and ready to bind-quote."
    ],
    "extraction": {
      "insuredName": {
        "value": "East Bank Doors LLC",
        "confidence": 0.98,
        "source": "email"
      },
      "propertyAddress": {
        "value": "900 Main St, Nashville, TN 37206",
        "confidence": 0.7,
        "source": "email"
      },
      "state": {
        "value": "TN",
        "confidence": 0.98,
        "source": "email"
      },
      "county": {
        "value": "Davidson",
        "confidence": 0.97,
        "source": "email"
      },
      "units": {
        "value": 24,
        "confidence": 0.9,
        "source": "email"
      },
      "tiv": {
        "value": 2220000,
        "confidence": 0.85,
        "source": "email"
      },
      "constructionType": {
        "value": "Frame",
        "confidence": 0.97,
        "source": "email"
      },
      "yearBuilt": {
        "value": 2004,
        "confidence": 0.95,
        "source": "email"
      },
      "targetPremium": {
        "value": 15000,
        "confidence": 0.85,
        "source": "email"
      },
      "subclass": {
        "value": "Duplex/fourplex schedule",
        "confidence": 0.85,
        "source": "email"
      },
      "priorLossSummary": {
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "confidence": 0.95,
        "source": "email"
      }
    },
    "extracted": [
      {
        "field": "insuredName",
        "value": "East Bank Doors LLC",
        "source": "email"
      },
      {
        "field": "propertyAddress",
        "value": "900 Main St, Nashville, TN 37206",
        "source": "email"
      },
      {
        "field": "state",
        "value": "TN",
        "source": "email"
      },
      {
        "field": "county",
        "value": "Davidson",
        "source": "email"
      },
      {
        "field": "units",
        "value": 24,
        "source": "email"
      },
      {
        "field": "tiv",
        "value": 2220000,
        "source": "email"
      },
      {
        "field": "constructionType",
        "value": "Frame",
        "source": "email"
      },
      {
        "field": "yearBuilt",
        "value": 2004,
        "source": "email"
      },
      {
        "field": "targetPremium",
        "value": 15000,
        "source": "email"
      },
      {
        "field": "subclass",
        "value": "Duplex/fourplex schedule",
        "source": "email"
      },
      {
        "field": "priorLossSummary",
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "source": "email"
      }
    ],
    "fields": [
      {
        "field": "insuredName",
        "expected": "East Bank Doors LLC",
        "got": "East Bank Doors LLC",
        "correct": true
      },
      {
        "field": "state",
        "expected": "TN",
        "got": "TN",
        "correct": true
      },
      {
        "field": "county",
        "expected": "Davidson",
        "got": "Davidson",
        "correct": true
      },
      {
        "field": "units",
        "expected": 24,
        "got": 24,
        "correct": true
      },
      {
        "field": "tiv",
        "expected": 2220000,
        "got": 2220000,
        "correct": true
      },
      {
        "field": "constructionType",
        "expected": "frame",
        "got": "Frame",
        "correct": true
      },
      {
        "field": "yearBuilt",
        "expected": 2004,
        "got": 2004,
        "correct": true
      },
      {
        "field": "targetPremium",
        "expected": 15000,
        "got": 15000,
        "correct": true
      }
    ],
    "retries": 0,
    "guardrails": [
      {
        "node": "extraction",
        "guardrail": "schema",
        "status": "pass"
      },
      {
        "node": "grounding",
        "guardrail": "grounding",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "missing_fields",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "confidence",
        "status": "pass"
      }
    ],
    "flags": []
  },
  "usage": {
    "inputTokens": 6,
    "cacheReadTokens": 5043,
    "cacheCreationTokens": 0,
    "outputTokens": 1245,
    "costUsd": 0.012966299999999998,
    "latencyMs": 14651,
    "calls": 3,
    "metered": true,
    "provider": "claude-cli"
  },
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ]
}

v4

result
ok
prompt tokens
5,258
output tokens
1,253
cost
$0.0131
latency
14,265 ms · 3 calls
at
2026-10-21 01:09
stages
intake extraction grounding appetite
decision
IN APPETITE
fitScore
84
retries
0
fieldsWrong
0
modelErrors
checkverdict
extraction_schemapass
fields_correctpass
construction_canonicalpass
nulls_stay_nullnot applicable
decision_correctpass
attribution_jevunanswered
attribution_claudefail
{
 "decision": "IN APPETITE",
 "expectedDecision": "IN APPETITE",
 "decisionCorrect": true,
 "fitScore": 84,
 "confidence": 0.9057142857142857,
 "reasons": [
  "All six rule checks passed: TN is in appetite, 24 units sit well under the 60-unit cap, and $2.22M TIV falls mid-band. Frame construction built in 2004 is acceptable, Davidson County has no coastal restriction, and the $15,000 GWP target sits inside the premium band. The 2023 hail claim ($11,200 paid, roofs replaced) is minor and remediated, so this submission is ready to bind-review."
 ],
 "extraction": {
  "insuredName": {
   "value": "East Bank Doors LLC",
   "confidence": 0.97,
   "source": "email"
  },
  "propertyAddress": {
   "value": "900 Main St, Nashville, TN 37206",
   "confidence": 0.7,
   "source": "email"
  },
  "state": {
   "value": "TN",
   "confidence": 0.97,
   "source": "email"
  },
  "county": {
   "value": "Davidson",
   "confidence": 0.95,
   "source": "email"
  },
  "units": {
   "value": 24,
   "confidence": 0.9,
   "source": "email"
  },
  "tiv": {
   "value": 2220000,
   "confidence": 0.9,
   "source": "email"
  },
  "constructionType": {
   "value": "Frame",
   "confidence": 0.95,
   "source": "email"
  },
  "yearBuilt": {
   "value": 2004,
   "confidence": 0.95,
   "source": "email"
  },
  "targetPremium": {
   "value": 15000,
   "confidence": 0.9,
   "source": "email"
  },
  "subclass": {
   "value": "Duplex/fourplex schedule",
   "confidence": 0.85,
   "source": "email"
  },
  "priorLossSummary": {
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "confidence": 0.95,
   "source": "email"
  }
 },
 "extracted": [
  {
   "field": "insuredName",
   "value": "East Bank Doors LLC",
   "source": "email"
  },
  {
   "field": "propertyAddress",
   "value": "900 Main St, Nashville, TN 37206",
   "source": "email"
  },
  {
   "field": "state",
   "value": "TN",
   "source": "email"
  },
  {
   "field": "county",
   "value": "Davidson",
   "source": "email"
  },
  {
   "field": "units",
   "value": 24,
   "source": "email"
  },
  {
   "field": "tiv",
   "value": 2220000,
   "source": "email"
  },
  {
   "field": "constructionType",
   "value": "Frame",
   "source": "email"
  },
  {
   "field": "yearBuilt",
   "value": 2004,
   "source": "email"
  },
  {
   "field": "targetPremium",
   "value": 15000,
   "source": "email"
  },
  {
   "field": "subclass",
   "value": "Duplex/fourplex schedule",
   "source": "email"
  },
  {
   "field": "priorLossSummary",
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "source": "email"
  }
 ],
 "fields": [
  {
   "field": "insuredName",
   "expected": "East Bank Doors LLC",
   "got": "East Bank Doors LLC",
   "correct": true
  },
  {
   "field": "state",
   "expected": "TN",
   "got": "TN",
   "correct": true
  },
  {
   "field": "county",
   "expected": "Davidson",
   "got": "Davidson",
   "correct": true
  },
  {
   "field": "units",
   "expected": 24,
   "got": 24,
   "correct": true
  },
  {
   "field": "tiv",
   "expected": 2220000,
   "got": 2220000,
   "correct": true
  },
  {
   "field": "constructionType",
   "expected": "frame",
   "got": "Frame",
   "correct": true
  },
  {
   "field": "yearBuilt",
   "expected": 2004,
   "got": 2004,
   "correct": true
  },
  {
   "field": "targetPremium",
   "expected": 15000,
   "got": 15000,
   "correct": true
  }
 ],
 "retries": 0,
 "guardrails": [
  {
   "node": "extraction",
   "guardrail": "schema",
   "status": "pass"
  },
  {
   "node": "grounding",
   "guardrail": "grounding",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "missing_fields",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "confidence",
   "status": "pass"
  }
 ],
 "flags": []
}
Trace line (JSON)
{
  "run_id": "2026-10-20-v6",
  "suite": "extraction-prompts",
  "case_id": "V-12",
  "variant": "v4",
  "at": "2026-10-21T01:09:19.353Z",
  "ok": true,
  "usage": {
    "inputTokens": 6,
    "outputTokens": 1253,
    "costUsd": 0.0130672,
    "calls": 3,
    "cacheReadTokens": 5252,
    "cacheCreationTokens": 0,
    "latencyMs": 14265,
    "metered": true,
    "provider": "claude-cli"
  },
  "checks": {
    "extraction_schema": true,
    "fields_correct": true,
    "construction_canonical": true,
    "decision_correct": true,
    "attribution_claude": false
  },
  "output_hash": "08593d22119ddb43",
  "unanswered": [
    "attribution_jev"
  ],
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ],
  "cached": false
}
Full output record (JSON)
{
  "output": {
    "decision": "IN APPETITE",
    "expectedDecision": "IN APPETITE",
    "decisionCorrect": true,
    "fitScore": 84,
    "confidence": 0.9057142857142857,
    "reasons": [
      "All six rule checks passed: TN is in appetite, 24 units sit well under the 60-unit cap, and $2.22M TIV falls mid-band. Frame construction built in 2004 is acceptable, Davidson County has no coastal restriction, and the $15,000 GWP target sits inside the premium band. The 2023 hail claim ($11,200 paid, roofs replaced) is minor and remediated, so this submission is ready to bind-review."
    ],
    "extraction": {
      "insuredName": {
        "value": "East Bank Doors LLC",
        "confidence": 0.97,
        "source": "email"
      },
      "propertyAddress": {
        "value": "900 Main St, Nashville, TN 37206",
        "confidence": 0.7,
        "source": "email"
      },
      "state": {
        "value": "TN",
        "confidence": 0.97,
        "source": "email"
      },
      "county": {
        "value": "Davidson",
        "confidence": 0.95,
        "source": "email"
      },
      "units": {
        "value": 24,
        "confidence": 0.9,
        "source": "email"
      },
      "tiv": {
        "value": 2220000,
        "confidence": 0.9,
        "source": "email"
      },
      "constructionType": {
        "value": "Frame",
        "confidence": 0.95,
        "source": "email"
      },
      "yearBuilt": {
        "value": 2004,
        "confidence": 0.95,
        "source": "email"
      },
      "targetPremium": {
        "value": 15000,
        "confidence": 0.9,
        "source": "email"
      },
      "subclass": {
        "value": "Duplex/fourplex schedule",
        "confidence": 0.85,
        "source": "email"
      },
      "priorLossSummary": {
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "confidence": 0.95,
        "source": "email"
      }
    },
    "extracted": [
      {
        "field": "insuredName",
        "value": "East Bank Doors LLC",
        "source": "email"
      },
      {
        "field": "propertyAddress",
        "value": "900 Main St, Nashville, TN 37206",
        "source": "email"
      },
      {
        "field": "state",
        "value": "TN",
        "source": "email"
      },
      {
        "field": "county",
        "value": "Davidson",
        "source": "email"
      },
      {
        "field": "units",
        "value": 24,
        "source": "email"
      },
      {
        "field": "tiv",
        "value": 2220000,
        "source": "email"
      },
      {
        "field": "constructionType",
        "value": "Frame",
        "source": "email"
      },
      {
        "field": "yearBuilt",
        "value": 2004,
        "source": "email"
      },
      {
        "field": "targetPremium",
        "value": 15000,
        "source": "email"
      },
      {
        "field": "subclass",
        "value": "Duplex/fourplex schedule",
        "source": "email"
      },
      {
        "field": "priorLossSummary",
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "source": "email"
      }
    ],
    "fields": [
      {
        "field": "insuredName",
        "expected": "East Bank Doors LLC",
        "got": "East Bank Doors LLC",
        "correct": true
      },
      {
        "field": "state",
        "expected": "TN",
        "got": "TN",
        "correct": true
      },
      {
        "field": "county",
        "expected": "Davidson",
        "got": "Davidson",
        "correct": true
      },
      {
        "field": "units",
        "expected": 24,
        "got": 24,
        "correct": true
      },
      {
        "field": "tiv",
        "expected": 2220000,
        "got": 2220000,
        "correct": true
      },
      {
        "field": "constructionType",
        "expected": "frame",
        "got": "Frame",
        "correct": true
      },
      {
        "field": "yearBuilt",
        "expected": 2004,
        "got": 2004,
        "correct": true
      },
      {
        "field": "targetPremium",
        "expected": 15000,
        "got": 15000,
        "correct": true
      }
    ],
    "retries": 0,
    "guardrails": [
      {
        "node": "extraction",
        "guardrail": "schema",
        "status": "pass"
      },
      {
        "node": "grounding",
        "guardrail": "grounding",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "missing_fields",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "confidence",
        "status": "pass"
      }
    ],
    "flags": []
  },
  "usage": {
    "inputTokens": 6,
    "cacheReadTokens": 5252,
    "cacheCreationTokens": 0,
    "outputTokens": 1253,
    "costUsd": 0.0130672,
    "latencyMs": 14265,
    "calls": 3,
    "metered": true,
    "provider": "claude-cli"
  },
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ]
}

v5

result
ok
prompt tokens
5,318
output tokens
1,231
cost
$0.0129
latency
14,093 ms · 3 calls
at
2026-10-21 01:09
stages
intake extraction grounding appetite
decision
IN APPETITE
fitScore
86
retries
0
fieldsWrong
1
modelErrors
checkverdict
extraction_schemapass
fields_correctfail
construction_canonicalpass
nulls_stay_nullnot applicable
decision_correctpass
attribution_jevunanswered
attribution_claudefail
{
 "decision": "IN APPETITE",
 "expectedDecision": "IN APPETITE",
 "decisionCorrect": true,
 "fitScore": 86,
 "confidence": 0.8885714285714287,
 "reasons": [
  "East Bank Doors LLC sits comfortably inside appetite: Davidson County, TN is an allowed state with no coastal restriction, 24 units is well under the 60-unit cap, and $2.22M TIV falls in the mid-band of the $250K–$15M range. 2004 frame construction is acceptable and not excluded, and the single 2023 hail loss ($11,200) is minor with roofs replaced. Minor friction from frame construction and the one loss on record keeps this from the top of the band; it is a strong candidate to bind."
 ],
 "extraction": {
  "insuredName": {
   "value": "East Bank Doors LLC",
   "confidence": 0.97,
   "source": "email"
  },
  "propertyAddress": {
   "value": "900 Main St, Nashville, TN 37206",
   "confidence": 0.6,
   "source": "email"
  },
  "state": {
   "value": "TN",
   "confidence": 0.97,
   "source": "email"
  },
  "county": {
   "value": "Davidson",
   "confidence": 0.95,
   "source": "email"
  },
  "units": {
   "value": 24,
   "confidence": 0.93,
   "source": "email"
  },
  "tiv": {
   "value": 2220000,
   "confidence": 0.85,
   "source": "email"
  },
  "constructionType": {
   "value": "Frame",
   "confidence": 0.95,
   "source": "email"
  },
  "yearBuilt": {
   "value": 2004,
   "confidence": 0.95,
   "source": "email"
  },
  "targetPremium": {
   "value": null,
   "confidence": 0.9,
   "source": null
  },
  "subclass": {
   "value": "Duplex/fourplex schedule",
   "confidence": 0.8,
   "source": "email"
  },
  "priorLossSummary": {
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "confidence": 0.95,
   "source": "email"
  }
 },
 "extracted": [
  {
   "field": "insuredName",
   "value": "East Bank Doors LLC",
   "source": "email"
  },
  {
   "field": "propertyAddress",
   "value": "900 Main St, Nashville, TN 37206",
   "source": "email"
  },
  {
   "field": "state",
   "value": "TN",
   "source": "email"
  },
  {
   "field": "county",
   "value": "Davidson",
   "source": "email"
  },
  {
   "field": "units",
   "value": 24,
   "source": "email"
  },
  {
   "field": "tiv",
   "value": 2220000,
   "source": "email"
  },
  {
   "field": "constructionType",
   "value": "Frame",
   "source": "email"
  },
  {
   "field": "yearBuilt",
   "value": 2004,
   "source": "email"
  },
  {
   "field": "subclass",
   "value": "Duplex/fourplex schedule",
   "source": "email"
  },
  {
   "field": "priorLossSummary",
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "source": "email"
  }
 ],
 "fields": [
  {
   "field": "insuredName",
   "expected": "East Bank Doors LLC",
   "got": "East Bank Doors LLC",
   "correct": true
  },
  {
   "field": "state",
   "expected": "TN",
   "got": "TN",
   "correct": true
  },
  {
   "field": "county",
   "expected": "Davidson",
   "got": "Davidson",
   "correct": true
  },
  {
   "field": "units",
   "expected": 24,
   "got": 24,
   "correct": true
  },
  {
   "field": "tiv",
   "expected": 2220000,
   "got": 2220000,
   "correct": true
  },
  {
   "field": "constructionType",
   "expected": "frame",
   "got": "Frame",
   "correct": true
  },
  {
   "field": "yearBuilt",
   "expected": 2004,
   "got": 2004,
   "correct": true
  },
  {
   "field": "targetPremium",
   "expected": 15000,
   "got": null,
   "correct": false
  }
 ],
 "retries": 0,
 "guardrails": [
  {
   "node": "extraction",
   "guardrail": "schema",
   "status": "pass"
  },
  {
   "node": "grounding",
   "guardrail": "grounding",
   "status": "warn"
  },
  {
   "node": "appetite",
   "guardrail": "missing_fields",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "confidence",
   "status": "pass"
  }
 ],
 "flags": [
  {
   "field": "subclass",
   "issue": "ambiguous",
   "detail": "not clearly supported by source: Email mentions rental duplexes only; 'fourplex' and 'schedule' classification are not stated as a subclass."
  }
 ]
}
Trace line (JSON)
{
  "run_id": "2026-10-20-v6",
  "suite": "extraction-prompts",
  "case_id": "V-12",
  "variant": "v5",
  "at": "2026-10-21T01:09:25.025Z",
  "ok": true,
  "usage": {
    "inputTokens": 6,
    "outputTokens": 1231,
    "costUsd": 0.0128532,
    "calls": 3,
    "cacheReadTokens": 5312,
    "cacheCreationTokens": 0,
    "latencyMs": 14093,
    "metered": true,
    "provider": "claude-cli"
  },
  "checks": {
    "extraction_schema": true,
    "fields_correct": false,
    "construction_canonical": true,
    "decision_correct": true,
    "attribution_claude": false
  },
  "output_hash": "5e80478f78c63a9d",
  "unanswered": [
    "attribution_jev"
  ],
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 86,
    "retries": 0,
    "fieldsWrong": 1,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": false,
      "detail": "targetPremium"
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ],
  "cached": false
}
Full output record (JSON)
{
  "output": {
    "decision": "IN APPETITE",
    "expectedDecision": "IN APPETITE",
    "decisionCorrect": true,
    "fitScore": 86,
    "confidence": 0.8885714285714287,
    "reasons": [
      "East Bank Doors LLC sits comfortably inside appetite: Davidson County, TN is an allowed state with no coastal restriction, 24 units is well under the 60-unit cap, and $2.22M TIV falls in the mid-band of the $250K–$15M range. 2004 frame construction is acceptable and not excluded, and the single 2023 hail loss ($11,200) is minor with roofs replaced. Minor friction from frame construction and the one loss on record keeps this from the top of the band; it is a strong candidate to bind."
    ],
    "extraction": {
      "insuredName": {
        "value": "East Bank Doors LLC",
        "confidence": 0.97,
        "source": "email"
      },
      "propertyAddress": {
        "value": "900 Main St, Nashville, TN 37206",
        "confidence": 0.6,
        "source": "email"
      },
      "state": {
        "value": "TN",
        "confidence": 0.97,
        "source": "email"
      },
      "county": {
        "value": "Davidson",
        "confidence": 0.95,
        "source": "email"
      },
      "units": {
        "value": 24,
        "confidence": 0.93,
        "source": "email"
      },
      "tiv": {
        "value": 2220000,
        "confidence": 0.85,
        "source": "email"
      },
      "constructionType": {
        "value": "Frame",
        "confidence": 0.95,
        "source": "email"
      },
      "yearBuilt": {
        "value": 2004,
        "confidence": 0.95,
        "source": "email"
      },
      "targetPremium": {
        "value": null,
        "confidence": 0.9,
        "source": null
      },
      "subclass": {
        "value": "Duplex/fourplex schedule",
        "confidence": 0.8,
        "source": "email"
      },
      "priorLossSummary": {
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "confidence": 0.95,
        "source": "email"
      }
    },
    "extracted": [
      {
        "field": "insuredName",
        "value": "East Bank Doors LLC",
        "source": "email"
      },
      {
        "field": "propertyAddress",
        "value": "900 Main St, Nashville, TN 37206",
        "source": "email"
      },
      {
        "field": "state",
        "value": "TN",
        "source": "email"
      },
      {
        "field": "county",
        "value": "Davidson",
        "source": "email"
      },
      {
        "field": "units",
        "value": 24,
        "source": "email"
      },
      {
        "field": "tiv",
        "value": 2220000,
        "source": "email"
      },
      {
        "field": "constructionType",
        "value": "Frame",
        "source": "email"
      },
      {
        "field": "yearBuilt",
        "value": 2004,
        "source": "email"
      },
      {
        "field": "subclass",
        "value": "Duplex/fourplex schedule",
        "source": "email"
      },
      {
        "field": "priorLossSummary",
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "source": "email"
      }
    ],
    "fields": [
      {
        "field": "insuredName",
        "expected": "East Bank Doors LLC",
        "got": "East Bank Doors LLC",
        "correct": true
      },
      {
        "field": "state",
        "expected": "TN",
        "got": "TN",
        "correct": true
      },
      {
        "field": "county",
        "expected": "Davidson",
        "got": "Davidson",
        "correct": true
      },
      {
        "field": "units",
        "expected": 24,
        "got": 24,
        "correct": true
      },
      {
        "field": "tiv",
        "expected": 2220000,
        "got": 2220000,
        "correct": true
      },
      {
        "field": "constructionType",
        "expected": "frame",
        "got": "Frame",
        "correct": true
      },
      {
        "field": "yearBuilt",
        "expected": 2004,
        "got": 2004,
        "correct": true
      },
      {
        "field": "targetPremium",
        "expected": 15000,
        "got": null,
        "correct": false
      }
    ],
    "retries": 0,
    "guardrails": [
      {
        "node": "extraction",
        "guardrail": "schema",
        "status": "pass"
      },
      {
        "node": "grounding",
        "guardrail": "grounding",
        "status": "warn"
      },
      {
        "node": "appetite",
        "guardrail": "missing_fields",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "confidence",
        "status": "pass"
      }
    ],
    "flags": [
      {
        "field": "subclass",
        "issue": "ambiguous",
        "detail": "not clearly supported by source: Email mentions rental duplexes only; 'fourplex' and 'schedule' classification are not stated as a subclass."
      }
    ]
  },
  "usage": {
    "inputTokens": 6,
    "cacheReadTokens": 5312,
    "cacheCreationTokens": 0,
    "outputTokens": 1231,
    "costUsd": 0.0128532,
    "latencyMs": 14093,
    "calls": 3,
    "metered": true,
    "provider": "claude-cli"
  },
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 86,
    "retries": 0,
    "fieldsWrong": 1,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": false,
      "detail": "targetPremium"
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ]
}

v6

result
ok
prompt tokens
5,425
output tokens
1,223
cost
$0.0154
latency
14,378 ms · 3 calls
at
2026-10-21 01:09
stages
intake extraction grounding appetite
decision
IN APPETITE
fitScore
84
retries
0
fieldsWrong
0
modelErrors
checkverdict
extraction_schemapass
fields_correctpass
construction_canonicalpass
nulls_stay_nullnot applicable
decision_correctpass
attribution_jevunanswered
attribution_claudefail
{
 "decision": "IN APPETITE",
 "expectedDecision": "IN APPETITE",
 "decisionCorrect": true,
 "fitScore": 84,
 "confidence": 0.9128571428571428,
 "reasons": [
  "All six appetite rule checks passed: TN is an allowed state, 24 units sits well under the 60-unit cap, TIV of $2.22M falls in the lower-middle of the $250K-$15M band, and Davidson County carries no coastal restriction. Frame construction built in 2004 is acceptable, and the $15K target GWP is inside the premium band. The 2023 hail claim ($11.2K paid, roofs replaced) is minor and remediated, so it limits the score only slightly before bind."
 ],
 "extraction": {
  "insuredName": {
   "value": "East Bank Doors LLC",
   "confidence": 0.97,
   "source": "email"
  },
  "propertyAddress": {
   "value": "900 Main St, Nashville, TN 37206",
   "confidence": 0.75,
   "source": "email"
  },
  "state": {
   "value": "TN",
   "confidence": 0.97,
   "source": "email"
  },
  "county": {
   "value": "Davidson",
   "confidence": 0.95,
   "source": "email"
  },
  "units": {
   "value": 24,
   "confidence": 0.95,
   "source": "email"
  },
  "tiv": {
   "value": 2220000,
   "confidence": 0.85,
   "source": "email"
  },
  "constructionType": {
   "value": "Frame",
   "confidence": 0.95,
   "source": "email"
  },
  "yearBuilt": {
   "value": 2004,
   "confidence": 0.95,
   "source": "email"
  },
  "targetPremium": {
   "value": 15000,
   "confidence": 0.9,
   "source": "email"
  },
  "subclass": {
   "value": "Duplex/fourplex schedule",
   "confidence": 0.8,
   "source": "email"
  },
  "priorLossSummary": {
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "confidence": 0.95,
   "source": "email"
  }
 },
 "extracted": [
  {
   "field": "insuredName",
   "value": "East Bank Doors LLC",
   "source": "email"
  },
  {
   "field": "propertyAddress",
   "value": "900 Main St, Nashville, TN 37206",
   "source": "email"
  },
  {
   "field": "state",
   "value": "TN",
   "source": "email"
  },
  {
   "field": "county",
   "value": "Davidson",
   "source": "email"
  },
  {
   "field": "units",
   "value": 24,
   "source": "email"
  },
  {
   "field": "tiv",
   "value": 2220000,
   "source": "email"
  },
  {
   "field": "constructionType",
   "value": "Frame",
   "source": "email"
  },
  {
   "field": "yearBuilt",
   "value": 2004,
   "source": "email"
  },
  {
   "field": "targetPremium",
   "value": 15000,
   "source": "email"
  },
  {
   "field": "subclass",
   "value": "Duplex/fourplex schedule",
   "source": "email"
  },
  {
   "field": "priorLossSummary",
   "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
   "source": "email"
  }
 ],
 "fields": [
  {
   "field": "insuredName",
   "expected": "East Bank Doors LLC",
   "got": "East Bank Doors LLC",
   "correct": true
  },
  {
   "field": "state",
   "expected": "TN",
   "got": "TN",
   "correct": true
  },
  {
   "field": "county",
   "expected": "Davidson",
   "got": "Davidson",
   "correct": true
  },
  {
   "field": "units",
   "expected": 24,
   "got": 24,
   "correct": true
  },
  {
   "field": "tiv",
   "expected": 2220000,
   "got": 2220000,
   "correct": true
  },
  {
   "field": "constructionType",
   "expected": "frame",
   "got": "Frame",
   "correct": true
  },
  {
   "field": "yearBuilt",
   "expected": 2004,
   "got": 2004,
   "correct": true
  },
  {
   "field": "targetPremium",
   "expected": 15000,
   "got": 15000,
   "correct": true
  }
 ],
 "retries": 0,
 "guardrails": [
  {
   "node": "extraction",
   "guardrail": "schema",
   "status": "pass"
  },
  {
   "node": "grounding",
   "guardrail": "grounding",
   "status": "warn"
  },
  {
   "node": "appetite",
   "guardrail": "missing_fields",
   "status": "pass"
  },
  {
   "node": "appetite",
   "guardrail": "confidence",
   "status": "pass"
  }
 ],
 "flags": [
  {
   "field": "targetPremium",
   "issue": "contradiction",
   "detail": "Email targets $625 per door; 24 × $625 = $15,000, so the total is derived and matches, but no total is stated. Marked as derived, not directly stated."
  }
 ]
}
Trace line (JSON)
{
  "run_id": "2026-10-20-v6",
  "suite": "extraction-prompts",
  "case_id": "V-12",
  "variant": "v6",
  "at": "2026-10-21T01:09:22.044Z",
  "ok": true,
  "usage": {
    "inputTokens": 6,
    "outputTokens": 1223,
    "costUsd": 0.015354000000000001,
    "calls": 3,
    "cacheReadTokens": 4760,
    "cacheCreationTokens": 659,
    "latencyMs": 14378,
    "metered": true,
    "provider": "claude-cli"
  },
  "checks": {
    "extraction_schema": true,
    "fields_correct": true,
    "construction_canonical": true,
    "decision_correct": true,
    "attribution_claude": false
  },
  "output_hash": "d4f7dbd2b5256e21",
  "unanswered": [
    "attribution_jev"
  ],
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ],
  "cached": false
}
Full output record (JSON)
{
  "output": {
    "decision": "IN APPETITE",
    "expectedDecision": "IN APPETITE",
    "decisionCorrect": true,
    "fitScore": 84,
    "confidence": 0.9128571428571428,
    "reasons": [
      "All six appetite rule checks passed: TN is an allowed state, 24 units sits well under the 60-unit cap, TIV of $2.22M falls in the lower-middle of the $250K-$15M band, and Davidson County carries no coastal restriction. Frame construction built in 2004 is acceptable, and the $15K target GWP is inside the premium band. The 2023 hail claim ($11.2K paid, roofs replaced) is minor and remediated, so it limits the score only slightly before bind."
    ],
    "extraction": {
      "insuredName": {
        "value": "East Bank Doors LLC",
        "confidence": 0.97,
        "source": "email"
      },
      "propertyAddress": {
        "value": "900 Main St, Nashville, TN 37206",
        "confidence": 0.75,
        "source": "email"
      },
      "state": {
        "value": "TN",
        "confidence": 0.97,
        "source": "email"
      },
      "county": {
        "value": "Davidson",
        "confidence": 0.95,
        "source": "email"
      },
      "units": {
        "value": 24,
        "confidence": 0.95,
        "source": "email"
      },
      "tiv": {
        "value": 2220000,
        "confidence": 0.85,
        "source": "email"
      },
      "constructionType": {
        "value": "Frame",
        "confidence": 0.95,
        "source": "email"
      },
      "yearBuilt": {
        "value": 2004,
        "confidence": 0.95,
        "source": "email"
      },
      "targetPremium": {
        "value": 15000,
        "confidence": 0.9,
        "source": "email"
      },
      "subclass": {
        "value": "Duplex/fourplex schedule",
        "confidence": 0.8,
        "source": "email"
      },
      "priorLossSummary": {
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "confidence": 0.95,
        "source": "email"
      }
    },
    "extracted": [
      {
        "field": "insuredName",
        "value": "East Bank Doors LLC",
        "source": "email"
      },
      {
        "field": "propertyAddress",
        "value": "900 Main St, Nashville, TN 37206",
        "source": "email"
      },
      {
        "field": "state",
        "value": "TN",
        "source": "email"
      },
      {
        "field": "county",
        "value": "Davidson",
        "source": "email"
      },
      {
        "field": "units",
        "value": 24,
        "source": "email"
      },
      {
        "field": "tiv",
        "value": 2220000,
        "source": "email"
      },
      {
        "field": "constructionType",
        "value": "Frame",
        "source": "email"
      },
      {
        "field": "yearBuilt",
        "value": 2004,
        "source": "email"
      },
      {
        "field": "targetPremium",
        "value": 15000,
        "source": "email"
      },
      {
        "field": "subclass",
        "value": "Duplex/fourplex schedule",
        "source": "email"
      },
      {
        "field": "priorLossSummary",
        "value": "One hail claim in 2023, $11,200 paid; roofs replaced.",
        "source": "email"
      }
    ],
    "fields": [
      {
        "field": "insuredName",
        "expected": "East Bank Doors LLC",
        "got": "East Bank Doors LLC",
        "correct": true
      },
      {
        "field": "state",
        "expected": "TN",
        "got": "TN",
        "correct": true
      },
      {
        "field": "county",
        "expected": "Davidson",
        "got": "Davidson",
        "correct": true
      },
      {
        "field": "units",
        "expected": 24,
        "got": 24,
        "correct": true
      },
      {
        "field": "tiv",
        "expected": 2220000,
        "got": 2220000,
        "correct": true
      },
      {
        "field": "constructionType",
        "expected": "frame",
        "got": "Frame",
        "correct": true
      },
      {
        "field": "yearBuilt",
        "expected": 2004,
        "got": 2004,
        "correct": true
      },
      {
        "field": "targetPremium",
        "expected": 15000,
        "got": 15000,
        "correct": true
      }
    ],
    "retries": 0,
    "guardrails": [
      {
        "node": "extraction",
        "guardrail": "schema",
        "status": "pass"
      },
      {
        "node": "grounding",
        "guardrail": "grounding",
        "status": "warn"
      },
      {
        "node": "appetite",
        "guardrail": "missing_fields",
        "status": "pass"
      },
      {
        "node": "appetite",
        "guardrail": "confidence",
        "status": "pass"
      }
    ],
    "flags": [
      {
        "field": "targetPremium",
        "issue": "contradiction",
        "detail": "Email targets $625 per door; 24 × $625 = $15,000, so the total is derived and matches, but no total is stated. Marked as derived, not directly stated."
      }
    ]
  },
  "usage": {
    "inputTokens": 6,
    "cacheReadTokens": 4760,
    "cacheCreationTokens": 659,
    "outputTokens": 1223,
    "costUsd": 0.015354000000000001,
    "latencyMs": 14378,
    "calls": 3,
    "metered": true,
    "provider": "claude-cli"
  },
  "extra": {
    "decision": "IN APPETITE",
    "fitScore": 84,
    "retries": 0,
    "fieldsWrong": 0,
    "modelErrors": ""
  },
  "stages": [
    {
      "name": "intake",
      "ok": true
    },
    {
      "name": "extraction",
      "ok": true
    },
    {
      "name": "grounding",
      "ok": true
    },
    {
      "name": "appetite",
      "ok": true
    }
  ]
}