Categorization eval
437 reviewed lifetime
overall approval
77.3%
338 of 437 kept
latest (categorizer.v2.2)
79.5%
280 of 352
reviewed
437
lifetime, source='human' only
Per-category approval (worst first)
Routing-effectiveness
High kept % = routing reason is over-flagging (human always confirms). Low kept % = routing reason is doing real work.
| Reason | n routed | kept | kept % |
|---|---|---|---|
| high_materiality | 311 | 255 | 82% |
| anomaly | 88 | 59 | 67% |
| low_confidence | 38 | 24 | 63.2% |
Confidence calibration
Well-calibrated if kept % ≈ bucket midpoint (e.g., 0.7–0.8 bucket ≈75% kept).
| Confidence range | n | kept | kept % |
|---|---|---|---|
| [0.3, 0.4) | 7 | 4 | 57.1% |
| [0.4, 0.5) | 16 | 8 | 50% |
| [0.5, 0.6) | 30 | 17 | 56.7% |
| [0.6, 0.7) | 15 | 11 | 73.3% |
| [0.7, 0.8) | 47 | 35 | 74.5% |
| [0.8, 0.9) | 56 | 45 | 80.4% |
| [0.9, 1.0) | 266 | 218 | 82% |
Confusion (top 30 disagreements)
| Agent said | Human said | n |
|---|---|---|
| income_business | income_w2 | 35 |
| reimbursement | health | 9 |
| health | discretionary_other | 9 |
| travel | gifts | 6 |
| other | childcare | 4 |
| other | dining | 3 |
| income_w2 | income_business | 3 |
| housing | health | 3 |
| health | fitness | 2 |
| kids_discretionary | discretionary_other | 2 |
| kids_essentials | kids_discretionary | 2 |
| dining | groceries | 2 |
| returns | discretionary_other | 1 |
| housing | childcare | 1 |
| reimbursement | income_business | 1 |
| discretionary_other | kids_discretionary | 1 |
| kids_essentials | other | 1 |
| other | health | 1 |
| transport | dining | 1 |
| other | fitness | 1 |
| kids_discretionary | dining | 1 |
| transport | fitness | 1 |
| health | dining | 1 |
| discretionary_other | gifts | 1 |
| fitness | discretionary_other | 1 |
| income_business | other | 1 |
| discretionary_other | groceries | 1 |
| renovation | housing | 1 |
| housing | other | 1 |
| kids_discretionary | childcare | 1 |
Per-merchant (top 20)
| Merchant | n | kept | approval % |
|---|---|---|---|
| Poppins Payroll Purchase Ccd | 124 | 124 | 100% |
| 148376 Ramp Busi Dir Dep 35 | 35 | 0 | 0% |
| Check | 31 | 31 | 100% |
| Parkslopefoodcoo 782 U Brooklyn Ny | 19 | 19 | 100% |
| Park West Tenant | 15 | 15 | 100% |
| Zelle Payment To Monalisa Baston | 8 | 8 | 100% |
| Www.gadventures.com | 6 | 0 | 0% |
| Robert Kaye-walsh Lcsw | 6 | 6 | 100% |
| Crossfit South Brookly | 5 | 1 | 20% |
| Atm Withdrawal | 4 | 0 | 0% |
| Rescue Spa | 4 | 0 | 0% |
| Park West Tenant Purchase | 4 | 4 | 100% |
| New York Presbyterian | 3 | 3 | 100% |
| Park Slope Dental Arts | 3 | 3 | 100% |
| Venmo | 3 | 2 | 66.7% |
| Poppins Payroll | 3 | 3 | 100% |
| Nutrafol | 3 | 0 | 0% |
| Sp Magnetic-me | 3 | 3 | 100% |
Per agent version
| Version | n reviewed | kept | approval % |
|---|---|---|---|
| categorizer.v1 | 29 | 18 | 62.1% |
| categorizer.v2 | 42 | 29 | 69% |
| categorizer.v2.1 | 14 | 11 | 78.6% |
| categorizer.v2.2 | 352 | 280 | 79.5% |