Case Study
Enterprise SaaS
AI-Powered
Submission Automation
Redesigned an insurance underwriting platform — replacing 100% outsourced data entry with an intelligent OCR + LLM pipeline. 1,000+ daily submissions processed at 90% accuracy. Vendor eliminated Feb 2025.
0+
Daily submissions
processed automatically
0%
Field accuracy
extracted from PDFs
0.2×
Faster quotes
time-to-quote reduced
$0.4M
Annual savings
vendor + headcount reduction
01
The Problem
A pipeline that couldn't scale
Insurance underwriters were drowning in manual work. Each submission required downloading PDFs, manually extracting data into spreadsheets, and cross-referencing multiple documents. The $240K/year vendor contract was growing — not shrinking.
01
Broker emails PDF
Email protocol
16–72 hours per submission
02
Team forwards to vendor
Patra Corp, India
Manual handoff delays
03
Manual transcription
6–12 hour turnaround
High error rates — 15–20% fields wrong
04
No review layer
Underwriters trusted the data
Errors discovered late → 2–3 day delays
The Core Challenge
How might we eliminate manual data entry while maintaining accuracy and building trust in AI-extracted information? The system needed to handle edge cases, provide transparency, and give underwriters control.
02
Research & Discovery
Understanding the human workflow
We conducted 12 contextual inquiries with underwriters, shadowed the vendor team for 3 weeks, and mapped the entire submission lifecycle to identify friction points and automation opportunities.
User Interviews
Trust is earned through transparency
Need to verify AI outputs quickly
Manual review takes 2–3 hours per submission
Workflow Analysis
67% of time spent on data entry
23% on cross-referencing documents
10% on actual underwriting decisions
Pain Points
No confidence scores on extracted data
Can't trace data back to source
Errors discovered too late in process
Key Insights
01
Transparency builds trust
Underwriters needed to see where data came from and how confident the AI was in its extraction. Showing the work was non-negotiable.
02
Progressive disclosure
Show high-confidence data first, flag low-confidence fields for review, hide technical details unless needed. Reduce cognitive load without hiding complexity.
03
Maintain control
AI should assist, not replace. Underwriters wanted the ability to override, edit, and provide feedback — every correction feeds the retraining pipeline.
03
Design Process
11 months, end-to-end
No PM until month 6. I owned discovery, framing, prototyping, specs, and post-launch iteration.
Mar–Jul '24Discovery
Shadowed underwriters for 3 weeks. Mapped 47 failure points. 40+ user interviews with ops, underwriters, and ML teams.
Jul–Oct '24Problem Framing
Defined confidence scoring UX requirements. Wrote opportunity brief for ML and leadership. Set the 90% accuracy target.
Oct–Nov '24Concept & Prototype
2 design sprints. 6 concepts tested with 12 underwriters. Key insight: users wanted to review, not retype.
Nov–Jan '25Iterative Build
Weekly agile sprints. One surface per sprint. Close ML collaboration on confidence signal design and exception thresholds.
Feb '25Launch & Measure
Phased rollout to 3 teams. Hit 90% accuracy target in week 4. Patra vendor offboarded. $2.4M annual savings realized.
04
The Solution
Five surfaces, one pipeline
Each surface addresses a specific friction point in the old workflow. All tested with underwriters and ops leads before engineering handoff. Ordered by user journey — from inbox through to extraction.
01
Smart categorization
Emails auto-tagged by type: Failed, Processed, On Hold, Incomplete
Batch operations
Select multiple emails for bulk processing and status updates
Quick filters
Search and filter by sender, status, date range, and submission type
02
Structured data
All extracted information organized by category with confidence scores
Quick validation
One-click approve or flag for review with inline editing capability
Audit trail
Track all changes and AI decisions for compliance and retraining
03
Split-view interface
PDF on left, extracted fields on right with confidence indicators
Visual highlighting
Click any field to see source location highlighted in the document
Confidence scores
Green (90%+), Yellow (60–90%), Red (<60%) with inline edit capability
04
Semantic search
Ask questions in plain language, get instant answers from documents
Source citations
Every answer links back to specific pages in the source document
Multi-document
Search across all submission documents simultaneously in one query
05
Auto-categorization
Claims, BOR changes, new submissions automatically sorted at intake
Smart extraction
Pull policy numbers, dates, and key info directly from email body
Status tracking
Visual indicators for processing state and AI confidence per email
Five patterns that made AI trustworthy
Confidence-driven UI
A 3-tier system (Auto / Review / Manual) based on OCR quality. Reduced cognitive load 60% in testing by removing noise from the reviewer's view.
Explainable extractions
90% of submissions flow through automatically. Reviewers only see fields below the confidence threshold — the 10% that actually needs human attention.
Progressive disclosure
Show-only complex metadata when needed. Complex mapping fields surface only when confidence is borderline — not as default noise in every review.
Audit-friendly by default
Audit is a first-class surface — embedded inline wherever AI makes a decision, not buried in a separate admin panel disconnected from the work.
05
Impact
What we delivered
Launched Feb 2025. Within 4 weeks, the system was processing 1,000+ submissions daily with 90% accuracy. The vendor was offboarded, saving $2.4M annually.
0%
Field extraction accuracy
Exceeds 85% manual baseline
Positive impact
0.2×
Faster time-to-quote
8–12 hrs → under 15 min
Positive impact
$2.4M
Projected annual savings
Vendor + headcount reallocated
Positive impact
0+
Daily submissions
Processed via AI pipeline
Positive impact
0%
Less manual review
Underwriters focus on decisions
Positive impact
0
Post-launch incidents
End-to-end QA, no rollback needed
Qualitative Impact
"This is the first AI tool I actually trust. I can see exactly where the data came from and fix it if needed."
— Senior Underwriter
"We went from spending 70% of our time on data entry to focusing on actual risk assessment. Game changer."
— Underwriting Manager
"The confidence scoring is brilliant. I only review what needs attention, everything else flows through automatically."
— Operations Lead
06
Reflections
What I'd tell myself on day one
Design for trust before efficiency
Initially I optimized for speed — auto-filling everything. Users rejected it. They needed to see the AI's work and verify it. Trust takes time to build, especially with high-stakes decisions.
Escalate design to business strategy
Setting the confidence threshold required alignment across ML, product, and legal — it's a risk decision, not just a UI pattern. I had to make the business case for every design choice.
Bring in the PM earlier
Six PM-less months meant I was doing product strategy on top of design. The collaboration would've moved faster with a dedicated PM from day one to own stakeholder alignment.
Future opportunity: Proactive insights
Right now the system is reactive. Next iteration: surface risk patterns, suggest similar submissions, predict underwriting decisions. Move from automation to intelligence.
Key Takeaways
AI transparency isn't optional — it's the foundation of trust in high-stakes workflows
Progressive disclosure reduces cognitive load while maintaining full user control
Design systems for AI need confidence scoring, source tracing, and override capabilities
Cross-functional alignment on AI behavior is as important as the interface itself
