AI-Assisted Workflow Platform

Our healthcare SaaS platform supported 50+ clinics and thousands of patients, but the operations team had become the system bottleneck. They spent most of their day reconciling insurance eligibility responses, investigating claim discrepancies, and resolving escalations that required cross-referencing data across multiple internal systems.

Ticket resolution was slow enough to block downstream work — customer onboarding slowed because operations could not keep pace; signed contracts were waiting on implementation. Leadership asked me to design a solution that would multiply operational capacity without increasing headcount — and without weakening our HIPAA or SOC2 posture.

This was not an AI feature.
It was an operational leverage problem inside a regulated environment.

• HIPAA requirements: All PHI access required encryption at rest and in transit, audit logging, and strict role-based access control. No PHI could be sent to external models without de-identification or contractual coverage.
• SOC2 Type II compliance: AI model access had to follow the same controls as any production data path — scoped IAM roles, request logging, and quarterly access reviews.
• LLM hallucination risk: In a regulated environment, the system could suggest but never auto-apply changes. Every output required human approval before touching production data.
• Operational scale: ~15K jobs/month with 4–8 second LLM latency per request.
• Budget constraint: Must run on existing AWS infrastructure. No new vendor security review cycles.
• Non-goals: Not customer-facing. Not replacing the ops team. Not full automation.

The objective was leverage — not autonomy.

Design principle: decouple AI latency and uncertainty from user workflows while preserving auditability and human control.

I designed an asynchronous, human-in-the-loop pipeline built on infrastructure we already operated. Each ops ticket was decomposed into a structured job, analyzed by an LLM, and returned as a suggested resolution for human review.

Key components:

• Async job processing — SQS + Lambda decoupled the UI from LLM latency. Jobs were idempotent via deterministic hashes to prevent duplicate suggestions during retries.
• LLM service layer — A thin TypeScript wrapper with exponential backoff (3 retries), circuit breaker on repeated 5xx failures, and tier fallback between model classes. This isolated model volatility from production systems.
• Human-in-the-loop approval — Suggestions stored in PostgreSQL with pending status. Ops users reviewed, edited, approved, or rejected. Only approved suggestions triggered downstream writes.
• Audit trail — Every LLM request/response pair, approval decision, and data mutation was written to S3 as immutable, partitioned JSON logs, feeding directly into our SOC2 evidence pipeline.

AI drafted. Humans decided.

Model-tier routing (illustrative)

The router classified each job and picked a model tier — cheap models for high-volume reconciliation, capable models for multi-source escalations. Fallback was triggered by repeated 5xx or schema-validation failures, not by latency alone.

// model-router.ts — classify the job, pick the right tier, fall back on failure.
type Complexity = "simple" | "complex";
type Tier = { id: "haiku" | "sonnet"; maxRetries: number };

const TIERS: Record<Complexity, Tier> = {
  simple:  { id: "haiku",  maxRetries: 2 },
  complex: { id: "sonnet", maxRetries: 3 },
};

function classify(job: OpsJob): Complexity {
  if (job.requiresCrossSystemLookup) return "complex";
  if (job.escalationLevel >= 2)      return "complex";
  return "simple";
}

export async function runJob(job: OpsJob): Promise<Suggestion> {
  const tier = TIERS[classify(job)];
  try {
    return await llm.invoke(tier.id, job, { maxRetries: tier.maxRetries });
  } catch (err) {
    if (tier.id === "haiku" && isTransientOrSchemaError(err)) {
      // One-step escalation; the audit log records both attempts.
      return llm.invoke("sonnet", job, { maxRetries: 1 });
    }
    throw err;
  }
}

Sync vs. async — I chose async processing via SQS and Lambda. LLM calls averaged 4–8 seconds, and blocking the UI would have degraded operator workflow. Async introduced state tracking complexity, but it improved UX and throughput.

Build vs. buy — Training a custom model was evaluated but rejected due to cost, ramp time, and ongoing ML operations overhead. Managed model APIs provided sufficient reasoning quality for our use case within acceptable cost bounds.

AI autonomy boundaries — Full automation was technically possible but operationally unacceptable. Every suggestion required human approval. This added workflow latency but eliminated the risk of hallucinated production writes — an unacceptable compliance risk.

Cost optimization — A routing layer classified job complexity. Simple tasks were routed to lower-cost models; complex reconciliation tasks used higher-capability models. This reduced monthly LLM spend substantially without measurable degradation in quality.

• Resolution time: Median case resolution moved from multi-day turnaround to same-day for AI-suggested tickets.
• SLA reliability: Backlog growth flattened; the team stopped losing ground week over week.
• Ops leverage: The same operations team now handles more volume without a headcount increase.
• Revenue impact: Onboarding work that had been queued behind ops capacity began clearing within the first quarter of rollout.
• Compliance posture: The AI pipeline runs inside the same HIPAA / SOC2 Type II controls as the rest of the platform — immutable audit logging, scoped IAM, and human-in-the-loop approval on every write. No AI-generated data has reached production without human review.

What worked — The human-in-the-loop boundary was the most important architectural decision. In production use, no AI-generated data has reached customers without explicit review. Adoption was high because operators remained in control.

What I'd improve — Observability lagged initial rollout. Per-workflow token cost tracking and prompt version management should have been first-class from day one. Retrofitting them increased operational overhead.

Future direction — Extend the pattern into selectively customer-facing workflows with stricter output validation schemas, tighter token budgets, and formalized prompt lifecycle management.