1. What was built
The prototype models an AI-assisted support centre with six predefined customer scenarios and a free-form testing playground. It classifies an inquiry, assigns category and priority, extracts entities, retrieves a verified FAQ, proposes a response and decides between automated guidance and a simulated human queue.
The browser client is built with React and Vite. The public deployment uses Netlify Functions for the analysis endpoint. Gemini can assist the classification, while a deterministic engine provides a transparent fallback. Both paths must return the same normalized response schema. Demo tickets and FAQ edits remain in the visitor’s browser and are automatically cleared after 24 hours.
2. The real testing problem: defining an oracle
Fluent text is not a pass criterion. Each guided scenario therefore received an explicit contract: expected category, priority, route, FAQ identifier, required entities and prohibited entities. This turns an open-ended AI response into something testable without pretending that every free-form customer message has one perfect answer.
Risk-based coverage focused on state isolation, conflicting truth sources, low-confidence decisions, safety wording, entity semantics, input boundaries, dashboard metrics and deployment claims.
3. What the first audit found
| Finding | Why it mattered | Initial status |
|---|---|---|
| Sarah was addressed as “Marcus”; Elena as “Robert” | Cross-request state contamination is a trust and privacy defect. | Major |
| Scenario card said Automated FAQ while runtime used Human Queue | Two sources of truth produced contradictory behavior. | Major |
| Low FAQ match displayed both automated and human decisions | A user could not tell which route was actually selected. | Major |
| Simulation promised a manager would call | The demo implied a real human action that could not occur. | Major |
| Express API was described as Netlify-ready | A static deploy would not automatically host the backend. | Major |
| Escalation KPI and status filter used different semantics | Summary numbers could not be reconciled with detailed records. | Minor |
Negative tests also covered empty and whitespace input, HTML/XSS-like strings, low FAQ relevance and long transcripts. The tested XSS-like input rendered as text, but that single check was not presented as comprehensive security proof.
4. How the system was corrected
- Customer identity was bound to the active request and previous names were forbidden in the prompt and normalized response.
- One normalized decision object became the source for UI badges, routing, FAQ explanation and ticket creation.
- All six guided scenarios received automated contracts instead of isolated hard-coded fixes.
- Entity rules separate order and invoice IDs from product SKUs and omit absent values instead of rendering raw
null. - Input is limited to 4,000 characters in the UI and function, with an explicit 400 response.
- Escalation language is hypothetical and visibly labelled as a simulation.
- The Express-only path was complemented by Netlify Functions, redirects, request validation, rate limiting, deterministic fallback and server-only Gemini credentials.
- Browser data now has automatic 24-hour expiry, plus manual Reset Demo.
5. Regression was a separate product phase
The first regression confirmed several fixes but reopened Delivery routing and found missing order extraction, raw null values, a 4,001-character boundary failure, an inaccurate localStorage statement and a badge mismatch. The second focused round closed most of those issues but detected a new regression: the damaged-item refund had drifted from Billing and Payments to Delivery, and the SKU appeared as an order number.
The third round verified the corrected normalized contracts in different scenarios. All six passed. This chronology is intentionally preserved: the audit report is not overwritten by the later PASS result, because the defect history is part of the evidence.
| Scenario | Expected contract | Result |
|---|---|---|
| Delayed Delivery | Delivery · Medium · Automated FAQ · US-98214 | PASS |
| Damaged Item Refund | Billing and Payments · High · Human Queue · SKU-4401 · $89.50 | PASS |
| Duplicate Billing | Billing and Payments · High · Human Queue · INV-7731 · $49.00 | PASS |
| Mobile App 503 | Technical Support · High · Automated troubleshooting | PASS |
| Angry Complaint | Complaints · Critical · Human Queue · Manager Escalation | PASS |
| Support Hours | General Questions · Low · Automated FAQ · no invented entities | PASS |
6. Deployment is part of QA
A successful AI Studio preview does not prove a Netlify build. The release gate therefore checks the actual Functions routes, Gemini success and deterministic fallback, HTTP 400 and 429 behavior, CSP and security headers, absence of secrets in client assets, browser-data expiry, cold-start behavior and desktop/mobile rendering.
The Gemini key is read only inside the serverless function. The frontend never receives it. CORS is not opened with a wildcard because the application calls its API on the same origin.
7. What this demonstration does not claim
- It is not a real customer-service operation and does not contact agents or process refunds.
- The in-memory function limiter is suitable only for a low-traffic educational demo, not a distributed production service.
- Microphone start/stop was tested; full speech-to-text accuracy was not certified with a controlled audio corpus.
- Guided scenario contracts stabilize demonstrations. Free-form playground inputs remain intentionally open to evaluation.
8. Reusable lessons for AI testing
- Test the same request after every other scenario to expose stale context.
- Assert contradictions across fields, not only whether each field looks plausible.
- Separate model judgment from deterministic business and safety rules.
- Treat safety copy, persistence statements and deployment claims as testable behavior.
- Keep the original failures and later regression evidence separate.
Try the controlled demonstration
Use fictional data and observe the normalized decision, matched FAQ, entity extraction, route and execution engine. The interface clearly identifies Gemini-assisted and deterministic modes.
Open AI Customer Support Center ↗
Leave a comment
Share your thoughts about the case study. No account is required.
Comments
Loading comments…