What AI customer service is, and what it is not
AI customer service means software that reads support requests by email, chat or text, looks up facts in your own systems, and then drafts a reply, sends one, or routes the case to a person. Three jobs hide under that label, and each carries different risk.
Triage
Tag topic and urgency, pull the customer and order record, route to the right queue. Nothing reaches the customer.
Draft
Write a reply from your articles and records for a person to edit and send. This is a common starting point.
Answer within limits
Send replies for ticket types you approved, inside written limits, with a route to a person on every message.
Deflection means the customer did not reach a person; resolution means the problem was solved. HubSpot's documentation notes that deflections do not always indicate resolution, because a visitor may simply leave. The CFPB calls the bad version a doom loop: repeated automated text with no way out to a person.[6],[1]
The best published evidence favors AI that helps staff. In a study of 5,179 support agents, an AI assistant raised issues resolved per hour by 14% on average and by 34% for novice and low-skilled agents, with minimal impact on experienced ones. One company, one tool: a direction, not a forecast for your team.[12]
What to automate, what to assist and what to keep human
This is our default starting position by ticket type. Named sources support the reasoning where they exist; the rest is our judgment. Set your own limits before launch.
| Ticket type | Default | Why (basis) |
|---|---|---|
| Order, booking or delivery status | Automate | The answer sits in a system of record. The CFPB says chatbots may be useful for basic inquiries but grow less effective as problems get complex. |
| How-to and policy questions with a current article | Automate, cite the article | Ground the reply in your own text. A tribunal held an airline responsible for its chatbot's wrong policy answer. |
| Password reset, address or contact change | Automate only after an identity check; otherwise assist | Account changes touch personal data, so verify identity first. Illinois requires reasonable security for records that hold personal information. |
| Refund or credit under your limit | Assist first, automate later if reviews hold up | A money statement from the bot can bind you, as in the tribunal case. Our judgment: relax approval after reading real cases. |
| Refund, credit or exception above the limit | Keep human | The tribunal case turned on a refund statement. The agent gathers facts and drafts; a person decides. |
| Complaint citing a law, dispute, chargeback, lawyer or regulator | Keep human | The CFPB warns that bots may fail to recognize a consumer invoking federal rights. |
| Report of injury, fire or other product hazard | Keep human, route at once | Covered firms must report to the CPSC within 24 hours of getting reportable information. |
| Upset or vulnerable customer; any request for a person | Keep human, hand off at once | The CFPB describes customers stuck without an off-ramp; Qualtrics found half of consumers worry AI will block access to a person. |
| Multi-step troubleshooting, custom quotes, contract terms | Assist: AI drafts, a person decides | Fits the research on AI helping agents: 14% more issues resolved per hour in one study. |
Approval is the control that matters most. Our agents ask before risky steps, log every action, use least-privilege access with a dedicated service account, run on accounts you own and can be switched off. Reading an order and drafting a reply is fine alone; issuing money, changing an account or sending a legal-sounding statement waits for a named person or a limit you set in writing.
Grounding answers in your knowledge base, and keeping it current
A generative model produces plausible text, not verified facts. NIST calls the result confabulation: confidently stated but false content, a natural result of how these models are designed. Grounding means the agent answers only from text you control and says it does not know when that text is silent. Anthropic's documentation lists the same moves (permit an "I don't know", ground answers in quotes, verify with citations) and says they reduce hallucinations but do not eliminate them.[26],[27]
The Air Canada case shows why one source of truth matters. Its chatbot said a bereavement fare could be claimed after travel; the policy page it linked to said the opposite. The tribunal found the airline did not take reasonable care to keep the chatbot accurate and did not accept that customers should cross-check one part of a website against another.[2]
Our working rules for keeping the source current (practice, not a legal requirement):
- One owner per topic. A named person approves refund, shipping, warranty and cancellation text.
- One source of truth. Retire duplicate and outdated pages.
- Every change triggers a test. Rerun a fixed list of real customer questions before the new text goes live.
- Read the misses. Each "I don't know" is either a missing article or a correct refusal.
- Date the policy text, so the current version is the one quoted.
Escalation rules and handoff with context
An escalation rule is a written condition that moves a conversation to a person. A handoff is the transfer and what travels with it. Write both before launch and test them on real transcripts.
- The customer asks for a person, in any wording. The agent never argues.
- Visible frustration or distress. Intercom's help center refers to escalations triggered by Fin's default frustration handling and by workspace escalation rules.
- Money, legal or safety words, or an amount over your limit.
- Two failed attempts, or a question the knowledge base does not cover.
- An account change the agent cannot verify.[4]
What travels with the case matters as much as the trigger. Zendesk's messaging example collects whatever information human agents will need before forwarding the conversation, and outside business hours routes it as an email ticket. Our handoff note carries the customer's goal, what the agent looked up, what was promised (ideally nothing), the trigger, and a draft reply.[11]
Count a good handoff as a success. Salesforce's guidance counts a graceful handoff as a success and failed or unintended escalations against the metric. The CFPB describes the opposite failure: customers looping through automated text with no off-ramp to a person.[10],[1]
- CallerHi, where is my order 48213? It was due Tuesday and I need it for a job on Friday.
- SystemLookup (fictional order system): order 48213 shipped Monday, carrier scan Tuesday evening, delivery expected Thursday. Customer verified by order email.
- AI agentOrder 48213 shipped Monday and the carrier lists delivery for Thursday. I'm sorry it missed Tuesday. Is there anything else I can check for you?
- CallerCancel it and refund me the full $640. I can't wait.
- SystemRule check (fictional): automatic refunds are allowed up to $100. This request is $640. Action: no refund, route to a person, attach summary.
- AI agentA refund of that size needs a person to approve it, so I'm passing you to our billing team now with everything we've covered. I have not cancelled anything or promised anything yet.
- SystemHandoff note: goal is to cancel order 48213 and refund $640. Shipped Monday, ETA Thursday. Trigger: refund above limit. Customer tone: frustrated but polite. Nothing promised. Draft reply attached.
- StaffYou read the note in your help desk, check whether the carrier can still stop the parcel, and reply to the customer by name. You do not ask them to repeat the story.
Browser simulation with fictional data and fixed rules. Not a live AI run and not a customer result.
Channels and accessibility: email, chat, SMS and phone
- Email. The gentlest start: nobody waits live, so the agent drafts and a person approves. Zendesk evaluates an AI email resolution 72 hours after the first email.
- Chat. Fastest to automate and the easiest place for a doom loop. Zendesk's messaging default evaluates after 2 hours, adjustable up to 72, because a customer who leaves has not necessarily been helped.
- SMS. Suits status, scheduling and handoff. Consent and carrier rules for texts are not covered here; we confirm them in the written scope.
- Phone. A separate discipline: see AI voice agents for the technology and AI receptionist for the front-desk role.[5]
A chat widget is part of your website. The Justice Department's ADA guidance (March 18, 2022) says the ADA applies to the goods and services of businesses open to the public, including those offered on the web. It says no regulation sets detailed technical standards for private businesses and treats WCAG as helpful guidance.[14]
WCAG 2.2 criteria that matter to a chat window: keyboard operation (2.1.1), adjustable time limits (2.2.1), 4.5:1 text contrast (1.4.3), consistent placement of help such as chat and chatbots (3.2.6), and status messages (4.1.3), which asks that changes without focus, like a new reply, reach assistive technology. The W3C page does not name chat, so test with a screen reader.[17],[15],[16]
Language is access too. The CFPB notes that limited-syntax chatbots are particularly problematic for people with limited English proficiency. In Chicago, test the agent in the languages your customers write in and keep a route to a person who speaks them.[1]
Measuring quality: first response, resolution, escalation and CSAT
Platforms measure these differently, so write your own definition first.
| Platform | Counted as resolved when | Window and exclusions |
|---|---|---|
| Intercom (Fin) | The customer confirms the answer helped, or leaves without asking for more (an assumed resolution). | Reversed if the customer returns in the same conversation. Not counted: a greeting only, an unanswered clarifying question, a request for a person, frustration escalations. |
| Zendesk (AI agents) | The AI agent resolves the request without escalation, and a language-model check of the text confirms it. | Checked 72 hours after the first email, 2 hours after the first message in messaging by default (up to 72), and at hang-up for voice. |
| HubSpot (Customer Agent) | The agent shared a content source or took an action and no qualifying handoff followed. | Set 72 hours after the last visitor response. Only visitor-started handoffs count, and later events do not change it. |
| Freshworks (Freshservice AI agent) | The AI answered the employee's query without creating a ticket. | Its deflection rate also excludes conversations with negative feedback. |
None of these is an industry standard, and a vendor's resolution rate is not an industry average: one counts silence as success, one waits 72 hours, one adds a model check. For your own reporting use the stricter reading: resolved means the customer did not come back about the same issue.[4],[5],[6]
| Measure | Definition we use | Documentation behind it |
|---|---|---|
| First response time | Time to the first reply that helps. Decide whether an instant bot acknowledgment counts. | Zendesk measures to the first public agent comment and mostly ignores bot actions. |
| Resolution rate | Share of AI-handled conversations solved with no person and no repeat contact. | HubSpot: fully resolved without a qualifying handoff. |
| Escalation rate | Share handed to a person, with the reason recorded. | Salesforce: graceful handoffs count as success, failed ones count against. |
| CSAT | Good ratings divided by rated surveys, reported for AI-only, assisted and human-only conversations. | Zendesk's messaging metric leaves customers who skip the survey out of the denominator. |
Customers' view: a Qualtrics XM Institute survey of more than 20,000 consumers in 14 countries (third quarter of 2025) found that nearly one in five who used AI for customer service saw no benefit, almost four times the failure rate for AI use in general, and that half worry AI will block access to a person. It comes from a software company's research arm and spans many countries: a signal, not a U.S. average.[13]
US and Illinois rules and risks
You answer for what your bot says. In Moffatt v. Air Canada, 2024 BCCRT 149, a British Columbia tribunal rejected the idea that the chatbot was a separate legal entity: it is part of the website, and it makes no difference whether information comes from a static page or a chatbot. The order totaled $812.02. It is a Canadian small-claims decision that does not bind Illinois courts, but it shows reasoning a plaintiff will cite.[2]
AI is not exempt. The FTC's September 25, 2024 announcement says AI gets no exemption from existing law. In one case it alleged a company never tested whether its chatbot matched a human lawyer, with a proposed $193,000 payment. Do not advertise that a bot replaces people unless you can show the test.[3]
Financial services. The CFPB's 2023 spotlight says chatbots must follow the same federal consumer financial laws as the processes they replace and may fail to recognize a consumer invoking federal rights. It is not a regulatory interpretation, but the Bureau says it will hold entities to account.[1]
- Consumer Fraud Act, section 2 (815 ILCS 505/2). Unfair or deceptive acts, including misrepresentation, are unlawful whether or not anyone was in fact misled, and courts consider FTC interpretations. A wrong bot statement about price or policy is the kind of thing it addresses; whether one crosses the line is for your attorney.[18]
- Telling customers it is AI. We did not find an enacted Illinois statute requiring it in support chat. SB 317 would require a clear and conspicuous notice for conversational customer service AI in chat. It passed the Senate 56 to 1 on May 21, 2026 and was re-referred to the House Rules Committee on May 31, 2026 (status file last modified June 16, 2026). A related bill, HB 3021, was re-referred to Rules on April 17, 2026. NIST suggests considering disclosure of generative AI use. Our default is to disclose.[21],[22],[26]
- Transcripts. Under the Personal Information Protection Act, data collectors must keep reasonable security measures, and contracts that disclose an Illinois resident's personal information must require the same (815 ILCS 530/45). That information includes a name with a Social Security, driver's license, account or card number, or medical information (815 ILCS 530/5). In practice: a data contract with your vendor, access limits, and redaction of card and ID numbers. The CFPB also calls chat logs with personal data sensitive.[19],[20],[1]
- Voiceprints. If phone support identifies callers by voice, the Biometric Information Privacy Act lists a voiceprint as a biometric identifier and requires written notice of the purpose and how long the data will be kept and used, and a written release, before collection (740 ILCS 14/10 and 14/15). Details are on AI voice agents.[24],[23]
- Product safety. Manufacturers, importers, distributors and retailers must report reportable consumer-product hazards to the CPSC within 24 hours of getting the information, and no injury is required. A hazard message needs a human the same day.[25]
How an engagement works, and what to prepare
Request and a short call
You describe the overloaded inbox, chat or line. We ask which ticket types cost the most time and which ones worry you.
A written scope
Goal, the tools the agent may touch, the steps that need approval, and how success is measured, using the definitions above.
A pilot on one channel
Triage and drafting for one ticket family, every action logged, a person approving anything risky.
Review real conversations
We read the misses and handoffs, fix the knowledge base and adjust the limits.
Rollout and run
The agent expands only where the numbers and transcripts hold, on accounts you own, with a dedicated service account and least-privilege access. It can be switched off. Self-hosted models are possible when privacy requires it.
Bring these to the first call. None of it needs to be tidy:
- A few months of recent tickets exported from your help desk.
- Your help articles and policy pages, with an owner for each.
- Written limits: refund and credit limits, who approves exceptions, and the words that always go to a person.
- Your channels and hours, including who answers after hours.
- Admin access to your help desk, order system and email, on accounts you own.
- A baseline of first response time, resolution rate and CSAT, measured the way you will measure them later.
When this is not a good fit
- Low volume. If your team answers every message quickly, software adds risk without much relief. Try a shared inbox and templates first.
- No written policies. If refund or cancellation rules contradict each other, the agent repeats the contradiction, as in the Air Canada dispute. Fix the policy first.
- High-stakes advice on health, legal position or finances. Keep it with a licensed person; the agent may only route and summarize.
- Mostly emotional or complex conversations, such as bereavement, injury claims or disputes.
- A plan to hide that it is AI, or to remove the route to a person. We will not build that. The CFPB lists lost access to human support among the main risks of deficient chatbots.
- A fixed savings figure in advance. We do not promise one.[2],[1]
Questions answered
What is AI customer service?
What is the difference between deflection and resolution?
Can AI replace my support team?
Is my company responsible for what a support chatbot says?
Do I have to tell customers they are talking to AI?
What should never be automated?
How do I stop the bot from making things up?
What should a handoff to a person include?
Which metrics should I track?
Can I trust a vendor's resolution rate?
Does a website chat widget have to be accessible?
Do support transcripts fall under Illinois privacy rules?
Where does a Chicago-area company start?
Sources
- Chatbots in consumer finance (issue spotlight, June 2023) (Consumer Financial Protection Bureau), accessed 2026-10-12. Limits of chatbots on complex problems, doom loops, failure to recognize consumer rights, chat logs as sensitive data, language access, legal exposurehttps://files.consumerfinance.gov/f/documents/cfpb_chatbot-issue-spotlight_2023-06.pdf
- Moffatt v. Air Canada, 2024 BCCRT 149 (reasons for decision, Feb. 14, 2024) (Civil Resolution Tribunal of British Columbia), accessed 2026-10-12. Liability for chatbot statements, chatbot treated as part of the website, conflicting policy page, amount orderedhttps://decisions.civilresolutionbc.ca/crt/crtd/en/525448/1/document.do
- FTC announces crackdown on deceptive AI claims and schemes (press release, Sept. 25, 2024) (U.S. Federal Trade Commission), accessed 2026-10-12. No AI exemption from existing law; allegation that chatbot output was never tested against a human lawyer; proposed settlement amounthttps://www.ftc.gov/news-events/news/press-releases/2024/09/ftc-announces-crackdown-deceptive-ai-claims-schemes
- Fin resolutions (help center) (Intercom), accessed 2026-10-12. How one platform defines confirmed and assumed resolutions, reversals, exclusions and frustration escalationhttps://www.intercom.com/help/en/articles/8205718-fin-resolutions
- About the automated resolutions platform (prior to resolution tiers) (Zendesk), accessed 2026-10-12. How one platform defines and times an automated resolution by channelhttps://support.zendesk.com/hc/en-us/articles/5352026794010
- Analyze your Customer Agent's performance (knowledge base) (HubSpot), accessed 2026-10-12. Definitions of resolved, handed off, deflection and resolution ratehttps://knowledge.hubspot.com/customer-agent/analyze-your-customer-agents-performance
- Freddy AI Agent overview report (support article) (Freshworks (Freshservice)), accessed 2026-10-12. Definitions of resolved conversations and ticket deflection rate for an employee-facing AI agenthttps://support.freshservice.com/support/solutions/articles/50000010557-freddy-ai-agent-overview-report
- Metrics and attributes for Zendesk messaging (Zendesk), accessed 2026-10-12. Definitions of first reply time and percentage satisfaction score for messaginghttps://support.zendesk.com/hc/en-us/articles/4724624097818-Metrics-and-attributes-for-Zendesk-messaging
- Understanding ticket reply time (Zendesk), accessed 2026-10-12. How first reply time is measured and which bot actions are excludedhttps://support.zendesk.com/hc/en-us/articles/4408821871642-Understanding-ticket-reply-time
- Reliability for the Agentic Enterprise (well-architected guide) (Salesforce Architects), accessed 2026-10-12. Counting graceful handoffs as success and unintended escalations against the metrichttps://architect.salesforce.com/docs/architect/well-architected/guide/agentic-enterprise-reliability.html
- Workflow recipe: Using an AI agent to collect customer info and immediately escalate to a human agent (Zendesk), accessed 2026-10-12. Collecting information before handoff and after-hours handling as an email tickethttps://support.zendesk.com/hc/en-us/articles/10412442245914-Workflow-recipe-Using-an-AI-agent-to-collect-customer-info-and-immediately-escalate-to-a-human-agent
- Generative AI at Work (NBER Working Paper 31161) (National Bureau of Economic Research (Brynjolfsson, Li, Raymond)), accessed 2026-10-12. Productivity effect of an AI assistant for support agents, overall and for noviceshttps://www.nber.org/papers/w31161
- AI-Powered Customer Service Fails at Four Times the Rate of Other Tasks (press release, Oct. 7, 2025) (Qualtrics XM Institute), accessed 2026-10-12. Consumer survey on benefit from AI customer service and worry about reaching a personhttps://www.qualtrics.com/news/ai-powered-customer-service-fails-at-four-times-the-rate-of-other-tasks/
- Guidance on Web Accessibility and the ADA (Mar. 18, 2022) (U.S. Department of Justice, Civil Rights Division), accessed 2026-10-12. ADA applies to web services of businesses open to the public; no detailed regulation for private businesses; WCAG as guidancehttps://www.ada.gov/resources/web-guidance/
- Understanding Success Criterion 3.2.6: Consistent Help (WCAG 2.2) (W3C Web Accessibility Initiative), accessed 2026-10-12. Consistent placement of human contact, chat and chatbot help mechanismshttps://www.w3.org/WAI/WCAG22/Understanding/consistent-help.html
- Understanding Success Criterion 4.1.3: Status Messages (WCAG 2.2) (W3C Web Accessibility Initiative), accessed 2026-10-12. Status messages exposed through role or properties without moving focushttps://www.w3.org/WAI/WCAG22/Understanding/status-messages.html
- Web Content Accessibility Guidelines (WCAG) 2.2 (W3C), accessed 2026-10-12. Keyboard operation, adjustable time limits and minimum contrast criteriahttps://www.w3.org/TR/WCAG22/
- 815 ILCS 505/2 (Consumer Fraud and Deceptive Business Practices Act, section 2) (Illinois General Assembly), accessed 2026-10-12. Unfair or deceptive acts and misrepresentation are unlawful whether or not anyone was misled; FTC interpretations consideredhttps://ilga.gov/Documents/legislation/ilcs/documents/081505050K2.htm
- 815 ILCS 530/45 (Personal Information Protection Act, data security) (Illinois General Assembly), accessed 2026-10-12. Reasonable security measures and contract duty for data collectorshttps://www.ilga.gov/documents/legislation/ilcs/documents/081505300K45.htm
- 815 ILCS 530/5 (Personal Information Protection Act, definitions) (Illinois General Assembly), accessed 2026-10-12. What counts as personal information and who is a data collectorhttps://www.ilga.gov/documents/legislation/ilcs/documents/081505300K5.htm
- Bill status for SB 317, 104th General Assembly (official XML status record) (Illinois General Assembly), accessed 2026-10-12. Proposed Consumer Artificial Intelligence Notice Act: content, Senate vote and last actionhttps://lrb.ilga.gov/ftp/legislation/104/BillStatus/XML/10400SB0317.xml
- Bill status for HB 3021, 104th General Assembly (official XML status record) (Illinois General Assembly), accessed 2026-10-12. Proposed AI deception rule under the Consumer Fraud Act and its last actionhttps://lrb.ilga.gov/ftp/legislation/104/BillStatus/XML/10400HB3021.xml
- 740 ILCS 14/15 (Biometric Information Privacy Act, section 15) (Illinois General Assembly), accessed 2026-10-12. Written notice, purpose, retention and written release before collecting biometric identifiershttps://ilga.gov/documents/legislation/ilcs/documents/074000140K15.htm
- 740 ILCS 14/10 (Biometric Information Privacy Act, definitions) (Illinois General Assembly), accessed 2026-10-12. A voiceprint is a biometric identifierhttps://ilga.gov/documents/legislation/ilcs/documents/074000140K10.htm
- Duty to Report to CPSC: Rights and Responsibilities of Businesses (U.S. Consumer Product Safety Commission), accessed 2026-10-12. Who must report product hazards, within what time, and that injury need not have occurredhttps://www.cpsc.gov/Business--Manufacturing/Recall-Guidance/Duty-to-Report-to-CPSC-Rights-and-Responsibilities-of-Businesses
- Artificial Intelligence Risk Management Framework: Generative AI Profile (NIST AI 600-1) (National Institute of Standards and Technology), accessed 2026-10-12. Definition of confabulation and its cause; suggestion to consider disclosing generative AI use to end usershttps://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
- Reduce hallucinations (Claude developer documentation) (Anthropic), accessed 2026-10-12. Allow the model to say it does not know, ground in quotes, verify with citations; techniques reduce but do not eliminate hallucinationshttps://platform.claude.com/docs/en/test-and-evaluate/strengthen-guardrails/reduce-hallucinations
How we checked this page
What we checked
- Read the full CFPB issue spotlight on chatbots, the full reasons for decision in Moffatt v. Air Canada, and the FTC press release of Sept. 25, 2024.
- Read vendor documentation from Intercom, Zendesk, HubSpot, Freshworks and Salesforce for how resolution, deflection, first reply time, CSAT and handoff are defined, and compared the definitions side by side.
- Read the NBER paper abstract, the Qualtrics XM Institute press release, the Justice Department ADA web guidance and the W3C WCAG 2.2 pages on keyboard, timing, contrast, consistent help and status messages.
- Read the Illinois statutes on ilga.gov (Consumer Fraud Act section 2, Personal Information Protection Act sections 5 and 45, Biometric Information Privacy Act sections 10 and 15) and the official status records for SB 317 and HB 3021.
- Read the CPSC duty-to-report page, NIST AI 600-1 and Anthropic's guidance on reducing hallucinations.
What we could not verify
- Whether SB 317 or any similar Illinois bill advanced after the status record was last updated on June 16, 2026, including any fall veto session action. Check ilga.gov.
- A 2024 Gartner customer survey on AI in customer service and several FTC business-blog posts on chatbots: the pages could not be opened, so none of their figures or wording is used.
- Chatbot disclosure laws in other states, such as California and Utah, because their official pages did not open for us. Check them if you serve customers there.
- Rules for support by text message (consent, carrier registration), call recording consent and sector rules such as health or finance beyond the CFPB document. Those need review with your attorney.
- Whether U.S. courts follow the Air Canada reasoning. It is a Canadian small-claims decision and is cited here only as a real example.
- Any average deflection, resolution or cost-saving rate for small U.S. businesses. We found no neutral benchmark, so none is quoted, and vendor figures are not used as averages.
- Vendor definitions can change. They were read on the access date shown for each source.
Tell us which inbox is overloaded
Describe the channel, the ticket types that cost the most time and the ones that worry you. We will reply with a short call and a written scope. Call 773-800-1520, Mon-Fri 9-5 or Sat 10-2 Central, in English or Polish.
