HVAC AI receptionist scorecard: 25 tests before going live

Create a representative call set

Use recordings or scripts based on real demand: no cooling, no heat, maintenance, estimate requests, warranty questions, wrong numbers and callers who cannot explain the equipment. Include accents, background noise, interruptions and incomplete addresses. Remove personal details before sharing examples with a vendor.

Each test needs an expected outcome. The agent should collect the minimum booking record, use approved safety language and either book, route or decline according to policy.

• Five normal booking calls
• Five emergency or safety calls
• Five noisy or confused calls
• Five out-of-area or unsupported jobs
• Five human-handoff requests

Protect safety boundaries

Write approved responses for gas odor, electrical burning, carbon-monoxide alarms, flooding near equipment and vulnerable occupants. The agent should never improvise technical diagnosis or promise that a situation is safe. It should deliver the approved instruction and escalate according to your policy.

Review transcripts and audio, not just the vendor summary. A correct final disposition can still contain risky wording on the way there. Treat one serious safety failure as a launch blocker.

Verify booking and handoff

Check name, callback number, service address, equipment or problem, urgency and appointment window in the destination system. Test duplicates, rescheduling and a caller changing details late in the conversation.

Ask for a person in several different ways. Human transfer must be easy, and failed transfers need a callback task with a named owner. Never trap an upset or confused customer inside the automation.

Score outcomes and monitor launch

Weight booking accuracy, safety compliance and human handoff above voice naturalness. Also score service-area decisions, spam filtering, latency, transcript usefulness and cost per resolved call. Publish the pass threshold internally before testing.

Launch during a controlled window with staff watching every call. Review daily for two weeks, then sample weekly. Keep the existing phone fallback until results are stable.