How to access the scorecard builder, use a template or start from scratch, and the three phases before launching an evaluation with the scorecard.
This article supports Super User and Supervisor roles only.
In PeopleQX, you will create scorecards to drive the accuracy of AI scoring, or what is known as AutoQA in the system. In brief, AI scoring works rather differently from human-based scoring, hence the reasons to develop AI-optimized scorecards. If you are ready to dive in, let us briefly go through what you can expect in the process of building a scorecard.
Access the builder
The first step is to find the scorecard builder. Anywhere you see the profile icon, click on it and select ‘Scorecards’ from the drop-down menu.
Select a building method
You will be provided with two methods to create a scorecard — from scratch or with a template. Assess your needs and select a method to continue.
Phase 1: Add questions
Questions are to guide both AutoQA (AI scoring) and Manual QA (human evaluators) in the process of assessing a customer-agent interaction. They should be organized into critical or non-critical sections and tagged into a pre-determined category.
Essentially, you’ll need to craft questions that can only be answered with “Yes” or “No”. There’s more about how to add questions, so make sure you’re not skipping any steps.
Phase 2: Test questions
Testing allows you to identify gaps and fine-tune the questions before AutoQA evaluation runs on the conversations. You can also validate AutoQA’s performance against Manual QA by comparing both scores so you’ll know what works and what needs to be improved for future scorecards.
Phase 3: View quality
This phase breaks down the quality of your scorecard post-testing. However, it only runs if you add benchmark scores. The information includes:
- overall AutoQA quality
- the quality of each question
- comparison between AutoQA and Manual QA
Caught a sense of the scorecard-building process? Do remember that the three phases are not linear — they do not depend on one another. Testing and viewing quality are not mandatory but they could provide a more comprehensive view of the scorecard’s performance.
Ready to create some scorecards? Find out how building your own and templates differ at their starting points.