QA Calibrations are organized meetings that allow reviews to sync and align on scoring, making sure all reviewers are thinking and reviewing around the same level, allowing consistency and clarity across all QA reviews.

To start a QA calibration, click on QA in the navigation bar and select Calibrations. Next, click the Create New Calibration button.
​

Starting a new calibration will require you to go through a three-step process.

First, find the interaction you would like to calibrate on. You will be required to indicate the interaction ID, team member, and scorecard. You can also optionally include an interaction link. Required fields are indicated on the screen.
​

If you are in an integrated state, Stella Connect will automatically pull in either the transcript or audio file associated with the interaction ID. Otherwise, you can manually insert a transcript or upload a file.

Note: only the calibration owner will have access to view the associated team member. The team member name will be hidden from all calibration participants to eliminate any bias within the calibration process.

If you haven't already selected question in the Calibrations Overview report, next identify the questions you would like to calibrate on. It may not be necessary to consistently calibrate on questions from your scorecard that your team has consensus on. We recommend you focus on a smaller subset of questions to make sure your calibration meetings are fruitful for the entire team while still building confidence that graders are aligned in how they grade.

If you need to calibrate on all questions, you can still select all questions.
​
​

Finally, invite teammates to participate in the calibration. You can also optionally include a meeting date for informational purposes.
​
​

Once you start your calibration, all added participants will be notified by email that they have a calibration to complete.

Any interaction for you to review as part of a calibration will be listed in your Calibration Sessions section under Calibrations. While a calibration is in progress, you will not have access to view how any other participant scored the same interaction.

To review an interaction select Review interaction. You will be able to grade the interaction and add any relevant comments. We recommend you provide comments while scoring an interaction for purposes of a calibration to help facilitate a better calibration meeting.

Once you have completed your review of an in-progress calibration, the interaction will appear in the Completed / In progress list, and you will have access to view the responses of any other participant.

To calibrate across reviewers, select View results from the Completed / In progress list. From there you can view the responses of each calibration participant, and select the final consensus answer. You can also add meeting notes to document decisions or outcomes from the calibration.
​
​

Once all consensus answers have been selected, complete the calibration from the results page. The outcomes saved in your meetings notes will be available for future reference from your "Completed / In Progress" calibration list.

Delete a calibration by selecting the delete action from a calibration assignment. Deleting a calibration will remove it from the calibration list, and any participant’s to-do queue. The delete action cannot be undone.
​
To delete a calibration that is already "In-Progress", the user that created it will have to complete the calibration. Once the calibration has been completed by that user they will be able to delete the calibration.

Anybody with access to review an interaction can initiate or participate in a calibration. At this time, team members do not have access to participate in calibrations.

All users with permission to calibrate will have access to view all completed or in-progress calibrations. Calibration participants will not be able to view calibration results until they have already submitted their scores for the calibration.

Any user with permission to calibrate can select the consensus responses, or delete a calibration.