How to Track Human Feedback
Keeping track of user feedback is a critical aspect of improving the quality of any AI system. With AI Assistants, there are two ways you can track feedback:
- In the Simulator: track your own feedback while developing your Assistants.
- Via the API: track feedback from your users once the Assistant is integrated into your application.
Info
The Feedback API currently supports tracking and reading feedback only. We recommend starting to track feedback now so it can be retroactively analyzed with upcoming analytical features.
As you are developing with AI Assistants, you can use the Simulator to test your Assistant and understand what's happening behind the scenes.
When you encounter particularly good or bad responses, you can provide feedback about the response using the "thumbs up" and "thumbs down" controls below the message.
When your Assistant is integrated into your own application, your customers are the best judges of the quality of its responses.
You can store user feedback collected from your application using the Feedback API. Feedback is represented in a score from 0 (worst possible message) to 1 (best possible message). You can use this as a binary, such as thumbs down = 0 and thumbs up = 1, or introduce your own scale.
Additionally, you can include an optional text property with any notes attached to the feedback, such as whether an issue was resolved.
| Property | Type | Example |
|---|---|---|
session_id | ID of the Assistant session where the user provided feedback. | "demo-session" |
score | The score of the feedback score from 0 (worst) to 1 (best). | 1 |
text | Notes to attach to the Feedback record, such as whether an issue was resolved | "problem resolved" |
1ASSISTANT_SID=<your-assistant-sid>2curl -X POST -H 'Content-Type:application/json' \3https://assistants.twilio.com/v1/Assistants/$ASSISTANT_SID/Feedbacks \4-d '{"session_id":"demo", "score": 1, "text": "problem resolved" }' \5-u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN
Output
1{2"assistant_id": "aia_asst_11111111-aaaa-bbbb-cccc-222222222222",3"created_at": "",4"id": null,5"message_id": "",6"score": 1,7"session_id": "example",8"text": "problem resolved",9"updated_at": ""10}
The following API endpoint displays all feedback received for a specific Assistant via the Simulator or the API. Feedback is not currently viewable in the Twilio Console.
1ASSISTANT_SID=<your-assistant-sid>2curl -X GET -H 'Content-Type:application/json' \3https://assistants.twilio.com/v1/Assistants/$ASSISTANT_SID/Feedbacks \4-u $TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN
Output
1[2{3"assistant_id": "aia_asst_11111111-aaaa-bbbb-cccc-222222222222",4"message_id": "7ea35776-9b68-4b39-a2f6-d60e3d25580f",5"score": 0,6"session_id": "example"7},8{9"assistant_id": "aia_asst_11111111-aaaa-bbbb-cccc-222222222222",10"message_id": "",11"score": 1,12"session_id": "example",13"text": "problem resolved"14}15]