To build empathy and understand the target market I ran a User Persona Workshop with the CEO of AWS-Asia. Together we developed the target demographic.

AWS Chatbot - Target Persona

In my research, I found that most Chatbots are designed to be assistants (e.g., Siri) or made for simple tasks, such as ordering pizza. Very few could accommodate back end calculations needed for the survey scoring and multi-pathway nodes that allowed multiple responses in the same question.
‍
Ultimately we chose to use Tars Chatbot. The system provided integration HTML code, chat exports, analysis of chat data and an internal scoring system that could be used for the survey. This covered most of the list of requirements generated from the client brief.

AWS Chatbot - List of Requirements

AWS provided a brief containing 13 questions with scoring metrics to develop a Minimal Viable Product (MVP). Users could qualify for platinum, gold or silver labels if they reached a certain score. The goal of the initial build was to construct the dialogue flow for the 13 questions and test the scoring system worked.

AWS Charbot - Initial Build

The MVP had a major issue with the scoring system as multiple choice answers were not registering. To resolve this I worked directly with the Tars development team to generate custom code that compiled the scores.

The full build expanded to 21 questions and included conditional responses to receive a “net-zero” rating. I redesigned the chatbot flow to encompass these new questions and built a conversational experience that was more personal and friendly.

AWS Chatbot - Full Build

To test the build we ran user tests with a small team at AWS. Using the feedback we developed a few variations of Ally to test different features.

These tests resulted in the following features:

• To increase the goal conversion rate we introduced a skip question option to reduce users abandoning the survey.
• For questions with multiple answers, we discovered users would make mistakes and wouldn't be able to correct them. I developed a repeat function to allow users to redo the question if necessary.
• The scoring system was too hard with most participants not reaching the Silver level. I adjusted the system to balance the scoring.
• I adjusted the timing of conversational nodes to enhance the user experience.

AWS Chatbot - User Testing

A soft release was conducted with 100 users. Where possible users were interviewed for feedback.

The data resulted in the following changes:

• Some questions were repetitive and annoyed users. Where possible these questions were consolidated to shorten the length of the survey.
• The scoring system had become too easy from the last adjustment. A large number of users were scoring platinum. We adjusted the scores again to push users towards the silver range.

Once we completed these changes we proceeded to a full release.

The full release occurred on the 22nd March (World Water Day). To preview the campaign click here

AWS Chatbot - Full Release

Ally currently sits at a conversion rating of 80% and has been accessed all accross Australia. The AWS is now pitching the system to water authority bodies for a wider release.

Having very limited experience with chatbots I wasn’t sure what to expect or if the end goal was even possible. Thankfully all the functionality came together and the AWS team was very pleased with the result. 
‍
If I could do anything differently I would explore surveying techniques to optimise the interaction and length of the chat flow.