Project

/

kaichatbot

Kai Chatbot

A chatbot framework aimed at helping teams with internal products get answers to their customers faster without taking precious time away from the developers

UI design

UX research

Service design

Internal Tooling

About

Problem

Internal teams were experiencing repetitive support questions that were funneling through multiple channels and consuming individual team bandwidth. Teams were also independently creating their own Chatbot solutions resulting in repeated efforts.

There was an opportunity to centralize these processes to eliminate creation complexity and build infrastructure for these bots and improve the developer experience.

My Role

I was a scrum master, UX designer, creative lead
I worked with 6 person development team, tech lead, and product owner over a period of 8 months at Capital One Bank


Research


Competitive Analysis

Here we performed an audit on industry chat bots and what current chat bots were existing within our Capital One tools. We uncovered and documented process flows, gathered visuals, and gained an understanding of the complexities that chat bots entailed. We found a few internal chatbot teams (such as Emmi) in flight and continued to partner with them and learn from their algorithms.

Empathy Interviews

After understanding the Chatbot landscape, we conducted empathy interviews with 20 users from a variety of support teams to learn more about our customers’ pain points, hopes/wishes, desires.

Findings

We discovered that our internal tech teams were spending on average 10 hours a day covering support channels and answering hundreds of slack requests per day

Our Empathy Research Findings

1. Users needed a simpler onboarding experience (less manual labor)
2. Users needed an easy to digest platform, where they can edit, add and update their Q&A
3. Users needed a way to see how their bot was performing so they could take action if they needed to update any bad data
4. Users needed the bot to perform better and be able to answer complex questions


Understanding Customer and Journey

We began to plot the journey map of our customers down and summarize the demographics/persona’s of what our customer teams are needing

Our Empathy Research Findings

1. Users needed a simpler onboarding experience (less manual labor)
2. Users needed an easy to digest platform, where they can edit, add and update their Q&A
3. Users needed a way to see how their bot was performing so they could take action if they needed to update any bad data
4. Users needed the bot to perform better and be able to answer complex questions

Understanding Customer and Journey

We began to plot the journey map of our customers down and summarize the demographics/persona’s of what our customer teams are needing

Process Flow

After consuming and understanding our information architecture, I created a process flow chart to understand the decisions our customers make in order to onboard and use Kai and what pages they need to land on.

Process Flow

After consuming and understanding our information architecture, I created a process flow chart to understand the decisions our customers make in order to onboard and use Kai and what pages they need to land on.

Ideation/Stakeholder Workshop


We hosted multiple hot houses to understand where Kai could exist in different communication channels to expand beyond slack messaging. We generated ideas on how to host a centralized place for all Kai’s questions to live and what they would entail. Here we thought of a spectrum of ideas: maybe we should add a widget? Or email notifications? How about SMS alerts? Or just stick to slack?

After a thoughtful group critique, we didn’t want to further complicate the tool so we re-energized our focus on improving the slackbot dashboard experience and generating a widget tool.

Wireframing


After having a clearer direction of the experiences we needed to focus and tackle on first we began wire framing and prototyping a new onboarding page and main dashboard. It helped us understand what metrics were able to be pulled for users and guide us into configuring our slack bot output returns.

High-Fidelity Designs


After wireframing, I completed these designs on my own, and was in charge of branding so I mocked up widget designs, logos and other iconography.'

Q&A Dashboard

Kai Chatbot training portal allows users to edit, add and delete questions according to their datasets. It contains a suggestioned section where users can add commonly asked questions not in their data sets that Kai has combed.

Metrics Page

I conducted separate empathy interviews to understand the different use cases for metrics displaying how our Chatbot was performing. Based on the most requested and plausible metrics that the team could implement, I designed our new metrics feature using Chart.Js.

Onboarding

Before our redesign we were receiving the negative feedback about the complications of onboarding. I worked with my product team to streamline this process and decrease time spent back and forth between customers and our support team.

Slackbot & Widget


Part of my role in Kai was to design the widget feature that would live on customer’s portal – this was extremely beneficial to customers visiting our support teams sites as they did not need to leave the application to get help. Here, I headed the creative direction by designing the logo, branding iconography and style guides.

The slackbot was also carefully designed, as often it needed to receive customer responses in order to improve it’s learning algorithm as well as create an easy experience for customer to select the option towards their solution. We considered different color patterns and interactions here.

Validation


Usability Testing

After creating our designs, we conducted a task based usability test with 25 of our customers, where we watched them complete a set of prompts based around their jobs to be done and synthesized the following conclusions:

• 70% of our users struggled with the verbiage of our buttons/actions
• 40% of them were confused on how to use the “Suggested Question” section
• 50% of users thought the “Edit a question flow” was too time consuming (too many clicks)
• Nearly 100% of users requested a metrics page to understand how the bot performance

I was able to redesign a more intuitive “Suggested Questions” section, clean up the verbiage, scope out and design the metrics feature, and reduce time spent editing and saving questions.


Guerrilla Testing at Road Shows

Our team presented Kai at SECON, an internal engineering trade show/fair, where we demoed our new product to hundreds of engineers. We gave out surveys and did some guerrilla interviewing. Took the new feedback collected to continue iterating on our current onboarding experience which we got the most concern with.

Learnings

During a MVP validation session of 20 users where Kai was first tested in front of users (during production), it was apparent that there was no design process taken when designing the product previously by the team. Customers complained about a visually confusing product that wasn’t intuitive and didn’t have the features they even wanted.

Having to retrace our steps to completely, I reworked a lot of product features to make sure we were building a customer-centric bot. I learned a lot about the detriments of implementing bad design.

Results

Learnings

During a MVP validation session of 20 users where Kai was first tested in front of users (during production), it was apparent that there was no design process taken when designing the product previously by the team. Customers complained about a visually confusing product that wasn’t intuitive and didn’t have the features they even wanted.

Having to retrace our steps to completely, I reworked a lot of product features to make sure we were building a customer-centric bot. I learned a lot about the detriments of implementing bad design.

Learnings

During a MVP validation session of 20 users where Kai was first tested in front of users (during production), it was apparent that there was no design process taken when designing the product previously by the team. Customers complained about a visually confusing product that wasn’t intuitive and didn’t have the features they even wanted.

Having to retrace our steps to completely, I reworked a lot of product features to make sure we were building a customer-centric bot. I learned a lot about the detriments of implementing bad design.