How Pockit Scaled to 70% Resolution with AI

Live in weeks, with benefits for the long term

Pockit approached the implementation of Gradient Labs’ AI agent with a clear understanding that long-term ROI would beat out short-term costs. This holistic view acknowledged that an AI agent would not immediately be more cost-efficient than their legacy chatbot, but the investment would pay off over time from improved resolution quality, increased customer loyalty, and the ability to maintain a high-performing, lean operation as the company scaled. 

The setup process took about three weeks to go live, which included integration work, knowledge ingestion, intent generation, and testing. Notably, the customer operations team had limited engineering resources, but it was still possible to launch quickly because of close collaboration with Gradient Labs’ team and case prioritisation. 

Gradient Labs’ built-in guardrails and extensive familiarity with financial services regulations meant that early tests quickly proved to be safe, reliable, and effective, which made it possible to launch with real customers in a timely manner.

Every query type, not just the common ones

With AI agents, cost savings are maximised with higher automation rates. Pockit’s goal from the start was to achieve a 70% resolution rate as quickly as possible. 

While some companies achieve this benchmark purely by having the AI agent handle the most common customer queries, maximising efficiency at high-volume points, this approach wouldn’t work for Pockit. 

The team analysed inbound requests and found there were no dominant query types significantly contributing to inbound volume. Rather, most of Pockit’s query types made up only 4% or less of total possible topics. 

So to achieve a high rate of automation, Pockit knew it would need to automate the full breadth of its customer support queries as quickly as possible, which meant everything from pending transfer questions, requests for account review timelines, reporting unauthorised payments, refunds for pending pre-authorisation holds, physical cards missing, and much more. 

To meet this challenge, the Pockit team started by getting organised internally. They introduced the AI agent to as many teams as possible, creating buy-in at all levels. They also began sharing specific use cases and results, as well as progress, on a weekly basis. This internal culture of knowledge sharing fostered investment in the AI initiative and helped the team get priority when it mattered, boosting the chances of automating more quickly. 

While team buy in was a critical component to automate for breadth in a short time period, Pockit also needed the right AI agent to see meaningful results. 

In early testing, they found that Gradient Labs is extremely adept at handling a large amount of internal knowledge and customer context, balancing that with empathy, speed, and safety in highly-regulated conversations. 

Real resolution yields better CSAT

Before deploying live, the Pockit team tested the Gradient Labs AI agent in a large swath of use cases, a process that took only a few weeks. They shared their extensive knowledge base and company context with the agent, and after ingestion saw strong results from this alone. The extra weeks were to close any knowledge gaps, supplement information based on specific scenarios, and to guide the AI agent’s tone of voice. 

During this testing process, the Pockit team also put long-term processes in place to ensure the knowledge base would remain current and updated, and that regular quality checks would be performed. 

Gradient Labs’ AI agent demonstrated consistent behaviour across chat and email and was unfailing in its regulatory compliance, something possible because of the finance-specific guardrails that came pre-baked into the agent. 

Once live, Pockit prioritised rolling out new intents and use case types as quickly as possible. In just a few months, the AI agent was able to handle nearly all inbound queries on two channels, chat and email. 

Most of these queries are consistently resolved end-to-end with no human interaction. More importantly, Pockit saw CSAT ratings rise to 80% for these cases.

This indicates that unlike the legacy chatbot, the Gradient Labs’ AI agent was able to make customers feel that their problems were truly addressed, across a wide range of topics, while staying compliant in a highly-regulated space.

Human in the loop

For some queries, resolution requires human involvement. This is often due to regulatory requirements or fringe cases. These include scenarios such as requesting a different outcome to a support decision, or compliance-sensitive cases like requesting an account review or questions related to restricted accounts. 

With a large portion of inbound queries successfully automating without a human, the Pockit team turned its next focus to these types of cases. 

The team aimed to achieve partial automation with Gradient Labs’ Ask a Human feature and to smooth out the operational flow by reducing the number of human touchpoints needed, keeping only the essential ones.

With Ask a Human, the AI agent handles everything it can, including gathering context, asking clarifying questions, pulling the customer’s details, and identifying action items that need to happen. Then it surfaces a clean handoff to a human for any necessary decisions, maintaining conversation with the customer in the meantime. Once it has the answer it needs, the AI agent can close the conversation. 

For example, before introducing this feature, Pockit’s flow for handling any query that involved the FinCrime team was: 

With Ask a Human, the new flow reduces the number of human touchpoints to one, speeding up the process. The new flow keeps the human in the loop and has the AI agent do most of the heavy lifting. It looks like this: