Introduction

EVO Bank’s virtual assistant had grown unpredictable. With 535 intents and close to 5,000 training phrases feeding it, nobody could reliably tell where a conversation would end up. When it failed, fallback was effectively random: most of the time it didn’t trigger at all, leaving the conversation without an answer.

For the person writing to their bank at eleven at night, that meant one of two endings. They were routed to a human agent who handled the request the assistant was built to handle, or they gave up.

The audit changed how we understood the problem. It wasn’t comprehension. The assistant understood plenty. It was overexposure: an unstructured mass of training data had left too many intents competing for the same utterance, so the model kept finding plausible routes that led nowhere useful.

I owned conversational design for the rebuild, from intent architecture through the assistant’s tone, leading a team of Spanish-language linguists and working with EVO’s marketing team on a three-month deadline.

Framing

The core decision was to rebuild from zero rather than patch the existing model intent by intent. We started from the simplest compositions and worked up, confirming at every step that a change didn’t break a flow or leave the user at a dead end, with each intent’s purpose and coverage explicit before adding the next layer on top of it.

The alternative we discarded was incremental cleanup of the existing 535-intent model, the same direction that had produced an assistant nobody could predict in the first place.

The hardest constraint was time. A three-month fixed deadline left almost no room to test against real customers before launch, so verification had to be built into the rebuild itself rather than scheduled after it. That constraint shaped the method more than any design decision did.

Crafting

A compositional intent structure

The model was rebuilt from the most basic and robotic formulations upward, towards composed phrases layered on top of them. Structuring it that way meant each intent had a stated purpose and a bounded coverage before the next one was added, and it made the growth of the model legible instead of accumulative. The structure became a blueprint we reused to approach other assistants.

A regression battery against derivation

With no window for customer testing, we built a battery of tests and ran it as new intents went in, checking that each addition didn’t pull utterances away from an existing intent. It caught derivation while the model was still small enough to fix, and it replaced the per-change judgment call that the previous model had relied on.

Fallback as a guaranteed exit

Flows were rebuilt so that a failed match always resolved into a clear next step. Fallback stopped being the state where the conversation ended and became the state where the assistant said what it could not do and where to go instead.

Tone and personality

Personality was built to match EVO’s own web copy rather than a generic assistant voice, and validated directly with the marketing team so the tone stayed consistent with what customers already read on the site. The assistant launched in Spanish on EVO’s web channel, the surface where the rebuilt model and the aligned personality could be observed against real usage before expanding further.

Outcome

Training phrases dropped from roughly 5,000 to 586, and intents from 535 to 491. The reduction was almost entirely in training data, not in what the assistant could do. On the initial sample, intent recognition accuracy went from 67% to 98%.

What started as a three-month, fixed-deadline engagement became a long-term service contract with a dedicated team, and EVO expanded the assistant beyond its original web channel.

The lasting output was the method. The audit and rebuild produced a repeatable set of guidelines and a client workshop for assessing an assistant’s state and defining lines of action before committing to a rebuild, which made the analysis work available to the rest of the team instead of depending on one person to run it.

What the deadline cost was external validation. The regression battery proved the model behaved as designed, but it could not tell us what customers would actually ask. Those two things are not the same, and the gap only closed once the assistant had been live long enough to produce real conversations.