The agent works. It qualifies leads, it generates them and it converts, but part of the client considers it a failure. Both are true at the same time, and that is why this case is worth more than any results deck.
I am telling it as self-criticism, because half of what went wrong is our responsibility.
The client
A financial services firm of about 120 people. The technology team bought an AI agent to qualify and generate leads. It works, and it works well.
The trouble started when the point of contact changed
The problem was not technical. It started when the point of contact changed.
The new people wanted the agent to also pitch products and to answer the way they answer. Fine, we said, how do you answer? Everything stopped right there, because nobody knew. They had nothing written down about how one of their salespeople replies, or why they reply that way and not another. They were asking us to make the agent talk like the team, and the team had never written down how it talks.
That is the baseline problem exactly, wearing a different costume. You cannot ask a machine to reproduce judgement that nobody ever wrote down.
What we fixed and what stayed open
It got resolved. We mapped the answers with them, tuned the agent, and today it generates leads and converts. But the new team is still unhappy, because on top of that they want the agent to interpret images with no context, adding fresh expectations of their own. That was never discussed, never scoped and never part of the engagement. It was assumed.
Who failed?
Both, and I will say our half first. We did not scope tightly enough and we let new stakeholders in without restarting the conversation from the beginning. The sales and marketing people who joined the team never went through the demo and never talked to us about the scope. They arrived with assumptions nobody had checked against anything, and we allowed that to happen.
The other half belongs to the client. We asked them to bring the use cases down to earth and explain what the agent had to answer and why. They did not know.
“We will have to do something”
And then came the question that stayed with me. We asked what they were going to do with all the leads the agent was going to generate.
The answer was: “we will have to do something”.
That is when I understood the real problem. It was not the agent. It was that they expected artificial intelligence to do all the work by magic, including the part only they can do.
What we would do differently
None of these four is a textbook lesson. They are the four we were missing on this project.
Freeze the scope in writing and have the decision-maker sign it. Not a feature list, but the use cases enumerated, with what is in and what is out spelled out. If “interpret images” is not on that list, it is not in the project, and you have to be able to show that on one page.
When the point of contact changes, start over. Demo again, scope again, expectations again. It looks like bureaucracy and it is the opposite: it is the only way the assumptions new people bring surface before they turn into a complaint.
Do not accept “make it answer the way we do” when no document exists describing how they answer. If it does not exist, writing it is part of the project, it takes time and it gets quoted. It is not a configuration detail.
Ask, before signing, what they will do with whatever the agent produces. If the answer is vague, the project has a problem no technology fixes, and it is better to know on day one than in month six.
Why I call it a success
Because the agent does what was asked and brings in business. That is measurable and it has been measured. The project that was contracted works.
The failure sits somewhere else: in the gap between what was agreed and what the people who arrived later imagined they had bought. No platform closes that gap. You manage it before signing, with uncomfortable questions, or you do not manage it.
The underlying conversation about why this happens, with the data on what companies expect from AI and what can actually be measured, is in Is your company ready for AI?.