How one service agent served more than a million customers
Advance
Advancing connectivity to accelerate the digital future
Advance means that Telenor is at the forefront of technological development, constantly innovating and exploring for the benefit of customers, businesses and society.
Meet the team of young AI experts training the next generation of digital customer service.
“Humans are very unpredictable. To respond adequately to the challenges people throw at you, you need to possess a whole array of instincts and memories. If not, you are doomed to fail,” says Morten Lossius, as he parries yet another lunge from his fencing partner.
We’ve caught up with Lossius, Telenor Norway’s senior content developer, at his weekly fencing training. This is a sport where accuracy isn’t just optimal, it’s essential.
“Actually, the challenges a chatbot faces are quite similar to the ones that confront a fencer,” he continues.
“Both need to gain a certain skill set to master unpredictable scenarios to accomplish their tasks successfully. Each person who contacts a chatbot wants the same thing. Not to land a hit, of course, but to get accurate information to solve an issue. And since there are countless ways people can present an issue, we have to train our chatbot, Telmi, how to respond to them. We have to train it a lot…” says Lossius.
‘Telmi’ what I can do for you
Telmi, Telenor Norway’s very own chatbot, was ‘born’ on 29 January 2019. From its first day of operations to the time of writing, the AI robot has handled more than one million customer inquiries.
“It’s been a fascinating journey so far, and the number of Telmi users has steadily increased since we launched it,” says Lossius, adding that the chatbot processed more than 630,000 customer inquiries in 2020.
“According to our estimates, this will increase by 10 percent in 2021, as we continue implementing Telmi in other Telenor Norway services. For 2023, our goal is for Telmi to process more than 770,000 customer inquiries.”
The introduction of Telmi into new services requires a lot of calibrating and updating of the chatbot’s algorithm.
“We constantly train Telmi to get better at recognising new words, meanings, questions, and to better interact with both our internal systems and with humans. Because, as I said earlier, humans are unpredictable.”
Robot fine-tuning
That unpredictability comes with layers of nuance, as we humans do not always say or write what we actually mean.
“If a customer states ‘I would like to add more 4G to my phone’, it basically makes no sense. Humans, with the right knowledge, understand that the customer is actually referring to data, but it’s harder to teach a text-based machine to correctly interpret improper use of terms.”
As providers continue to deliver new and ever more complex services to customers, the latter invent and adopt new ways of talking about them, explains Lossius.
“It’s no wonder chatbots often struggle to find the answers customers are looking for. But with help from the research arm of Telenor, we’ve been able to fine-tune Telmi into serving customers more accurate information than ever before.”
More accurate each day
Some of Telmi’s increased accuracy stems from analyses of conversations between Telmi and customers. ‘Intents,’ as the chatbot professionals call it.
“Every customer has an intention for contacting Telmi. Right now, Telmi can handle about 2,000 different predicted customer intents. For Telmi, an intent is, easily explained, a code. What you and I write in the chatbot log translates into codes, which tell the chatbot’s algorithm what sort of information it needs to provide,” explains Eleonora Freddi, Senior Research Scientist at Telenor Research.
During spring 2020, the Telmi team gathered data from 150,000 conversations, which they handed to Freddi and her colleagues in Telenor Research. The result from the enormous material analysis has proved itself hugely valuable.
Increased customer satisfaction
“From our analysis, we found that one percent of the intents accounted for 34 percent of all the conversations. This was a very important finding. It told us that improving only 27 of Telmi’s interpretation could potentially improve a third of all interactions between Telmi and customers,” Freddi tells us.
Based on the results, the AI trainers implemented changes to the chatbot. They have since seen a significant difference in measured customer satisfaction. However, providing more accurate information won’t necessarily boost every customer’s satisfaction.
“A customer’s satisfaction is mostly dependent on whether the issue was solved or not, which is very understandable. A chatbot can’t fix all problems, for instance, network coverage issues. So, even though the correct information is provided, the customer may be frustrated because the issue isn’t solved. Because of this, the customer may rate the chatbot as non-helpful. In general, it’s a challenging exercise to measure customers’ chatbot satisfaction,” says Freddi, adding that she and her colleagues are currently analysing a new set of data from Telmi conversations.
Despite the potential for improvement, today’s chatbots have come a long way compared to where it all started, says one of the legends within the AI research field in Norway.
Interested in Artificial Intelligence? Telenor allows employees to upskill within AI
The ‘friendly computer’
“In the 90s, we envisioned the friendly computer. A machine that spoke to humans in a ‘human’ way. It was merely a fantasy at that time, but today people are getting used to speaking to computers and virtual assistants,” says Telenor Research’s Knut Kvale, who started working with speech-based services in Telenor back in 1994.
The senior research scientist points to the chatbot’s ability to fill a growing customer need as the reason for its increasing success.
“Internal statistics show that about 50 percent of customers have visited our web page before calling our customer service desk. That’s a clear indication that we as customers are trying to find solutions by ourselves. Therefore, it can be beneficial for us to get help along the way from a friendly computer,” he says, stating that we should expect to be acquainted with a rising number of helpful machines in the future.
A chatbot breakthrough on the horizon?
“I believe we are standing on the doorstep of a huge chatbot breakthrough. Because, in addition to chatbots’ ability to serve customers at a fast pace, they provide a form of anonymity. People are rather uncomfortable with discussing certain topics with other humans. Therefore, I expect we’ll see a future increase in the implementation of chatbots in sectors such as health, finance, and local and regional government, to name a few.”
Kvale hopes that the more we interact with chatbots, the better the quality of each conversation will become.
“All communication must be adjusted based on who the recipient is. We usually need to communicate differently with a five-year-old than with an adult. The same applies when we’re communicating with machines. In most cases they only need the key information spelled correctly to serve you in the best possible way. What I’m saying is that we humans need to adapt a little to the chatbots as well to get the best experience possible. Because, as we all know, it takes two to tango.”