Why AI Implementations Fail Without Change Management

Why AI Implementations Fail Without Change Management 

Maciej Kłodaś [MK]
Hello everyone, my name is Maciej, I’m the leader of Analytics Experience Competency Group at C&F and this is C&F Talks, a place where professionals discuss challenges, risks and trends from the perspective of an IT partner. Today my guest is Kylene Merritt, Senior Manager at C&F. Tell us, what is your area of expertise, what you do at C&F?

Kylene Merritt [KM]
I help with business development, I also am a client engagement manager, I work with clients on implementation of various different data applications, AI tools, helping with business processes and helping get their implementations and their value realized.

[MK]
Business processes, this is one of the most important things when we’re going to discuss the change management. These are those processes. In the era of AI and the fear, the hype, the pressure around it, the change management is even more important to bring this proper value.

[KM]
It is. Without change management, you can build a tool, but it doesn’t mean you’re going to receive the value and you need people in order to receive that value and to adopt the technology that you’re building. And AI has uncertainty around it. Without all of that, and without using change management techniques throughout, most likely your solution won’t receive the value. It may not have the adoption, it may have minimal adoption. Really looking at change management throughout is super important.

[MK]
I’ve seen a lot of, or maybe a lack of change management everywhere with our clients. I have hundreds of examples. I remember that when we were discussing with one of the clients in France, if I remember correctly, we’ve been discussing this implementation of standardized reporting. And they were imposing this reporting into local markets. And the biggest problem was that nobody was using this. There was shadow reporting. And I was asking, did you discuss this topic with your users and those other companies that you are introducing this tool to them? And they said, no, because this is the kind of a standard that we are having right now globally. And they need to use it. And it was, do they need to use it? Are they used to using something else? And this is exactly the same thing. We can see that all over the place.

[KM]
Exactly. That’s one of the key things is a lot of times people build a tool. They don’t engage the users throughout. And then at the end they say, here’s a new tool, you need to use it. And the users don’t understand why they need to do it, how it’s supposed to benefit them. Some of the other issues that happen is that since you didn’t engage users throughout, then sometimes you build things that make their process longer. It’s much more difficult for the users to use the tools because they weren’t part of the process from the beginning and couldn’t provide their feedback for you to make those adjustments. The users are going to not want to use it, even if you tell them that they need to use it.

[MK]
You can train them even, but then they have their own Excel spreadsheets.

[KM]
Exactly. They have their own Excel spreadsheets. And human nature is always, if you don’t like something, you’re going to try to find the workaround. As you said, with the shadow processes, what they’re going to end up doing on the shadow processes is they will find that for you. That’s one of the other keys, as you’re working through this and with change management, you have to shut off whatever you’re replacing. If you leave it around by human nature, they’re going to go after what they’re used to. People don’t mind change. They don’t like being forced to change. If you don’t work with them throughout, then they don’t understand the why all the time. They don’t understand what is this going to bring for them? How is this going to make it easier for them? And not that everything’s always about that user. But again, somebody has to use the tools. It doesn’t matter how good you think the tool is. If users aren’t willing to use it, it’s going to fail.

[MK]
And now with the pace of implementing AI tools, you have this pressure everywhere. There are budgets that you need to implement AI. I need to find this value. And because there is lack of time, usually nobody is talking to the users. They have their targets and KPIs and companies that they need to implement something with AI. But nobody asks the user if they are going to use it and what value will it bring to them.

[KM]
Exactly. And with AI especially, there’s a whole hype of fear. Around it you have everybody thinking they’re going to completely be replaced by AI. Companies especially have to spend extra time with the change management explaining to the users how this is going to benefit them, how this can make their lives easier. And it’s not always about replacing the people, it’s about giving users time to allow them to do the most valuable piece of work. Using AI to help offload all of those things that we all have to do, but take time. How do you get that more mundane pieces and how do you show that value based on the use case that you’re providing for that user in order to adopt it in the end?

[MK]
We usually fear what we don’t know and understand. This is the time we need to understand why we are using that and how we’ll benefit. Starting from the beginning, what are the principles of proper change management?

[KM]
Proper change management, get users engaged early. Right when you’re starting those use cases, start working on change management. Whether that’s communication, finding your change champions, but coming up with that plan. It’ll help you be successful. Not even with the end users, but engaging those change champions, those people that are willing to accept change, they love change.

[MK]
Early adopters.

[KM]
Yes, your early adopters, those tech focused business users. Who can provide that information for you and use them throughout because their excitement will be contagious. They feel they’re being heard throughout. Also there’s things that they may come up with throughout. For example, we were implementing an AI tool. It was a knowledge management tool. And within that, we identified that all of the articles that had to be ingested into the tool were not written in a way that AI could read them properly. We needed to train all of the knowledge owners to rewrite that ahead of time so that we could also make the tool, when we implemented it, be useful for the end users. The last thing you also want to do is have a tool, you implement it, and it doesn’t work because you typically have one shot with users for them to accept it. I know exactly what you mean. By starting early, you can identify all of those pieces. Whether it’s a process that you need to update, whether it’s information to make the tool better for users in the end. And for you to get the value of the tool. Starting early is one of the biggest key principles. The second one is looking at your stakeholders. Understanding what is your user base? Who, what is their adoption of change level? Who are you rolling this out to? What’s that value? Really putting that into your plan. Understand how is your organization built? What are the user types? How much work do you need to do? And plan for that, just like you would plan for a project, just like you would plan for your requirements. Doing all of those types of things. Making sure that you include process throughout. What are those use cases? Understanding the business side as well. Not just focusing in on the tech team, on what they can build. Make sure that whoever is going to use that, again, back to the second principle. Who are the end users going to be? Because they need to be the ones to be able to use it. Making sure that you have them involved throughout. Plan for training. Make it fun for users. Really look at how you’re going to get in the end. What’s the training schedule going to be like? One of the key principles is don’t overload users with one training in the end. If you have one really long training and then you go, here’s the tool, go use it.

[MK]
It’s going to fail.

[KM]
It’s going to fail because users don’t feel they’re being supported. And especially depending on the level of change. That goes back to that use case. Understanding how much change are you asking these users to do and building the implementation plan and that training plan to make sure that they are successful. The other piece is make it fun for them. Have a reward system. If your company has an internal point system that you can do.

[MK]
Gamification.

[KM]
Make it complete gamification. So that they get something out of it. The biggest thing for users is they want to know that you care, that you’re listening to them. And that you’re there to support them in this change.

[MK]
And they like to compete with each other. They totally like to compete with each other. If you implement gamification and competition. It will work even better.

[KM]
It will. And especially there’s fun things that you can do with an AI tool. We all know that chatbots and agents are a big thing these days. If you plan, you can use some of these agents if you develop them correctly throughout. You can also use them to feed information. You can even use the tool to put in gamification and give them little tidbits to keep them engaged. And then the big thing is build out what your KPIs are going to be on how you’re going to measure that adoption. So that you can continue to see what’s working, what isn’t working. As people are going in for that adoption. You can adjust along the way.

[MK]
Is there a feedback loop with those group of users? A usability study to implement changes to the process or tool?

[KM]
Yes. You should understand those KPIs early. And you should be using that with your change champions that I talked about throughout. Test your training with them. Test how much even your champions are engaged. And then make those adjustments. And then at the end, when you go for full adoption, that should be part of it as well is what is that feedback loop? Because no matter how much you do throughout, nothing’s ever going to be perfect. There’s always going to be something somebody does. Something somebody comes up with an idea. As you continue to build out your tools, have that feedback loop. Make sure that that’s part of the training, that they know how to do that. They know how to get help, but they also know how to provide that information to make the tool better.

[MK]
Tell me how long such process takes.

[KM]
It really depends on the tool. And what you’re implementing. If you have something that’s going out for an entire enterprise, that’s going to take more effort in your change management adoption. If you’re talking, if your company is a huge company that’s Fortune 5, or something like that, you have 300,000 people that you’re going to try to get to use this tool. It’s going to take you a lot more planning, several more tools throughout. You’re going to want to get more change champions. You’re going to have to spend more time having those conversations along to management, leadership, making sure people know about this. Lots of transparency.

[MK]
Identifying those groups of users.

[KM]
Identifying those groups of users.

[MK]
And different needs and expectations.

[KM]
Exactly. Whereas, if you have something where you’re a smaller team and you’re trying to get a smaller group of users to do it, it doesn’t mean that you still don’t follow the same principles. But you’re not going to have to go to as many user groups. If you have only one stakeholder group, for instance, that you’re going to go to, then that’s going to take less time. You don’t have to have as many different types of presentations. You don’t have to create communications that may go on an intranet site. You have possibly your emails. You have your team meeting. But you still would follow the same steps. Who’s going to be using those tools? What is that communication methodology? Who are your change champions? Even if you have a small one, you still need your change champions because they’re the ones who are going to meet with you, spend that time, get that engagement throughout in order to make your company receive the value of the tool that you implemented. The other thing I would also say is that it’s very much of a partnership. Whether you’re doing it on your own or you have a consulting company like C&F working with you, it’s a partnership between the company representatives and whoever you’re partnering with. It’s important that both sides are there in order to receive that value.

[MK]
I’ve seen one of the biggest challenges that you don’t have a buy in from your major stakeholders. You are implementing a tool and they don’t see the value of this change management, proper change management process. They would like to implement it, introduce it and expect that the users will use it. And they don’t measure the adoption rate afterwards. And then after a few months, they come back with a problem that we spent a lot of money and nobody is using that. This change management part for your stakeholders is a totally different thing.

[KM]
It is. That goes into a lot of why adoptions fail exactly as you said, because people forget about change management. They forget that in order for you to receive that value, you need people. People have to use whatever you’re building. Whether it’s a new process that you’re building, whether it’s a tool that you’re building, anything. You can’t do it without the people. In the past, different technologies have required more change management than others. I think with how fast people are moving these days, they often tend to forget how important it is to bring those users along. Especially AI is tweaking how even things within companies are having to be ingested.

[KM]
It’s taking different data. It’s allowing you to have access to that data very quickly. But in a magnitude that you’ve never had before.

[MK]
And you have habits. Different people have different habits. It’s hard to change habits with people with different age, for instance. I tend to use Google nowadays. Always use Google. Where my younger colleagues use, for instance, chat GPT to search for answers.

[KM]
And it’s a different way of learning how to work. It’s a different way to learn how to interact with AI. I was the same as you. I would always use Google. I use chat GPT, Grok, all the different cloud and various other tools. For different exercises. And it takes effort. It’s easy to start thinking when you’re using these tools, they’re going to read my mind. I can be very, it doesn’t matter what language I’m using as I’m trying to interact with these tools. But it really does. You have to be more explicit with what your expectations are in order to get the right answer. The other thing is that using these tools, especially AI related tools. It’s very much that they come back and they make you feel as a human that they’re authoritative. And that they’re telling you the full truth.

[MK]
All depends on the settings. You can change the settings. Then they can be sensitive.

[KM]
Correct. With all of that, it’s a different way of interacting with technology that we’ve never experienced before. And you have different people in your organizations, too, that are used to those types of tools. But then when we’re implementing AI tools within companies, they’re very specific to that company most of the time. You’re only using the data for that company. You’re building different tools. There are different levels of maturity than what people are possibly used to using outside. It takes a lot. As you’re building that out and as you’re using change management so that you can ensure people are going to use it. Because some of these tools, especially if it’s a very narrow scope, may seem more rudimentary to some of those users who are using chat GPT. And they think, well, chat GPT could do all of these things.

[MK]
Exactly. And this is the changing of the mental model. If you’ve been using BI reports, analyzing this data, getting your insights, and then you have been introduced with a convo model, chat. And you chat with the data instead of looking at it. This is the change that you need to have. And this is the change management you need to go through in order to switch from one to another. These are the risks and challenges of the change management. What are the other risks? Why change or not? Maybe not why change management doesn’t work, but why those tools fail or are not being used?

[KM]
More of the tools fail because people underestimate the importance of the human at the end interacting. Another reason why they fail is that the end users don’t understand the why. The people they are asking to use it at the end, they don’t understand how this is going to be beneficial. They’re stuck in their ways at times. They have their habits. They’re used to doing these certain processes in order to do their job every single day.

[MK]
And it worked.

[KM]
And it worked. Why do I need to do something different? The third one is that sometimes if you’re not using that change management throughout, you make things harder. There was an example where there was different stakeholders that were supposed to use a financial budgeting tool for the next year. A company was hired. They came in. They were implementing various tools. We were called in later. But they made the process so much harder for one group of stakeholders because they only focused in on the finance stakeholders. They did not focus in on all of the users that actually had to provide their budgets. We came in afterwards and helped fix the tree along the way.

[MK]
Any other challenges?

[KM]
Underestimating the training, making users not feel heard.

[MK]
Okay.

[KM]
If they don’t feel they’re supported, then they’re not going to adopt the tool.

[MK]
I remember one example with reps, essentially. There were different groups of reps, younger reps and there were the champions. And there were older reps, maybe not senior, but still older. Those younger people were used to using digital tools and they’ve been trained with this new tool introduced by the company. Whereas the older group was still using notebooks. They were calling their clients with mobile. When we came in, it occurred that they have tested this training and this tool against those younger users. Where this other group, and this was 80% of those reps, they have not been using this tool at all. They’ve been noting things down in their notebooks with pencil and the old fashioned way. Because there was no change management involved there. Nobody was asking those people, do you want to use it? Is this something that will help you? Will you benefit from it? This is exactly the same thing. This change management today, it’s important to implement it whenever you are building a new tool. Is it a report or a web app or something like that? But it’s even more important now when we are implementing AI tools.

[KM]
It is. It’s extremely important. Again, we’re asking users to do a massive amount of change. People don’t not adopt things on purpose. It’s not a cognitive resistance. Saying I’m not going to use this tool just because. But for reasons. We have to continue to adapt. We have to continue to see how can we streamline the way the company works. AI is one way to do that. It’s so much change that you’re asking users to do. Forgetting about them along the journey, you’re going to have more failures. You’re going to have the end where you possibly build something that’s not going to work for them. You don’t get their buy-in. You can’t always mandate people to do things. You can try. But in the end, it’s going to be painful for everybody. Also you may not have the best process. Whether you were trying to get something more efficient, if you don’t have those champions along the way, then you may end up with something that wasn’t what your original desire was. You have to have the people who are going to use it as part of this to help teach you what is their current process. But you need those people, too, to challenge the current process because you’re not just trying to build and replicate what’s already there. That’s why as you’re picking those change champions, you really need those people that are front thinkers. Those people who love to be inventive and change, but they also don’t just change for change’s sake either. They’re willing to really look at processes and say, this is how this can help be more efficient and get that buy-in at the end.

[MK]
And you need to also reinforce their authority in the company. Have you seen this report from MIT where there was this report about the value of AI tools implemented throughout 2024? And it stated that 90 something percent didn’t bring expected value. I bet most of it didn’t bring value because of lack of change management. There was no KPI for that. How do you sell the value of change management to senior management?

[KM]
Really helping them understand that in order for them to receive the value in the end, they really need the people. Educating them on here’s the issues that will possibly be the downfall if we don’t take into account the people.

[MK]
It’s a cost, it takes time, involvement. It’s painful. They need to be really involved instead of just implement because they have their KPIs. You need to implement this tool. It will bring value. We do need that. But this change management, it’s the same effort as implementation of this tool. Why should we do it? Is it a cost or an investment? The same with UX.

[KM]
It is an investment. It’s an investment that you’re going to get the value out of the tool. As you said with the MIT, with other experiences and everything, more often than not, and especially if you look at AI, it’s going to fail because people aren’t going to use it. If people also feel that their job is in jeopardy because proper communication wasn’t done, they’re going to resist it. They will do anything in their power to work around whatever tool that you’re going to build. Not that they’re making a conscious effort in it, but you spend all this money on putting in a tool, and if you don’t have people who are going to use it, or it ends up being built where it’s not going to receive that value. It’s not going to be more efficient, which is one of the biggest things people are looking at AI to do. Then you can spend all this time and money, and you’re still going to end up where you have a tool that isn’t going to be able to be used. And then also people’s worth. How much they enjoy working at your company, that will continue to go down because they’re going to feel like you don’t care about them. And that’s a big deal, especially in this AI trend when you constantly see, what was it last week? I think Amazon came out and laid off 13,000 people in the name of AI.

[MK]
They will lay off even more because they want to switch from people to robots. Something that I’ve heard recently.

[KM]
When you have all of that in the news.

[MK]
It builds fear.

[KM]
It builds fear.

[MK]
And resistance.

[KM]
Exactly. Those are some of the principles that we also talk about, especially around AI and the importance of change management in AI with senior leaders, because you do see all that stuff. It’s not just what’s happening in the company. It’s what’s happening outside the company that people are also reading and seeing.

[MK]
Kylene, thank you very much. This was a very interesting talk.

[KM]
Thank you.

[MK]
Thanks, everyone, for being with us. Be sure to leave a comment and subscribe to our channel and see you in the next episode of C&F Talks.