Dante Healy [00:00:02]:

Hello, everyone. Welcome to another episode of Business Breaks Your Project Management Edge. Now that we've covered the people side of change, in this episode two of a two part series on change management, we're going to talk about measuring success, and whether your change is actually working. So we'll get into defining KPIs, and learning from both successes and failures in transformation. So stay tuned. So change isn't just about implementations. It's also afterwards, assessing the impact and ensuring that your efforts lead to tangible results. And in this episode, we'll dive into practical metrics and tools that can help you evaluate the effectiveness of your change initiatives.

John Byrne [00:00:55]:

So starting off, John, in terms of defining success, and being clear about what success looks like. What have you encountered in terms of projects, and where does it all start from? Is it at the inception when you create your project charter in your business case? What does is is success about really pure ROI, or is there anything else that goes beyond that to determine whether a project has been successful or not?

Dante Healy [00:01:28]:

I'm probably going to be a little bit, you know, controversial here and say, for most projects, even systems implementations, I think ROI is pointless as a KPI. It's made up. You're guessing. And there'd be very rare that you'll actually get to work it out because by the especially on a big project, by the time it goes live, so many other things will have changed. So any change, you know, your your ROI may be great, but it's not from what happened to do with the project or it might be terrible because other things have changed. And, ultimately, I think the best measurement of change there's there's lots of so you you come up with, you know, as far as talking about our management account backgrounds, they had the critical success factors and KPIs were used then to try and measure the critical success factors almost. And I think the critical success factor is are people using the change? Has it has it embedded? Was the change good enough that it makes enough of an improvement over what they were doing before that they're actually now using it. Even the people who were against the change early on, have they started using it.

Dante Healy [00:02:40]:

If everybody is using it and is kind of happy with it, then it was successful. And that's the ultimate. All your KPIs then are just kind of trying to reinforce that that is not what happens. That that's my opinion. Anyway, how about yourself? What do you think about ROAS and and other various KPIs for for change management and success.

John Byrne [00:03:01]:

Well, it's funny you should mention that because I probably I think this is the first time we'll probably be slightly on opposite sides of the argument because I've I've been in projects and transformation projects where in order to get the funding approval, we've had to have an ROI. It's interesting the ROI in accounting and finances. We're a cost center, So the benefit is we gotta cut our costs in order to be competitive. And that meant efficiency, re engineering processes, making sure we could cut costs in order to justify the investment in the project. Now the tangible benefits were cost savings. First one was wage arbitrage. So moving routine tasks from high cost countries to low cost countries through shared service outsourcing. And that was all managed in house, so it was pretty controlled.

John Byrne [00:03:59]:

The second one was rationalizing systems. So where you save are on licensing, server costs, and IT support, and maintenance. So rather than having five different systems, we had one common system. So common operating model was a compelling driver for change. Now admittedly, a lot of these costs when you actually tally up the numbers versus the original assumptions, your benefits are usually never as high as you originally made out because you still need an a certain element of support. So you're not gonna cut off support completely. There'll still be some residual licensing costs even on legacy systems for maintaining archives of historic data, etcetera. And also, the cost of of having to decommission your system.

John [00:04:54]:

So there were things that were missed on the original cost assumptions. But generally speaking, yeah, there are things that are tangible. If you're talking about ROIs based on growth, I think those are harder to measure. But costs definitely are an easy one to quantify.

Dante Healy [00:05:14]:

Although, a lot of the projects I suppose I've worked on have been, you know, there would be a lot of automation and that will we're not gonna cut costs because we're not gonna be firing people as a result. We're just gonna be freeing up their time to add more value to do value added tasks. And one thing that no company ever seems to want to be willing to quantify is they they won't, the the management, the top teams and they won't accept as a a cost cutting thing being able to tell them, well, you won't have to hire more people to do the same look. You know? That that that's not something that you know, as things are in a manual system, if you double the size of the work the amount of work done by by by, growth, you will have to double the size of your finance team. And that that makes sense, but they will not accept well, if we put in this new system and automate a lot of it, the existing mat the existing accounting, the assistant existing finance team will be able to handle this growth without having to hire twice the number of people. And the amount of times I've been told in the past that that's not something that they we can put in in the business case, that they will not accept or you won't have to hire more people as, an amount. And therefore, well, that's your return on investment. That's where your savings are going to be, so your return on investment has kinda become useless.

Dante Healy [00:06:35]:

I mean, you still have to do it. Everybody has to come up with a return on investment. But from measuring the success of a project, I don't think we're doing an investment on the types of projects that I know.

John Byrne [00:06:46]:

I guess it depends on how you analyze and frame it because what I found is that as businesses change their operating models, you find that transactions tend to be smaller, but they're they increase in volume. And therefore, in order to support the increasing volume of transactions being processed, these investments will actually prevent you having to double your team or burning them out. And yeah. But then, that that ties your physical performance to the actual cost performance then in that case. And it might not be growth in revenue because your net your net revenue per transaction might be dropping. But at least at least your cost base won't be doubling up if you're getting double the amount of business transactions going through.

Dante Healy [00:07:37]:

That's it. And it's very difficult, though, to get a lot of to get them to accept that. And that it's like, oh, yeah. We were never going to hire double twice the the amount of finance people. And, anyway, we were just going to leave the finance team as it is. And, anyway, I'm trying to say, yeah. But then your finance team as they are now would be born out if they they try

John Byrne [00:07:55]:

And the existing team will expect pay rises in line with inflation, so you have to support that as well. So if if they're saying, yeah, you can get 10% on the cost, but you have to pay your team 10% more because, I don't know, costs have gone up across the board 10%, and they need to keep up with their living standards to keep them happy or stop them from being outright rebellious, then yeah. But at the same time, there are there are things certainly maybe I was in a fortunate situation or unfortunate that there were people leaving positions that weren't backfilled. So they're through natural attrition, retirements, etcetera. But then there was also redeployment, so shifting people who were surplus to finance, but could be redeployed to sales, customer support, operations, or other departments. Because they they'd have attrition as well. And rather than hire externally, we could redeploy them elsewhere.

Dante Healy [00:08:56]:

I think those well. A lot comes on where the change came from. So that could be a part of where their experiences are are different. Often, I've gone in with the project management, putting in new systems and that, and that change has come from a point of where the previous system is basically broken. Whereas if its system is not broken, then, yes, you can kind of take people out of it when you've you and you've improved it. You can take people out of that. But when the system is already broken, it means they they're understaffed as is. So you can't let all people.

Dante Healy [00:09:29]:

You're putting in a system so that the the existing team Yeah.

John [00:09:32]:

Yeah. Yeah.

Dante Healy [00:09:33]:

Yeah. But it's very difficult to put an amount. How how many extra people would they have been willing to hire? You know? They're they're not willing to hire any, so how many can you convince them they would have happened to hire? You know? And that's that's where it comes down to your return your return on investment is going to be. If that's what what you're you're measuring.

John [00:09:49]:

You're avoiding incremental cost. Yeah. It's basically what you're doing by investing.

Dante Healy [00:09:55]:

Exactly. But, they they will never accept the incremental cost saving as, a genuine thing. They are looking at this.

John [00:10:01]:

Well, they can't see it. It's not a saving for them because it's not actually coming off their cost base. Their base is flat, but what you're doing is reducing the increase.

Dante Healy [00:10:10]:

Yeah. Exactly. So in in that regard, you know, we're when that's your kind of, the project now what difference is how projects some projects will be revenue generating, and then, obviously, return on investment is a valid better. But, when when you're doing a systems implementation or a finance transformation where there's not really extra revenue, costs are a little bit, you know, cost cutting is a little bit yeah. We're just kind of making this up to make it look like it's a good business case, and that's where it is a lot of the time. You're you're taking a bit of a guess because you know full well that yeah. But by the time we go live, other things will have happened that will have impact on our ability to actually cut those costs.

John [00:10:56]:

Yeah. You can even I I mean, we've we've done scientific workflow estimates where we measure for each task how long it took a person to perform the task per transaction. Usually, a lot of them, if they're any good, they'll batch and schedule. So they won't just, you know, process one invoice end to end, and then move on to the next one. They'll get five in a batch of a hundred invoices, do one task on them, and then do the next task on them, and then the next task. And usually, it's handed over to different people. One team will process the invoices, another team will go out and make sure that they're approved, another team will check those approvals and make sure that this the correct supplier is paid.

Dante Healy [00:11:36]:

Yeah.

John [00:11:37]:

And and it can be as complicated as that, especially in global organizations where you have distributed departments.

Dante Healy [00:11:46]:

Yeah. And and that's why I'd be kind of against using ROI as a measure of success, purely. As I said, some projects that they lend themselves to, especially if it's a project that's going to improve revenue, you know, putting in a a new, you know, project to set up a new product or something like that brand. But on the day to day ones, it's there. It's needed. I mean, it's nearly always in the business case, you're gonna have to come up with something. Mhmm. But it wouldn't be what I would use to measure the success of the project because it's it's a little bit too airy fairy.

Dante Healy [00:12:22]:

You know? It's it's it's it's rare that you're gonna actually get that return on investment because other things will have changed in the interim that and I'm trying to well, what impact did your project have and what impact did these other side things that have nothing to do with your project? Yeah. Very difficult, and they'll they'll all be tied in together.

John [00:12:41]:

I can see that in itself would justify if if a system is broken or it's on course for being unsuitable for your future path. Say your business is growing. I don't know. 20% year on year. But your system can only support the current rate of growth up to the next three years and beyond that, you're gonna break it. Yeah. And that's that's that's a justification. Right? That's continuity of service.

Dante Healy [00:13:09]:

Exactly. But it's very difficult to make an accurate return on investment data bank. You are making lots of assumptions in order to do it. You have to do it. Your business case

John [00:13:19]:

Well, you can see it's coming. It might not be precise, but you can see a trend.

Dante Healy [00:13:23]:

And it's not

John [00:13:25]:

it's it's it's not a blip. It's actually a proper trend.

Dante Healy [00:13:29]:

Yeah. And and that's it. So it it it's a bit loose with the the numbers. So that's why I I you know, you need a return on investment to make the business case. But to measure the true success of the project, I think other things are more important. Obviously, not to the main, you know, to the to the board.

John [00:13:47]:

So you're saying it's it's it's just a collection of intangibles, and the tangible piece, that's TBD.

Dante Healy [00:13:54]:

Yeah. Basically. Yeah.

John [00:13:55]:

Because it will be buried in other things. And I've seen it where you've made assumptions that the system will automate everything. And guess what? There's this one piece that actually increases the amount of work that someone does. So instead of getting rid of people, you've had to double up to compensate.

Dante Healy [00:14:13]:

And all the system the new system has done is is stop you having to actually double up that maybe you only have to fire a few more people to do it because of the new system. But, you know, trying to get that, how many would they have been how many would they have had to hire? How many did they have to hire? And even things like saving time, like you like you said, it'll turn out that, you you know, put put in an EPM, for example, and trying to get a return on investment, how much time would be saved at the month end? And our thing was, yeah, the the month end process, they're not going to save any time. As if from a noticeable point of view, the month end process will still end when it ends. Now the difference will be that the system will mean that people are not having to walk twelve hour shifts every day during the month end.

John [00:14:59]:

To meet the closed deadline.

Dante Healy [00:15:01]:

Yeah. Which again a lot

John [00:15:03]:

I remember I pulled twenty four hour all nighters. Yeah. And they were not fun. No. You'd you'd be burnt out until the next close.

Dante Healy [00:15:11]:

But trying to get that in as a return on investment is very difficult because the like, the board, especially if it's a big project, the big financial transformation, the board doesn't care because the the board were paid you a salary, so you weren't getting paid overtime for working twenty four hour shifts. All the board cared about was that on day 10, they had their accounts back, and that was it.

John [00:15:31]:

They had their numbers. They knew what the profit was.

Dante Healy [00:15:34]:

And and the fact that you and your whole team were, you know, having to go see psychologists and were on medication for all the twenty four hour shifts you have to pull during those ten days is irrelevant to them because they just know they got their report. And then you try to make a return on investment for we're putting in this new system. What will this new system mean? It'll mean you and your team only have to work eight hours a day, and they'll still get their report on day 10. Well, to them, there's no improvement. They're still getting the report on day 10. They don't care about how what they're not paying you extra. You know, if they were paying you overtime, then they might

John [00:16:08]:

a bonus, but that was worth a fraction of the unpaid overtime.

Dante Healy [00:16:13]:

Yeah. You know what I mean? And the bonus you're gonna get in anyway because you're still making your targets, It's just but the new system has made it easier for you, and and that's very difficult to measure. So Yeah. And and that is the biggest key to, I think, is the yes.

John [00:16:27]:

Yeah. Yeah. And and the other thing, especially in finance transformation, is you've brought back some unpleasant memories that, you know, we've been working on initiatives to automate stuff. And then the business would do a simple change, and that would eliminate the need for a certain task to be done. Even like a whole department, if you just reorganize the operations, makes things so much easier. Because finance is a support function, And unless unless they're driving commercial decisions, then they're not they're gonna be at the mercy of whatever structures are in place on the operation side.

Dante Healy [00:17:10]:

But even with that, like, you know, you put in an EPM system and that that does drive commercial decisions.

John [00:17:17]:

Yeah. If you if you try and do something outside of the process, then it will it will it will require something the outside of the EPM because the EPM won't accept it.

Dante Healy [00:17:28]:

Well, it's not even that. Well, how do you how do you measure that? How do you put a return on investment with that? Yeah. We're gonna put in a new EPM on this. We'll drive commercial decisions that will make your the commercial decision make it so much easier because we'll be able to do what if analysis, this, that, and the other. How do you put a value on that? Because that's not the the EPM will facilitate making better decisions, but it will not be the thing. How do you how do you put a value on what those decisions will be worth when they're made?

John [00:17:56]:

It's either gonna be how much to set it up or how much to operate it manually.

Dante Healy [00:18:01]:

But that's the cost part of it. How do you put a value on to get your return on investment, how are you saying what your return is?

John [00:18:08]:

You measure the value of the contract. The revenue you get from the customer.

Dante Healy [00:18:14]:

You put an APM in.

John [00:18:18]:

Well, you've got customers. I mean, it depends.

Dante Healy [00:18:21]:

It's impossible to do. At this point, it's impossible to do. You're putting the EPM in, which will facilitate better decision making.

John [00:18:28]:

I guess you could do direct. Well, it depends because

Dante Healy [00:18:33]:

You could do it.

John [00:18:34]:

I get I get what you're saying.

Dante Healy [00:18:35]:

You can do it after the fact in in, you know, we put in an EPM now. In three years' time, you

John [00:18:40]:

can Oh, yeah. Yeah. Yeah. I I hear what you're saying. When you're trying to project or forecast what's going on, it's gonna be estimates. That will have to be it will have to be estimates. Yeah. And even when you're doing doing cost allocations because you can you can you can allocate direct cost to material and customers.

John [00:18:59]:

Right? Mhmm. But indirect costs are gonna be impossible even with an ERP. And that's And those will be estimates.

Dante Healy [00:19:07]:

And that's exactly why I think

John [00:19:10]:

Allocating overhead is just a guess

Dante Healy [00:19:12]:

as well. And you take into account that most of my projects are ERP and EPM implementations and and the the these things. So that's why I don't think, an ROI is a good measure because,

John [00:19:25]:

you know, I wasting your time because it's just guesses.

Dante Healy [00:19:27]:

It's just guesses.

John [00:19:28]:

And also also agreements internally. Yeah. I I don't want this cost on my cost center or my manager from the other department. Can I charge that out to you?

Dante Healy [00:19:38]:

And the other thing is as well then how much of it when when the EPM will facilitate good decision making, but even if you even if you guessed exactly how much they they will make on their better decision making in the future, that doesn't come down to the EPM. That comes down to the person making the decisions. But the EPM facilitated it. So Yeah. Yeah. So the return on investment is guesswork. If if you make more, it's because somebody made better decisions. And if you make less, it's because somebody made worse decisions.

Dante Healy [00:20:07]:

But the EPM is where it is.

John [00:20:09]:

If someone's running their team at half the number of people because they've reprioritized, that might not necessarily be because of the system either.

Dante Healy [00:20:19]:

Exactly. Yeah. And so no matter what decisions, the the assumptions you make, there's too many outside factors that will affect the return on investment. So so that is why I don't use it as a measure of success for any of my

John [00:20:31]:

It's a moving target on top of a moving target. So then how would you measure success? What are the intangibles for a system? Let's let's focus on one system implementation. What success look like?

Dante Healy [00:20:46]:

System implementation, I suppose success. Process implementation is almost a little bit easier. Success is either using the new process or have they kept going back to the old process.

John [00:20:57]:

Adoption rates.

Dante Healy [00:20:58]:

Yeah. Adoption rates. System implementation, well, chances are you shut down the old system so they've no choice but to use the new system.

John [00:21:06]:

And then you you switch costs. You've you've effectively reduced the cost of the old system. If it costed more to maintain because it was, say, in house server. Although, I'd argue cloud is more expensive a lot of the time.

Dante Healy [00:21:19]:

There there is that, but that's kind of getting back to return on investment type of things. Whereas I'm I'm thinking ultimately, it's it's it's it's, are the people happy using the new system after they started using it for a while, Or are they longing for the old days all the time? If they're long for the old days, then you probably didn't make it. It wasn't an improvement. The new system was not better than the old

John [00:21:42]:

So staff morale?

Dante Healy [00:21:43]:

Staff morale. Yeah. Staff morale.

John [00:21:45]:

And that's impossible to measure because that's very intangible as well. Maybe they're working harder, but you give them a bonus or a pay rise and they're happy again.

Dante Healy [00:21:53]:

Yeah. That's it.

John [00:21:54]:

And it could be, you know, you you send out that morale survey after you've given them a pay rise and hope that they acknowledge it.

Dante Healy [00:22:02]:

That's it. But then then you're twist. And, or, you know, I I I I worked in a place once that, they they did a quarterly feedback form, you know, how are the company doing?

John [00:22:14]:

Surveys. Yeah.

Dante Healy [00:22:15]:

Surveys. Yeah. And, the the the results of that was tied into the the the amount of bonus that the the thing got. They they a few different ways to measure bonus, how the company did overall.

John [00:22:26]:

I'll tell you what. Nine 99% of those surveys, it's the number one problem was always compensation.

Dante Healy [00:22:34]:

But the problem really is with the way this company was doing it was part of the the weighting of the bonus was related to how well the company did in that survey. So who the hell is gonna give this company a bad? If you give the company a bad score, you're lording the bonus. So, you know, they they they and they they didn't do it deliberately. It was just they did it where

John [00:22:57]:

what it was. Yeah. Yeah. They

Dante Healy [00:22:58]:

did it without really thinking through. Hang on a second. If we want honest feedback, it should not have any bearing on my

John [00:23:04]:

This was a broken HR process. Yeah. You know what's funny was that when when we got it, it was always when I was a manager, it was always every time compensation was always the big problem. And yet, the one thing I, as a manager, couldn't do was determine how much my team were paid. And therefore, all I could do was try and boost it by having team events, going out for meals and drinks. So

Dante Healy [00:23:34]:

to get the measure of the success of a system implementation, A questionnaire could probably do it. A a but people will be I don't know if people will ask even if it's an, an anonymous one. The teams involved are usually so small that you can almost guess who's answering what.

John [00:23:59]:

Yeah. It's not anonymous. So it's coercive. So you basically if if if you really wanna have been caught in the crosshairs, you you actually say what you really think if you're negative about it. And then the manager will be on to you saying, why don't you like the new system? What can we do to make it better? Or or or better yet flip it, boomerang it back to them and say, what can you do to make it better?

Dante Healy [00:24:23]:

Yeah. And just give

John [00:24:24]:

them more work.

Dante Healy [00:24:25]:

That would be a a but and then that's the thing. So I don't think I don't think questionnaire would work for, you know, system adoption. How else would you measure it then? Well, I suppose other things like well, if you know how much time people were spending in the office working on us, the old system, has that improved on the new system? If they're still working, yeah, if they're still having to pull sixteen hour shifts during month ends that they were pulling on the old system, then the new system didn't really improve much, did it? Unless you have the team or something like that.

John [00:24:59]:

Unless you've added more works, that much more work that they're still carrying on, but they're doing they're more productive because they're producing more. They're doing their what originally took them sixteen hours plus for the same sixteen hours.

Dante Healy [00:25:13]:

True. So well, then, I suppose productivity. A measure get a measure of productivity and and stats your is has productivity increased that if so, the project the system's implementation was a success. The the change was worked on.

John [00:25:28]:

Yeah. And in terms of productivity and units of work, you'd have to measure the units. Yeah. Are they doing their best base job plus additional duties?

Dante Healy [00:25:41]:

I although I I'll ideally, I will say I don't think anybody should be having to work eighteen hours or sixteen hours away even during the Yeah.

John [00:25:49]:

Yeah. Yeah. Agreed.

Dante Healy [00:25:50]:

Agreed. If they are, then either the system if they are, then, basically, there's a fault there with the management. Either the managements are not providing the right system or the management have not hired enough people.

John [00:26:02]:

Yeah. Agreed.

Dante Healy [00:26:04]:

That's outside the project manager's role because the project manager doesn't get to hire anybody. They just get to put in the system. So measuring the success of the system with SQL productivity, if if even if they are having to work more than I think they should be having to work, are are they being more productive during that time? That's the new system then is beneficial, and it's just bad management if they're still having to do sixteen hour shifts.

John [00:26:26]:

Makes sense. So maybe this could be a very short episode.

Dante Healy [00:26:33]:

Well, I mean, I suppose there are there are other, things that need to be

John [00:26:37]:

So so we're talking about qualitative and quantitative measures, and trying to make sure I still think there's some there's some mileage in actually trying to quantify the benefit, whether it it can be quantified in, you know, maybe it's not true financial savings. It's not bottom line. But it could be bottom line reductions of profit avoided. So maybe you've avoided having to spend more on certain other items, like additional stuff because you've automated.

Dante Healy [00:27:11]:

I think the, I mean, the key thing is with with the likes of return on investment, I have never come across a business case that didn't require you to put in a figure there. So you have to evaluate. I just after the fact, if as as a as a as a project manager, I wouldn't I I wouldn't use that to determine whether you felt your project had been a success or not because you've no control over Yeah. And the thing. But things that you can well, the completion the basic stuff. Did you complete the project on time, in budget, and deliver? Scott?

John [00:27:47]:

Yeah. But that's execution performance. It's I'm thinking what happened after you executed. How do you make sure that the benefits are there? I'm sorry to to go on that, but, you know, pure project performance is easy to measure Because you know how many people are on the project, and have been assigned. You know how much it's costing to develop the system. You know how much time you have, and when you have to deliver. And if it goes into an overrun situation, then it will show up on your cost center. But I'm I'm just thinking in terms of is there data when you put an ROI together, is is it backed by assumptions or is it backed by say historical data? Maybe it's number of transaction, transaction volume, number of FTEs, cost cost per cost center.

John [00:28:44]:

And you're right. You can't just say I've implemented a system now. We're gonna assume some way or another we can shave 10% off our cost base.

Dante Healy [00:28:52]:

No. For example.

John [00:28:53]:

Yeah. That that that's just fiction. Right?

Dante Healy [00:28:57]:

Yeah. That's it. So well, your efficiency. So, yeah, are you able to, you know, to either do the same amount of work with less work hours, whether that's less people or whether that's just people not having to constantly stay late? Or are you able to do more work in the same number of hours? Or better still, the the ideal situation, can you do more work in less hours? And if you can do more work in less hours, then the system now, again, we're talking about well, I'm talking about finance transformation thing. So button in systems, obviously, the the likes of ERPs, EPMs, and and all the affiliated stuff, you know, traveling expenses, push systems, all the rest of it. I'm thinking them. So, obviously, all our projects will have different measures, different means depending on on what the project was. You know, projects could be anything.

Dante Healy [00:29:50]:

I suppose that's one problem we have is, we're coming at it from finance side of things. So we're really thinking about projects that are all cost, and you're trying to guess what the savings will be.

John [00:30:02]:

Rather than coming at it from the idea of revenue growth and then determining what the future growth projections are gonna be.

Dante Healy [00:30:10]:

Now they're difficult enough as well even if you are a project analyst.

John [00:30:13]:

Easier to analyze cost than analyze revenue because there's something tangible to hook onto.

Dante Healy [00:30:19]:

That's it. Yeah. Yeah. There there is. And and revenue the the project could be a complete success, but the market completely changed. So you didn't get the revenue you were expecting and had nothing to do with the success of the project. It was just

John [00:30:32]:

Yeah. You might have got increased market share, but the market might have shrunk. Yes. And you're still back to net zero in terms of revenue growth.

Dante Healy [00:30:42]:

Exactly. So, you know, and then it's a cost base then. Incremental costs tends to be very difficult to judge. You know? We're going to be able to hold the number of people as is at the moment and double the amount of work that we we are able to get done. But then you still end up having to hire some people because you've actually troubles the amount of work. So how do you, you know, measure that? And then there's other things as well. I mean, you know, perhaps you're getting ready to go on a you know, we're we're walking there off organic growth, but maybe you're gonna go on a purchasing spree and purchase some

John [00:31:21]:

other Acquisitions. Yeah.

Dante Healy [00:31:22]:

And the new system will just make it easier to to bring them in.

John [00:31:26]:

Realize synergies and, you know, absorb the new business you've acquired with the minimum cost of change.

Dante Healy [00:31:34]:

Yeah. Exactly. But trying to put a figure on that in advance is very, very difficult.

John [00:31:40]:

It's gonna be hard because, anyway, who's to say the synergies are there? Because strategically, you may you may think you've bought a very identical business, but it might be completely different in the way they operate and how they serve their customers. And then you've really got two different businesses rather than one one combined business.

Dante Healy [00:32:00]:

That's it. I think we did discuss way back when when we said

John [00:32:03]:

Yeah. Yeah. The success of of acquisitions or something. Yeah.

Dante Healy [00:32:08]:

Two companies that might be identical but have completely different cultures, will make that that cultural, amalgamation is more And

John [00:32:17]:

the culture difference creates different operating models and business models.

Dante Healy [00:32:22]:

That's it. And so trying to come up with a way to say whether your project was a success, your the change that you made was a success based on that even is is going to be difficult. That, you know, the the new system didn't do a heck of a lot to help us amalgamate. And you can try and try and say, yeah. But the old system would have made it a complete failure or we were only a partial failure. How do you you know, that is very difficult. So, yeah, it's it's, I'm just having a quick look at some notes here that that we I suppose one of the ways, which again is not a hard and fast number, so you probably won't be happy with the acceptability. You know? Are are the stakeholders, are they satisfied, and are they engaged with the new change, and are they using it? Mhmm.

Dante Healy [00:33:06]:

Where they have a choice? You know, obviously, a new system, they don't have a choice. But a new process, they do have a choice. Yeah. Are they are they using it? Do they I

John [00:33:14]:

think yeah. And at what point, when's that sweet spot where you you say acceptability is, for one of a better word, acceptable in the sense that it's run on long enough that you can say, yes. This is truly embedded change. Yeah. Because it could be, you know, coming back to last last week's episode or last time last episode. You know, people might go on until hyper care, and then once once people's eyes are off it, they revert back to type.

Dante Healy [00:33:48]:

Yeah. Exactly. So maybe this is, you know, these are probably true successes is measured on a a more medium term, you know, maybe a year, two years, three years later.

John [00:33:59]:

Yeah.

Dante Healy [00:33:59]:

But then with with with these types of projects with ERPs and EPMs, and if they're IT projects, even though they might be finance transformation, in three years time, what you put in now is out is is probably obsolete.

John [00:34:11]:

Yeah. Yeah. I mean, we're trying to hook on the idea that success is defined by that behavioral shift that leads to a business result. That might not be the case. That the business result might still not be realized. Or worse yet, you failed, but you still got the business outcome through factors outside of your control. Luck.

Dante Healy [00:34:33]:

Luck. Yeah. And, and it's hard to get people, you know. The the the if you

John [00:34:39]:

you overlay that.

Dante Healy [00:34:42]:

We mentioned kind of in the last episode about building that area of trust where people can give honest feedback on that. So if you have actually managed to do that to foster that area of trust, then I suppose one simple way would be, you know, you go back six months Mhmm. Later, and you ask people, was it working? Do they think that was working? If they've if they genuinely if the culture of the business and the culture of the thing is that they genuinely feel safe to say no. It wasn't worth it. Then that is that's probably the best the ultimate, source of it. It was a success. If they say yes, it was worth it, the change, then you know it was a success. But only if they feel safe to say no, it wasn't worth it.

John [00:35:23]:

You You know, it's funny, actually. I one one transformation I implemented, I went back. I didn't say, was it worth it? I said, what's changed in your world? And you know what they said? They said, well, I don't really know what we're what's gonna happen to us because we've got nothing left to do. So there is no safety, but, you know, I think that success, you've automated as much as possible. You've streamlined as much as possible. And in theory, there's people who who are struggling to justify what they have what they've got to do now. Well,

Dante Healy [00:36:02]:

that's yeah. That that's definitely success because if that's the case, then it means the system is walking the way it was. Yeah. But then, you know, you've just made some people unemployed, so that's makes you feel But then

John [00:36:12]:

they need to find some way to add value and support the business.

Dante Healy [00:36:16]:

That's it. Exactly. Yeah.

John [00:36:17]:

Assuming that they're not rather than just hiding behind administrative tasks.

Dante Healy [00:36:23]:

Yeah. That's it. That they that you're adding. And if you provide if you sell up in a way, you know, that you're providing training, not just to use the system, but training to get value from it. Because chances are, no matter how good the system is, it's not going to generate value. It's just going to facilitate somebody to generate that value. So you make sure you train them how to how to generate value that. Okay? You no longer have to do a whole load of that in in full.

John [00:36:47]:

You're not posting manual journals anymore. I'm not

Dante Healy [00:36:49]:

posting manual journals anymore. The system will do that for you, which means your time is now free to do this which you weren't able to.

John [00:36:56]:

Ask the questions. Should we be paying this much for this service, this product? Yeah. Or or should we be increasing our revenues? Why is this gone up?

Dante Healy [00:37:07]:

Exactly. And I think they're, you know, the the errors. Yeah. Jeepers, I you know, we started out with our

John [00:37:16]:

Agenda for KPIs.

Dante Healy [00:37:17]:

And, and we haven't really given any answers, have we? That it's very difficult.

John [00:37:22]:

I think we've concluded that we if if we're being honest, that the ROI is usually a forced financial measure that is based on assumptions. I'd still hold on to the fact that within limits, I that as long as you have a reasonable understanding of your cost base and your cost, you can still project something, but even that will change. Yeah. Because of other, you know, businesses are dynamic. So you have to be able to communicate clearly and overlay those die dynamic changes. For example, I was cutting costs or trying to cut reduce costs during a time when we knew our operating model was uncompetitive. But then Brexit came on, and what it ended up doing was limiting the impact because but what we did avoid was doubling our staff. We only had to increase it by about 10% rather than a %.

Dante Healy [00:38:26]:

And, you know, that's, I suppose that's that's the way to do it. Isn't it? Your k KPIs, you'll have them at the beginning. And you you have to set them early and and, be clear with them at the beginning. Yeah. But don't stand when you're trying to measure after the fact, was it a success? Don't go back and look at the KPIs you calculate. It's the actual numbers you calculated at the beginning because so many other things have changed. Recalculate what those numbers are.

John [00:38:54]:

What are the drivers as well, you know? It's number of customers, number of contracts.

Dante Healy [00:38:59]:

Yeah.

John [00:38:59]:

Number of yeah. That those sort of things you can hook onto. Yeah. And Transactions. Volume of transactions. Yeah. So recalculating them.

Dante Healy [00:39:08]:

Recalculating what they would have been if you if you can. You know, roughly, what would they have been had we not done this change? What are they now? Is what they are now better than what they would have been? And if the answer is yes, then right? So, yeah, I I take it back then. I suppose the ROI the the basic KPIs that you would do, like ROI, they are worth doing, but just don't stick with the original number.

John [00:39:33]:

And understand your number.

Dante Healy [00:39:34]:

Understand your number. Exactly. And then at the end, recalculate and take into account the things that changed that had nothing to do with change you Yeah. And and

John [00:39:44]:

some Overlays. You need to understand your cost base, the cost drivers, and then when you revisit your assumptions, because you've declared them, you should be able to understand the overlays. Because you can revisit the data at the start of the project versus the data at the end of the project, and then determine, well, given the change, are we more effective? And it's probably more relative measures than absolute measures and numbers in context.

Dante Healy [00:40:13]:

And and then also try to pair those particular metrics, you know, with

John [00:40:20]:

Qualitative measures as well.

Dante Healy [00:40:23]:

Yeah. Yeah. So, you know, get get feedback with true surveys and and things like that. It's assuming, like, if you've done what we suggested that the psychology of change piece, if you've made a trusting environment where people feel they can give negative feedback

John [00:40:38]:

Yeah.

Dante Healy [00:40:38]:

Then it's worth doing. And because then if you don't get negative feedback, you at least know it's because people are actually happy with the system, not because And

John [00:40:47]:

if you

Dante Healy [00:40:47]:

the same.

John [00:40:48]:

Yeah. And if you do get negative feedback, see what you can do to fix it.

Dante Healy [00:40:53]:

Exactly. I mean, that's the thing. The trust has to continue on even when the project's set. Not a

John [00:40:57]:

one and done. You know?

Dante Healy [00:40:58]:

You don't

John [00:40:59]:

say, well, that's my job done. If you're still if you're still in the organization, you should be doing everything you can to make sure Yeah. That something's left, a legacy or whatever.

Dante Healy [00:41:10]:

So you get the negative feedback then you follow-up with it. Don't just ignore them. Acknowledge it. And and see, you you know, go and and and figure out right what and ask them, well, what do you think we could do to fix this? I know you said it kind of joking earlier and then leave them to fix it, but but that's not what to do. It would be see if they have some advice, try and help them where and then see if, you know, as a smaller project, implement that that fix to try and make it even better. That that would probably be, you know, the the the and don't be yeah. I suppose they're just kind of main things I can think of. And I

John [00:41:52]:

I just think you you don't wanna be flying flying blind. No. Because someone's going to ask, well, why did you do that? And if you can't justify it, I think there's you're always gonna struggle saying, well, because we we we had to deliver a system that was in the project charter.

Dante Healy [00:42:11]:

Hello. No. I would never say like, I'm saying that I I'm sure purely meant, like, like I said, you have to do with the return on investment when you're doing the business case and that. But I just mean don't use that figure Yeah. As a measure of, did you actually achieve it? No. Then you failed. No. Too many good things.

Dante Healy [00:42:27]:

But as we kind of were mentioning there at the end, well, at the end, recalculate what would it have been had we not made the change.

John [00:42:32]:

And just make sure you understand the numbers and that they're correct. Even the non especially the non financial piece. Because if you can't if you can't tie the non financial to the financial, then you've got no no way of tracking. It's just a, you know, a cost could be x x hundred of thousands of pounds per year. And then or euros or whatever. And then the did it go up or down? And it might go up, but that's okay. If everything else went up, all things being equal, maybe overall, you were better off still.

Dante Healy [00:43:05]:

Yeah. That's it. I mean, yeah. Like that, you know, as you said, we you calculate, you know, the license for the old system is this, the license for the new system is this, and then it turns out, no. The license for the new system was higher, but then you look but would the license for the old system have stayed the same? No. It would have been twice as high. Okay. Yeah.

Dante Healy [00:43:22]:

Still beneficial. Just their exact numbers aren't the same, but the the idea is is right. So the actual what you're measuring is right is just the time.

John [00:43:31]:

Yeah. Exactly. It's the base is wrong, maybe.

Dante Healy [00:43:35]:

Yeah. Exactly. That that's it. And and actually that's a key one. I mean, you know, chances are when you're doing it, you're gonna be told what your KPIs are. You know, a lot of, PMOs and now you're putting together your business case, they're telling you calculate your return on investment, calculate a time saving if there is one, calculate this or that or the other. So you're gonna have to do them both to measure yourself whether your project was a success or not. Don't use the exact numbers you used before you start the project

John [00:44:03]:

because Mhmm.

Dante Healy [00:44:03]:

So many other things will have changed. You may have to try and recalculate.

John [00:44:07]:

Yeah. And and there's other things, you know. Certain things don't, you know, it's not, you know. I know I've been hopping. I've been vigorously defending having a tangible measure of success, but there was one system I implemented. It actually added cost, and it was a financial forecasting tool implementation. And coming back to it last week, I was listening to the people in the audience. So many people saying how I love this.

John [00:44:41]:

How I love this new tool getting off modeling in Excel. Being able to see all of the different roll ups and having it forecast automatically for you using AI, so you can focus on questioning the numbers. So much positive feedback and so much, you know, I love this system. Why didn't we have it sooner? That in itself is ultimate success.

Dante Healy [00:45:09]:

Exactly. And that's the the ultimate success. Yeah. In my opinion that even if you didn't meet all the other return on investment, all the other KPIs, people were using it. And it'd be impossible to come up with a realistic return on investment for that system.

John [00:45:21]:

Yeah. And it But having happy employees, what what more do you want?

Dante Healy [00:45:25]:

Exactly. And and employees, if they're feeling like that, that means it is adding a lot of benefits to the business. Yeah. They're making better decisions.

John [00:45:32]:

Yeah. And they're and they're not wasting time in the grimly putting numbers from one box to another box.

Dante Healy [00:45:41]:

That's it. And then looking at the numbers and thinking, oh god. That's that's completely not what I was expecting. And realizing that you've done a formula wrong somewhere hidden in some other page on Excel, some other sheet, and that's pulling in information from where is any the the that kind of forecasting planning EPN tool is just so much easier to find when you do make a mistake.

John [00:46:01]:

And the and the person at the top who or in corporate who's trying to question why did why is that number different? Oh, it's the fudge factor. Yeah. Yeah. So thank you, John. I guess that wraps up this second episode. I guess to conclude, what what were your takeaways from this conversation?

Dante Healy [00:46:27]:

Measuring success is is not straightforward, and there's different types of success. Yeah. How about you?

John [00:46:37]:

I think for me, and I didn't I didn't really mention it during the conversation, but this is what I get, is it's really important to define what what does success mean, and then you you anchor to that because that's going to be your North Star when you proceed with it implementing the project. And then, obviously, ensure you have a touch point. What point do you measure are you progressing? And I wouldn't leave it till the end. I would suggest always measure process progress to the extent you can. It depends how much time it takes because some some calculations are harder than others. But then, as you say, there's certain qualitative measures. Maybe it's regular surveys to say, how do you feel about it? And then you go, don't wait till the end. Always try and find what are the leading indicators, and ensure you have regular feedback loops to make sure you're on the right path to whatever that definition of success you have.

John [00:47:40]:

And then, to the extent you can, whilst you still have resources at play, aim to earn, learn, and adapt continuously to improve whatever it is you're implementing, because you don't wanna leave it at the end to implement process improvements. The best time to do it is as as early as possible, and ideally before you go live.

Dante Healy [00:48:04]:

And I suppose that's yeah. That just tie in a little bit then with the idea of, reevaluating, especially for long term projects. Reevaluating business cases, regular things to see is it still worthwhile doing the project. Also, reevaluate your your KPIs. Does it still make sense? Does this base that we've come up with still make sense? How has the base changed? Yeah. And and yet the continuous improvement, you know, the is it the the case on type of mentality? I think that's just just for change generally is, that's an important mentality to get into the culture of the the thing because then you you won't have to do as many big changes because you've been doing so many little changes that, oh, we don't need a big change now. We just need a slightly bigger change than what we've been doing.

John [00:48:50]:

It's not so drastic. Yeah.

Dante Healy [00:48:52]:

Exactly. And everybody's used to change. You you do small changes, but then when you do have to do a big change, it's not a complete it's not from a status quo to a big change. It's from little changes to a big change. So Yeah. I think that's a good way to have your just your business culture generally is to have Yeah. Constant improvements, rather than leaving everything to one huge project every few years.

John [00:49:13]:

Even your existing workforce is always healthy to rotate them out So they don't get stuck in their ways.

Dante Healy [00:49:20]:

Yeah. To to be constantly changing. I'm mixing people around, giving them, and that that's well the opportunities for them as well.

John [00:49:26]:

Yeah. Even if the job title changes, as long as what they're doing is different, the type of projects they're working on, the type of role they're doing. Skills should be transferable, ideally, but then you can always pick up new new new things along the way.

Dante Healy [00:49:43]:

Yep. Brilliant. That that just embeds the idea, the the culture of change within a business.

John [00:49:50]:

Yeah. And being able to change might be an ideal end in itself. The ability to continuously adapt and evolve.

Dante Healy [00:49:59]:

Yeah. Both both both for individuals and for the the organization.

John [00:50:05]:

Yeah. So with that inspiring end, conclusion, should I say, thank you very much, John. Very insightful discussion.

Dante Healy [00:50:15]:

Thank you, Dante.