Steve: Welcome to the latest episode of the ORX Operational Risk Podcast. My name is Steve Bishop and I'm the Research and Information Director at ORX.
In this episode, we'll be discussing the ORX Reference Risk Indicator Library, which is the latest in our line of structured reference libraries for risk management.
We’ll explore how this library aligns the other work we have published, and then we'll delve into other risk indicators themselves. Examining why it's important to have them, the challenges operational risk teams face with them, and some factors to consider when implementing them if you want to make them effective.
We’ll then be talking about the library itself that we developed in partnership with Oliver Wyman. I'm pleased to be joined by some co-hosts for today's podcast.
So firstly, you've got Mike, Mike Constantinou, he's a consultant at ORX and has a long and extensive background in operational and non-financial risk. Poor guy. Having worked for Barclays and HSBC previously.
Mike: Hi
Steve: And then we're also really pleased to be joined today by Tom Ivell. Tom is a partner at Oliver Wyman and heads up their Zurich office. Tom has been consulting with firms on risk management for ten years or so with Oliver Wyman. Has over 20 years’ experience in the management and measurement of financial and non-financial risks. Welcome Tom, it's great to have you on as a guest today.
Tom: It's a pleasure to be back.
Steve: Before we get into the indicators discussion, I just want to bring our listeners up to speed. So, the ORX Reference Indicator Library, is the latest, as I said, in line of strategically important risk management reference libraries aimed at supporting our members but also the wider industry with their indicators practice.
I work on standards and references commenced with the Reference Risk Taxonomy in 2019 and 2020. We've done to the award winning ORX Reference Control Library and now we have this indicator library. I think what's important to note, an important thing provides credibility is that these standards are built using our members data and with significant member and expert partner input.
So, Tom, from Oliver Wyman in this case helps us with both the taxonomy back in 2019 and 2020 and also the indicator library we’ve just published. And then we also work with McKinsey&Co is a knowledge partner for the control library. A control and indicator libraries are aligned to average taxonomy, and they set out controls, indicators that help monitor causes and preventative controls, as well as mitigating controls and impacts.
Hopefully that provides some context. It's definitely enough of me wittering on, so I think we should get into that, the meat of today's discussion.
Okay, so the first topic, and I think I'm going to turn and pick on Tom for this one. Can you start by giving us some background on risk indicators and why we think it's important for organisations to have them?
Tom: Sure. So maybe to start, I would say that the purpose of risk indicators is to allow management of a firm to track particular non-financial risks they may be concerned about. Typically, that's in the form of regular reporting, but of course also can be used as a formal expression of an organisation's risk appetite. Where you have both worded statements as well as metrics or indicators that you track.
The challenge with non-financial risk is that a lot of the risk types are quite difficult to measure because their occurrence is rare. So, you might have, you know, a rogue trader to pick a typical example every decade or so, which doesn't make for a good metric if, if that's what you're focusing on. And so financial institutions have had to be quite creative and have had to go beyond risk occurrences and what they measure.
So, for example, they might be measure metrics that speak to the effectiveness of the control environment or the inherent risk that they're exposed to. And because of that difficulty, a wide spectrum of practices has really developed over the years, and I think that's where our joint work on the library comes in.
Steve: Thanks, Tom. So, sort of from what I'm hearing on the surface, indicators should be a powerful risk management tool.
However, I think it's fair to say from our research and extensive discussions with our members on this topic, that it's an area that financial institutions generally struggle with and is seen as perhaps one of the less effective of the operational risk tools at the moment.
Turning to Mike. What challenges do financial institutions tend to face with indicators and how they implement them?
Mike: Well, we asked ORX members what their top challenges were with effectively using risk indicators, and they shared a long list of challenges. To wrap it up, the over-arching issue is basically not being able to report risk indicators to management, which are sufficiently insightful, accurate and timely to inform proactive management action. Why is this the case? Well, I'll share with you some of the main points that they raised.
Firstly, many firms struggle with poor data quality. I think we can all empathise with that, and this may either prevent a potential risk indicator being used in the first place, or if it is used, it might undermine its credibility with management if they subsequently discover that the data quality is misleading. Secondly, a majority of risk indicators is still manually collected rather than automated, and there's often a lack of quality in systems support for the timely reporting of risk indicators.
So, this means that even where risk indicators could be informative by the time the information gets to management, it's too historic to allow any pre-emptive action.
Thirdly, a majority of risk indicators used are also backward looking in nature. Such as incidents overreaches this, therefore, tells management what already happened and again doesn't really support pre-emptive action. And the final point I’ll highlight Steve, is that the highest profile risk indicators are normally the risk appetite indicators reported to board.
So, by their very nature, these risk indicators are top down and often generic in nature and therefore don't support the day-to-day management of the business.
Steve: Thanks Mike. I think that last point is a really interesting one, and I think in my experience and from the discussions that we've had, those board level indicators are often those used to support risk appetite tend to end up being in the primary focus for indicators and you could maybe argue, perhaps maybe distract from efforts to implement indicators that are sort of more effective for the day to day management in the first sign on that is definitely something that came out that will perhaps touch on a little later.
Okay. So, we say risk indicators is important but challenging to implement. So, what advice can we give our listeners on making their practice more effective? To help with this, we’ve published a guidance paper on this topic in 2022. ORX members can access this at orx.org on the website, from the reference indicator library page. And actually, for non-members who want to purchase the library, we've included it in the package. In this guidance, we have seven primary ways that we can see how you can improve practice. And today we're going to focus on the top four of those.
So, Mike, the first of these tips.
Mike: So, the first area that a number of members have focused on is increased involvement of the first line of defence.
And that involvement might be either through the selection of insightful risk indicators, or determining the thresholds, or sourcing the data, or even the automation of the collection process and the reporting of risk indicators. So, the first line of defence involvement utilises management knowledge of the business process and potentially allows resources in terms of manpower and funds to source the data collection.
Ultimately, first line involvement should increase, buy in and the likelihood of management actually taking action off the back of the risk indicators.
Steve: And I think that possibly lends itself to supporting my earlier comment regarding the need to have a sufficient focus on indicators that are useful for us managing the business and really trying to do that in partnership with the first line of defence. I think we heard quite a few success stories there during our round tables on this topic.
Okay. Thanks Mike.
So, Tom the second recommendation to improve practice is to standardise processes by developing a library. Perhaps you could let the listeners know why you think this is important, who should be responsible and how to go about it.
Tom: Sure. So, standardisation of risk indicators within a financial institution could, for example, mean standardisation across its divisions or geographic locations, and that has a number of benefits.
Firstly, the reporting becomes not only comparable but more easy to aggregate, which is a common challenge that we hear of a lot. Secondly, risk management performance can therefore be compared across businesses. So, if you are tracking risks that you are particularly concerned about, you then are able to determine who's been most effective in in managing those risks. And finally, risk appetite can also be cascaded.
So, if you think about it the other way around, standardisation means you are cascading the same type of metric down into an organisation, and therefore you are helping risk management have more bite through making those risk indicators, quote unquote local. In particular, parts of the business. Now, standardisation doesn't necessarily just mean using metrics that go by the same name, but within those metrics, you often have quite a lot of complicated definitions when it comes to what is actually being measured, what is a what is a transaction, who is a client, what counts as an employee versus a contractor versus somebody who is external.
And so, the standardisation, if taken seriously, really means to drive down those definitions into the organisation as well as the headline metric as well. Now, in terms of who does this, I think in smaller institutions we usually see the risk management department take the lead on this type of exercise, whereas in larger institutions, what you tend to find is that there are specialised teams by risk type.
Those could be in the first line or in the second line, and they would often then take the lead on those indicators that pertain to the risks they oversee. So, it's more of a federated model.
Steve: Thanks, Tom. So, carrying our countdown through our sort of greatest hits when it comes to advice? Mike, can you talk to us about the third point from our paper?
Mike: So, many members have been focusing on enhancing their risk indicators to provide a higher percentage forward looking in nature. In particular, risk indicators either look at the inherent risk and the example of that might be so third-party risk, the number of third party supporting critical services or indicators that measure control effectiveness. And an example of that might be the number of third parties with a valid contract in place.
And importantly, these can help management take pre-emptive action. Now, as Tom mentioned earlier, for some of these risk types, they can be difficult to identify good risk indicators, which is where potentially the ORX library can help. For those of you don't know, he's got 370 risk indicators mapped to the ORX risk taxonomy, and it includes a majority of forward-looking risk indicators focused on either the inherent risk or the control effectiveness I just mentioned.
And it can be used as both a benchmark and it can also be used to provide inspiration for identifying better risk indicators.
Steve: Thanks, Mike. Stealing my thunder slightly for the finish, but I'll come on to the library in a second as well and provide some more detail.
So, Tom the final piece of advice that we're going to talk about on today's podcast, our fourth recommendation for improving practice is to really take those indicators that are backward looking and try and work with them to make them more predictive and comparable, perhaps using trend analysis or relative measures.
You talked to us about why this is important, and I guess importantly, how people go about it.
Tom: Yes. So, as you mentioned, having forward looking risk indicators is an important objective for pretty much any organisation that we've been speaking to. And what we found when creating the library was that actually a majority of indicators in use today are backward looking.
So, what can you do about that? And I think in cases where a metric is inherently backward looking, such as when you're tracking actual incidents or losses, the question then is can you at least turn those into a trend such that you can track that metric in a way that is at least somewhat forward looking? And an example of that could be technology outages I think are a good example. Where there are a lot of different underlying reasons as to why you might be suffering those outages.
And it can be a lot of work to understand all of those. But by putting some trend analysis around your outages as you are measuring them, at least you are identifying trends that might be pointing towards a worsening environment or, you know, having to pay more attention to this. And so that's basically one of the recommendations to say even in a situation where the thing that you're measuring, as it were, is backward looking, right?
Because it's an incident or it's a financial loss, you can use trend analysis to at least be alerted to a situation that might be deteriorating and therefore requiring additional attention.
Steve: So, to be blunt, I think honestly, this is where risk managers can earn their money. I think turning to types of indicators you're talking about in the data into insights is really important.
The indicators can't do all of the work by themselves. And I think that sort of considering the implications of trends for the business, ensuring action is taken where required is, is really where we as a community can add value.
So, thank you both. As I mentioned earlier, there are seven recommendations provided in our practice paper, so I'd encourage you to check that out for the final three.
Before we end, I do just want to touch again on the ORX Reference Indicator Library. We really do hope that that will be a big help to institutions trying to tackle the issues around their practice. And we are already getting some good feedback from members that have started to do this in terms of how it's helping to supplement their thinking.
As I mentioned at the beginning, this library or reference library is the latest addition to our set of strategically important references and standards we have been developing for the industry. It builds off the ORX Reference Risk Taxonomy and has been developed with that tried and trusted approach that I mentioned is using member data.
Working with an expert partner starts with Tom and his team, Oliver Wyman in this case. And actually, also a very, very supportive member advisory panel. I think remember, and this is a reference, you know, the output should be able to help organisations benchmark or enhance their practice and provide an excellent reference for hopefully some future data sharing and benchmarking across the industry.
I really would like to thank our members that helped us with this work as well, that those that provided data, but really in particular those on the member advisory panel who gave up their time to help us design the approach, but also really importantly to help us challenge the drafts that we were coming up to and give us more industry perspective and insight.
So, before we go, I want to say thank you very much for listening. I hope you found the podcast valuable. Thank you also to the guests, Mike and Tom, thank you very much for your contributions.
Tom: Thanks for having me.
Mike: Thanks very much, everyone.
Steve: And don't forget ORX members get free access to the indicated reference library.
It's on the website, but it's also available for any firms to purchase. You can find out more by going to our website, our www.orx.org and we will also be adding a direct link to the page in the show notes. Thanks again and please tune in next time.
