The PRA has issued a Dear CFO letter giving thematic feedback on written auditor reports for 2019/20. Paul Young looks at regulatory expectations around the key issues of IFRS 9 ECL and model risk management.
Building on last year’s thematic auditor feedback, the letter notes key areas of progress and ongoing concerns around IFRS 9 expected credit loss (ECL) accounting and global benchmark reform, investment in technology and third-party controls. This article focuses on IFRS 9 ECL and model risk management, including areas under heightened scrutiny due to current economic conditions.
Government initiatives such as the Bounce Back Loan Scheme (BBLS), Coronavirus Business Interruption Loan Scheme (CBILS) and the furlough scheme offer much-needed support, but the risk of default on new and existing loans remains high. A significant increase in credit risk (SICR), will move a borrower’s status from Stage 1 to Stage 2, prompting the lender to provision for the lifetime of the loan, in line with IFRS 9 guidance.
Over provisioning can reduce the ability to lend, at a time when it is much needed, but under provisioning can reduce financial stability. Getting the balance right is critical. This highlights the need for robust modelling, with appropriate oversight and the right data inputs.
The Prudential Regulation Authority's (PRA) thematic feedback outlines high-quality practices to improve consistent and explainable IFRS 9 ECL calculations, helping lenders to provision appropriately. The regulator had expected high-quality practices by the end of 2019 but, while firms have made improvements, other areas are still a work in progress, as outlined below.
Model risk management
Firms must all follow the same IFRS 9 ECL method, but that includes a degree of expert judgment. As such, best practice implementation is still evolving, and many are bridging the gap with post-model adjustments (PMA). But PMAs are meant for short-term use, to address a specific issue. For example, where the model has a particular limitation, but the algorithm has not yet been updated to address it.
Last year, PMAs increased ECL calculations by around 5%, generally due to low-modelled provisions for mortgages or similar issues, and firms may factor PMAs into the model in the long term. In the context of the COVID-19 situation, firms use PMAs where the associated economic scenarios are outside the calibrated parameters of most core models.
In practice, many lenders have multiple PMAs that precede coronavirus and have been in place for 12 months or more. Further PMAs introduced in the last six months have created yet more complexity, providing challenges for model risk management and effective governance.
Addressing the root cause will help firms incorporate PMAs into the IFRS 9 ECL model, which is more appropriate for long term use. Root causes and key issues for model risk management include:
Testing and validation
This is essential to identify limitations in the model and remedy them. Many firms do not review all material models contributing to the ECL calculation, reducing the value of validation exercises.
Models draw on a lender’s performance data and use high-quality economic data for calibration. The performance data is often more than six months old. In the context of the coronavirus situation, the data used to calibrate these models may not reflect the current environment and new data may not be available in real-time.
There is an inherent lag in model risk management controls, so good governance and expert judgment will help identify anomalies. Detailed back testing over a wider range of models can provide greater insight to inform these processes.
Governance and monitoring
The current economic environment will affect model performance, so effective governance and monitoring is vital. Firms may improve consistency by reviewing the available metrics and inputs, increasing the frequency of model back-testing and model performance monitoring. PMAs must be subject to the same degree of challenge and rigor as the models themselves.
It is important that sound IFRS 9 ECL model risk management and governance are in place given its inclusion in financial statements, reporting and decision-making processes. The inherent complexity of modelling makes it challenging for those outside the development team to fully understand the mechanism or offer robust challenge.
Aggregated reporting for results of model testing, validation and monitoring would help management assess the significance of model limitations and offer effective challenge over outputs. This will help compare model performance beyond RAG ratings and improve oversight.
From developing the economic scenario to running the model, IFRS 9 ECL processes may be time consuming and resource heavy. Simplified and automated processes will help firms build and maintain up to date, relevant scenarios, with the PRA highlighting key areas for improvement:
Greater use of scenario analyses
Many firms do not routinely carry out sensitivity analyses for every input, but these are essential for understanding the model output, key risk drivers and associated model uncertainty. Firms often weight the scenario for different portfolios instead, but this is too high level. A greater focus on sensitivity analysis will give more options on the type of economic scenarios and model inputs to include.
Severity of scenarios
There is also significant variation between what firms consider as a severe scenario, with the PRA noting that some “severe scenarios were less severe than some peers’ base case forecasts”. Assessing more than one downside scenario is always important, more so in the context of the coronavirus situation. The PRA also stresses the need for scenarios that genuinely test the impact on IFRS 9 ECL, which may not always be clear from outputs. It is also important to assess the impact of multiple scenarios.
Finding mechanisms to effectively capture and reflect uncertainty within the model itself will reduce reliance on PMAs.
Performing retrospective controls such as back-testing more regularly can gather information to update the model or apply an appropriate PMA. Reviewing economic scenarios for plausibility will improve controls, as will checking the model design and calibration against the given parameters.
IFRS 9 Significant increase in credit risk
Significant increase in credit risk (SICR) is a key parameter for IFRS 9 models. This can have a significant impact on model outcome, but it can be tricky to monitor and validate. Firms measure probability of default differently and have varying SICR methodologies, so management may benefit from greater use of industry metrics to benchmark the firm’s SICR approach to facilitate recognition of a SICR on a timely basis.
Key issues include:
Arbitrary or limited thresholds to review SICR
This includes a lack of qualitative indicators and limited collective assessments. Poor SICR methodologies can see a greater number of loans moved to Stage 2, prompting lifetime provisioning, or an insufficient number of loans being moved to Stage 2, so it is crucial that the reasoning is robust, and the inputs are clear and validated.
Lack of consistency
Firms demonstrated significant variance around the percentage of loans in Stage 2, suggesting a wide variety of assumed economic outlooks, in addition to differences in firms’ portfolios.
Identifying all SCIR
IFRS 9 aims to identify all SICR. Collective assessments can help identify increases in risk in-line with the economic outlook or intervention programmes.
Lifetime of an exposure
Lenders demonstrated a range of modelling practices for revolving loan facilities in retail and corporate portfolios. In the modelling approach, firms used either the credit review date as, or the point at which most defaults will have occurred, as an approximation for the lifetime of the loan.
Reviewing these practices and monitoring the effectiveness of measures is essential for good model governance.
Meeting regulatory expectation
In the current macro-economic environment, good model risk management and governance is essential. In turn, this relies on the appropriate investment, with adequate levels of skilled resource. Improving the pace of change and adaptability of models will reduce long term reliance on PMAs, making IFRS 9 ECL model outputs more reliable and explainable in the long term.